From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <Jerin.Jacob@cavium.com>
Received: from na01-by2-obe.outbound.protection.outlook.com
 (mail-by2on0061.outbound.protection.outlook.com [207.46.100.61])
 by dpdk.org (Postfix) with ESMTP id BC04E5AB5
 for <dev@dpdk.org>; Sat,  7 May 2016 17:18:57 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;
 h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=dpFnuQ8xRqojCUj21CGcOb6CyWGyOYW1HjlkjIlXPN8=;
 b=h0vfISnW4+KnAlwxpG/0Zy5ySVlV+mK6DTy726cJy9Q9xKyWqDaGkoUf8L9ehrfLvXyB6T/GjjVTF3C5WLDu8iDrG5A0i0H2AVTrlAEK9hnFdWnRxxJy6J38fwefxgp8agCG49CNYCjVP/01Lqrp5CvPhi4Pp3/lh2ZMgDW5VBA=
Authentication-Results: dpdk.org; dkim=none (message not signed)
 header.d=none;dpdk.org; dmarc=none action=none
 header.from=caviumnetworks.com;
Received: from localhost.localdomain.localdomain (122.167.10.92) by
 BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17) with Microsoft SMTP
 Server (TLS) id 15.1.492.11; Sat, 7 May 2016 15:18:52 +0000
From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: <dev@dpdk.org>
CC: <thomas.monjalon@6wind.com>, <bruce.richardson@intel.com>, Jerin Jacob
 <jerin.jacob@caviumnetworks.com>, Maciej Czekaj
 <maciej.czekaj@caviumnetworks.com>, Kamil Rytarowski
 <Kamil.Rytarowski@caviumnetworks.com>, Zyta Szpak <zyta.szpak@semihalf.com>,
 Slawomir Rosek <slawomir.rosek@semihalf.com>, Radoslaw Biernacki
 <rad@semihalf.com>
Date: Sat, 7 May 2016 20:46:33 +0530
Message-ID: <1462634198-2289-16-git-send-email-jerin.jacob@caviumnetworks.com>
X-Mailer: git-send-email 2.1.0
In-Reply-To: <1462634198-2289-1-git-send-email-jerin.jacob@caviumnetworks.com>
References: <1462634198-2289-1-git-send-email-jerin.jacob@caviumnetworks.com>
MIME-Version: 1.0
Content-Type: text/plain
X-Originating-IP: [122.167.10.92]
X-ClientProxiedBy: PN1PR01CA0050.INDPRD01.PROD.OUTLOOK.COM (10.164.136.150) To
 BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17)
X-MS-Office365-Filtering-Correlation-Id: aa3b61ba-5f66-45a4-bead-08d3768ae862
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718;
 2:JPGjjIITcvYWfXJ0hCnJKimCR4GPesLnFBXPaIGK8zxOyUsnWnKni78uE7lcJ6n8UTWvIz2+2H24AhjRnpM+3LmNdRY7bOEElSBO4TREAZjXB36qArklk1YUY2rjIQWrgP50IrNTihi7TsAO2DVpAf0uaxsQaHzi7eb6nKbg4RzxeeCe50lHvT87Sxoh4GCT;
 3:0eGMKP/H7JRzrhCSk98m761HAZiZGKBJRIkrqabkST56aBcJYID4sQ6qEsdtStWB6IGczrk4GwipPcxibNQS5TFD3HEyHKC7Zn6XP05R9KyEW+8uWy4SdS22AKJSG7SH;
 25:JIf2ZuqEKilauuM5t7NE7Z70f+Y7LAnbx4TMp54mddKh6frLsiI2/xsArwE9FkW1GiEsEixnEv/0Hx1JAleZ/4/0PgfXMKuVl0rH4gfNKIchU8DHsh1aQzBBFVXM9Nz17Y3D1O1LWqPT+j/2mCpvE7j97Nl5iXP5XHPWhrx6y8HbtP3qV6pLJsrJJabJyEgVOWKpa8UHMu1ocMTsPKpd1oRcyXQcNWECFashItMF6HbZNo87At8Au1Ooui3avipS/4IulW7W5l0YByYmO/eBcOVmceQIFNG+ZHxcKY+19lc3DQIp3ihbOsVQu/19eJXo9E1egUvyDsEfotqzjkCUFuR076ZFNwu0fgoKSqGwc7I9slvQW2jlmekajofeioYUfX7nzPIrxe7BAbiQChxzEQ==
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1718;
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718;
 20:VgGEbMJ0J2Ox8ZOrbNRoz049mkhjOHfypGG/5Q/WI33SME8mia2hnwFIY25dMqgH6JoeKlW3iN9dEXyKx+biaxEIP166lD6ra/EDiJXL+UGb1xAX8Y0lCunSa09rFoenrHO7wCE5AHK+Jt8mpIWIikK+l5fkwLbFXapeoWijTC8vL9J5MXIIwB242SY4rce0RFYMofakDpftnmZNlbl8C0OywQ0QN00LPLnroclLiecjRFQOYZ1A7TDtvmfNzk7AW75WO21z/y58Jerngf0HgbinhfE3xj04LJRQYCTtrIcDipXTOrkbYZE9t0Y7KtXSRdPw/jJ/0fbb9077peNYY28wOVzDXuzQ6shnjHjtOmnMe991pK9gUgnbJAjaWZxBZJJia1inm2bIMuIwFmq0LKvocb23hcKWuGI5Odhbq62DcCNxgo1MC//dKgCrz+cGOvsbSo3/uf/Htc9G/lNSwyk3i4kR756IzWda/hvczSBCcb8Nptdwami3yICw7jzqOny674hMPH5LL+84eYEbFvPSvBxy5wx2hBkSafwWXiZgDhuA6bai5Z2Cg5+sn2R2Jndg319eE5TUezRxgdvHMq+DOFiN5iGPoI1UKtg90xw=
X-Microsoft-Antispam-PRVS: <BN3PR0701MB17181CBBDEB37E6EF19A8B62817E0@BN3PR0701MB1718.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:;
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0;
 RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046);
 SRVR:BN3PR0701MB1718; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1718; 
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718;
 4:OK00cjc+cEjH8SndR3cMLG/JdPjFQxNTwAS6cAQwQoFsveKUB08fo3Hf4DoerNZNHPzAwtqHS9AmHa8Z4hWMyVxK0Atzlg8HVliKMvdocWrPOB2poDZLUgAGsyIY0ZXxDzD6+qQNj+0E6mqDS0BXWKFYd+qhj7mCsgyCcTkmfiZtNg80+Sq4bW3Qu0LinSpr6lvWeHuI1QKq4+TTHBkKYPRiRKXXMdhmSSh1e8XJDBQZDLZo2UV7twr1sg3pIpiwTbInSRK5pBbNMwBot/Q/Ic6GvfbYpDWMOKSIX1mlRGKCurT/frwjfqGc8rAg7N/cZdd4LAlfQ0pE5HAh5VwvEtKJMRAX5hQm3u3NDqQp/zfC1xkA1SLxqX4P25lq30YY
X-Forefront-PRVS: 09352FD734
X-Forefront-Antispam-Report: SFV:NSPM;
 SFS:(10009020)(4630300001)(6009001)(6069001)(50226002)(77096005)(2950100001)(48376002)(81166005)(50466002)(66066001)(47776003)(92566002)(5003940100001)(4326007)(2906002)(110136002)(50986999)(2351001)(33646002)(586003)(3846002)(6116002)(229853001)(5004730100002)(76176999)(19580395003)(19580405001)(189998001)(5008740100001)(42186005)(36756003)(7099028);
 DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1718;
 H:localhost.localdomain.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; 
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718;
 23:woYC7OP6idxNJOfTdBtRmsmvL1gBs7XkmEGjaFuvW/2YlSGvJ6ZyDV705wO7WS1V+jHmAPXwjRkCvZ4SOndphBsshgPgllc9tOn1htRIxD88rrJs5FVGtDJ4RHgoyq5No0ahJjXUzE4lsue+lZbsGcGVUsk1VZ/DsVmoWd9M1u19no+OcU8D5qxDBpFpiENJYKVLbQteZEJ/zK8euXA1oz2OkJaXwEjNc1lW+B8OwJTVsBCSdUp0QE/a+LIMBCvJiv9Gj7QivXq9vMjjzXfS3IQZ/qvNMsDDTJDjQlXG3VEaeslcDr+0lhpOeyFitBjDEZjbZZvQ9RKA9JmyZdtL+bUa9XaEERG+5EEo3dTvUpH0eHaS8EgyvymZvViqndgDN+XLFfzWfs3k8RBrWXYK3qVVgg2fjbQ2jDM0IRmHwtThgC/v1YPusKTla1BxCXCaAvBCYb3WusB5KNRh6NiSS1CeBT1jsvEvHFwxJ3La0XRJB8fnz4Q5k4RZrSsTpsU9HunLiK5f5GRSuwPGWn0l0s+hHHiaratVWtDTQyd8FB1bczcGfii96hq89ho8f0T9fOrjoI0Q1YZTZPrK85Xnif2gKvKk0DlHiFYQOqvU2e3N+3UzHNchxA1KcKHRIsCLc7TP42ThtGnUl3kVEj+NskGyC2dHholMWiCoydQm1OWS/gUQTdTnKj9d0Z81kCMcY3NbZEF/wPDzt8hXjLmp/Tbiy0IdJigvkvef7anzTofzuWtQ0F4cZyTcPioJOWBVLo0tHTu7gZ/1axMJEvClxYm7j2+OxyfDlsBMh9gfGVe66Zr5TCFoxSKCO6Zr3zwrS4bgEPiOIGYUiPfzajLNQ9hOe/uF/xnvw/jfuFVCfp5DhNOj7aUbFeBlUx1ih1/O
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718;
 5:LdX32M9Tvbdv/lanTQwF76OWJ/A9x/dZOvujeDmF6fW9aJyLGril5HlxqYWfAoAe3LJJX/netiJX5Onk+/NAOIP+/9Ewo+XI2i8WR/P6/lGp1ueqgZ8sLyJx5CikiqVy35jpaMj6oyIqrRBkKgMRTQ==;
 24:tL4nbHDZ3akQn07Lvk2C+HhJBymwRXrsmkS5UI1D5ueG2aVaSjStE1lcSqieuYkqNhvA47YGHuSo8zl7gIiIw0HhmouOYTOG4nb1Ppj4f7U=;
 7:CdjuztHxAba1GRBltUvL4Tyhrtc6b3a/8WqqN0CXFUHCrqQILPIRBuudLI8C2M5cvPkdbFczZwxfnqjv1av8kOIjGvnT+IoZS37KJUrJWUQzY65V/O93z9p4YPCilUQs6JFq4olYMTJKZnmNDmAAS/5vexXvcWYyCU3MGUr2ANFFz9ztKykTA6rrtULT134k
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: caviumnetworks.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2016 15:18:52.2282 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1718
Subject: [dpdk-dev] [PATCH 15/20] thunderx/nicvf: add rx queue start and
	stop support
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Sat, 07 May 2016 15:18:58 -0000

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
Signed-off-by: Kamil Rytarowski <Kamil.Rytarowski@caviumnetworks.com>
Signed-off-by: Zyta Szpak <zyta.szpak@semihalf.com>
Signed-off-by: Slawomir Rosek <slawomir.rosek@semihalf.com>
Signed-off-by: Radoslaw Biernacki <rad@semihalf.com>
---
 drivers/net/thunderx/nicvf_ethdev.c | 174 ++++++++++++++++++++++++++++++++++++
 drivers/net/thunderx/nicvf_rxtx.c   |  18 ++++
 drivers/net/thunderx/nicvf_rxtx.h   |   1 +
 3 files changed, 193 insertions(+)

diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c
index 0c72201..9b917c1 100644
--- a/drivers/net/thunderx/nicvf_ethdev.c
+++ b/drivers/net/thunderx/nicvf_ethdev.c
@@ -88,6 +88,8 @@ static int nicvf_dev_rss_hash_update(struct rte_eth_dev *dev,
 				     struct rte_eth_rss_conf *rss_conf);
 static int nicvf_dev_rss_hash_conf_get(struct rte_eth_dev *dev,
 				       struct rte_eth_rss_conf *rss_conf);
+static int nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx);
+static int nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx);
 static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx,
 				    uint16_t nb_desc, unsigned int socket_id,
 				    const struct rte_eth_rxconf *rx_conf,
@@ -616,6 +618,54 @@ nicvf_tx_queue_reset(struct nicvf_txq *txq)
 	txq->xmit_bufs = 0;
 }
 
+
+static inline int
+nicvf_configure_cpi(struct rte_eth_dev *dev)
+{
+	struct nicvf *nic = nicvf_pmd_priv(dev);
+	uint16_t qidx, qcnt;
+	int ret;
+
+	/* Count started rx queues */
+	for (qidx = qcnt = 0; qidx < nic->eth_dev->data->nb_rx_queues; qidx++)
+		if (dev->data->rx_queue_state[qidx] ==
+		    RTE_ETH_QUEUE_STATE_STARTED)
+			qcnt++;
+
+	nic->cpi_alg = CPI_ALG_NONE;
+	ret = nicvf_mbox_config_cpi(nic, qcnt);
+	if (ret)
+		PMD_INIT_LOG(ERR, "Failed to configure CPI %d", ret);
+
+	return ret;
+}
+
+static int
+nicvf_configure_rss_reta(struct rte_eth_dev *dev)
+{
+	struct nicvf *nic = nicvf_pmd_priv(dev);
+	unsigned int idx, qmap_size;
+	uint8_t qmap[RTE_MAX_QUEUES_PER_PORT];
+	uint8_t default_reta[NIC_MAX_RSS_IDR_TBL_SIZE];
+
+	if (nic->cpi_alg != CPI_ALG_NONE)
+		return -EINVAL;
+
+	/* Prepare queue map */
+	for (idx = 0, qmap_size = 0; idx < dev->data->nb_rx_queues; idx++) {
+		if (dev->data->rx_queue_state[idx] ==
+				RTE_ETH_QUEUE_STATE_STARTED)
+			qmap[qmap_size++] = idx;
+	}
+
+	/* Update default RSS RETA */
+	for (idx = 0; idx < NIC_MAX_RSS_IDR_TBL_SIZE; idx++)
+		default_reta[idx] = qmap[idx % qmap_size];
+
+	return nicvf_rss_reta_update(nic, default_reta,
+				     NIC_MAX_RSS_IDR_TBL_SIZE);
+}
+
 static void
 nicvf_dev_tx_queue_release(void *sq)
 {
@@ -738,6 +788,32 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx,
 	return 0;
 }
 
+static inline void
+nicvf_rx_queue_release_mbufs(struct nicvf_rxq *rxq)
+{
+	uint32_t rxq_cnt;
+	uint32_t nb_pkts, released_pkts = 0;
+	uint32_t refill_cnt = 0;
+	struct rte_eth_dev *dev = rxq->nic->eth_dev;
+	struct rte_mbuf *rx_pkts[MAX_RX_FREE_THRESH];
+
+	if (dev->rx_pkt_burst == NULL)
+		return;
+
+	while ((rxq_cnt = nicvf_dev_rx_queue_count(dev, rxq->queue_id))) {
+		nb_pkts = dev->rx_pkt_burst(rxq, rx_pkts, MAX_RX_FREE_THRESH);
+		PMD_DRV_LOG(INFO, "nb_pkts=%d  rxq_cnt=%d", nb_pkts, rxq_cnt);
+		while (nb_pkts) {
+			rte_pktmbuf_free_seg(rx_pkts[--nb_pkts]);
+			released_pkts++;
+		}
+	}
+
+	refill_cnt += nicvf_dev_rbdr_refill(dev, rxq->queue_id);
+	PMD_DRV_LOG(INFO, "free_cnt=%d  refill_cnt=%d",
+		    released_pkts, refill_cnt);
+}
+
 static void
 nicvf_rx_queue_reset(struct nicvf_rxq *rxq)
 {
@@ -746,6 +822,69 @@ nicvf_rx_queue_reset(struct nicvf_rxq *rxq)
 	rxq->recv_buffers = 0;
 }
 
+static inline int
+nicvf_start_rx_queue(struct rte_eth_dev *dev, uint16_t qidx)
+{
+	struct nicvf *nic = nicvf_pmd_priv(dev);
+	struct nicvf_rxq *rxq;
+	int ret;
+
+	if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED)
+		return 0;
+
+	/* Update rbdr pointer to all rxq */
+	rxq = dev->data->rx_queues[qidx];
+	rxq->shared_rbdr = nic->rbdr;
+
+	ret = nicvf_qset_rq_config(nic, qidx, rxq);
+	if (ret) {
+		PMD_INIT_LOG(ERR, "Failed to configure rq %d %d", qidx, ret);
+		goto config_rq_error;
+	}
+	ret = nicvf_qset_cq_config(nic, qidx, rxq);
+	if (ret) {
+		PMD_INIT_LOG(ERR, "Failed to configure cq %d %d", qidx, ret);
+		goto config_cq_error;
+	}
+
+	dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED;
+	return 0;
+
+config_cq_error:
+	nicvf_qset_cq_reclaim(nic, qidx);
+config_rq_error:
+	nicvf_qset_rq_reclaim(nic, qidx);
+	return ret;
+}
+
+static inline int
+nicvf_stop_rx_queue(struct rte_eth_dev *dev, uint16_t qidx)
+{
+	struct nicvf *nic = nicvf_pmd_priv(dev);
+	struct nicvf_rxq *rxq;
+	int ret, other_error;
+
+	if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STOPPED)
+		return 0;
+
+	ret = nicvf_qset_rq_reclaim(nic, qidx);
+	if (ret)
+		PMD_INIT_LOG(ERR, "Failed to reclaim rq %d %d", qidx, ret);
+
+	other_error = ret;
+	rxq = dev->data->rx_queues[qidx];
+	nicvf_rx_queue_release_mbufs(rxq);
+	nicvf_rx_queue_reset(rxq);
+
+	ret = nicvf_qset_cq_reclaim(nic, qidx);
+	if (ret)
+		PMD_INIT_LOG(ERR, "Failed to reclaim cq %d %d", qidx, ret);
+
+	other_error |= ret;
+	dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED;
+	return other_error;
+}
+
 static void
 nicvf_dev_rx_queue_release(void *rx_queue)
 {
@@ -758,6 +897,39 @@ nicvf_dev_rx_queue_release(void *rx_queue)
 }
 
 static int
+nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx)
+{
+	int ret;
+
+	if (qidx >= nicvf_pmd_priv(dev)->eth_dev->data->nb_rx_queues)
+		return -EINVAL;
+
+	ret = nicvf_start_rx_queue(dev, qidx);
+	if (ret)
+		return ret;
+
+	ret = nicvf_configure_cpi(dev);
+	if (ret)
+		return ret;
+
+	return nicvf_configure_rss_reta(dev);
+}
+
+static int
+nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx)
+{
+	int ret;
+
+	if (qidx >= nicvf_pmd_priv(dev)->eth_dev->data->nb_rx_queues)
+		return -EINVAL;
+
+	ret = nicvf_stop_rx_queue(dev, qidx);
+	ret |= nicvf_configure_cpi(dev);
+	ret |= nicvf_configure_rss_reta(dev);
+	return ret;
+}
+
+static int
 nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx,
 			 uint16_t nb_desc, unsigned int socket_id,
 			 const struct rte_eth_rxconf *rx_conf,
@@ -984,6 +1156,8 @@ static const struct eth_dev_ops nicvf_eth_dev_ops = {
 	.reta_query               = nicvf_dev_reta_query,
 	.rss_hash_update          = nicvf_dev_rss_hash_update,
 	.rss_hash_conf_get        = nicvf_dev_rss_hash_conf_get,
+	.rx_queue_start           = nicvf_dev_rx_queue_start,
+	.rx_queue_stop            = nicvf_dev_rx_queue_stop,
 	.rx_queue_setup           = nicvf_dev_rx_queue_setup,
 	.rx_queue_release         = nicvf_dev_rx_queue_release,
 	.rx_queue_count           = nicvf_dev_rx_queue_count,
diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c
index 27e5e1c..7724054 100644
--- a/drivers/net/thunderx/nicvf_rxtx.c
+++ b/drivers/net/thunderx/nicvf_rxtx.c
@@ -604,3 +604,21 @@ nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx)
 	rxq = (struct nicvf_rxq *)dev->data->rx_queues[queue_idx];
 	return nicvf_addr_read(rxq->cq_status) & NICVF_CQ_CQE_COUNT_MASK;
 }
+
+uint32_t
+nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx)
+{
+	struct nicvf_rxq *rxq;
+	uint32_t to_process;
+	uint32_t rx_free;
+
+	rxq = (struct nicvf_rxq *)dev->data->rx_queues[queue_idx];
+	to_process = rxq->recv_buffers;
+	while (rxq->recv_buffers > 0) {
+		rx_free = RTE_MIN(rxq->recv_buffers, MAX_RX_FREE_THRESH);
+		rxq->recv_buffers -= nicvf_fill_rbdr(rxq, rx_free);
+	}
+
+	assert(rxq->recv_buffers == 0);
+	return to_process;
+}
diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h
index 10f4bf9..d2f57bb 100644
--- a/drivers/net/thunderx/nicvf_rxtx.h
+++ b/drivers/net/thunderx/nicvf_rxtx.h
@@ -53,6 +53,7 @@ nicvf_frag_num(uint16_t i)
 }
 
 uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx);
+uint32_t nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx);
 
 uint16_t nicvf_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts, uint16_t pkts);
 uint16_t nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts,
-- 
2.1.0