From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0054.outbound.protection.outlook.com [65.55.169.54]) by dpdk.org (Postfix) with ESMTP id 5993A3990 for ; Sun, 29 May 2016 18:55:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=okuTcsJPHg6daNiagR/jtWtYjfXQlUHAG94g4lAqN6s=; b=T98bHlfQ+NZZzFCr3ihgNQ1LC7+fBG/Rj8goutbQYBISGQknswt37jiTR3CLaw94iT4AZdc2Evihr+Uy30NLMUuWOYGi6/SQHxMc6Wrv0wiODk3+tXWyIAUKhfP00FYrC+cUWLV1V21kLvYep2Ag9DKJqhoncq5XbilqC4U+8W4= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain.localdomain (122.167.187.184) by BY1PR0701MB1722.namprd07.prod.outlook.com (10.162.111.141) with Microsoft SMTP Server (TLS) id 15.1.501.7; Sun, 29 May 2016 16:55:24 +0000 From: Jerin Jacob To: CC: , , , Jerin Jacob , Maciej Czekaj , Kamil Rytarowski , Zyta Szpak , Slawomir Rosek , Radoslaw Biernacki Date: Sun, 29 May 2016 22:24:22 +0530 Message-ID: <1464540867-13115-5-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1464540867-13115-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1462634198-2289-1-git-send-email-jerin.jacob@caviumnetworks.com> <1464540867-13115-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [122.167.187.184] X-ClientProxiedBy: PN1PR01CA0029.INDPRD01.PROD.OUTLOOK.COM (10.164.137.36) To BY1PR0701MB1722.namprd07.prod.outlook.com (10.162.111.141) X-MS-Office365-Filtering-Correlation-Id: 734cd247-c03e-4e16-b998-08d387e209b5 X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 2:wEeR5qcidLY9ZDWJiyqJ9LKH8lao3rSRNmbfcePCO2YfzWiDhYI77fARmBEvu+cMMGSSpzS0iNLAE8v5JdF5JqH9GVerQtu79udpy5tpDnTYdI0L1OzoU5OrBK9yky+yAy8kvM+dG0nKYuSZhGUp9sa1zXiF3k/jPDsoFcJUzZpZzbpZO53fBrN1EhcxVQI9; 3:feVJ6xmaptq2s0JAnNEH4tqQdXnktwrrlRWYK9gnpmYpNUObsnzidtBY0Ml75LxypJ7NNz5wUqvdDiEy1KbwecPzLMCV7D+2jOKEU9VuJXq9PUP1OzvAcQqix6OqkG3P; 25:acXMom6WdlSJzxZSwzMLr1Ox9l8NRATqbAzbktgIZmrwvYqLbpNHIn4F533bz1YXe6KjzuEUFEFcTB2sx8UMC883q2dGjhux5CNOp0OdOe/vyRyqWZaxC3H0EMEDNeys0NOyWYd+ca4Gau4Z42eB7aV7c+tMqQHrE6Uw2FVRh+G4mFfndSaMpxrhXCai/Hv4GatnGSMNr9C0CekER6F/mVmMWYDhAZge7dzbs4PabqkFn0mESPTwEAQob2dTZfYV7Ucp/KUfNYvD01yIuvSRB1i0SjyXnhLpG8a2T8fiWV+NIOThzkTBuwICSfqfl9YDqEfdzhBS0WXbJCktNcVOWnMsDzSm42Qz+sgr8OTOdg4tR3/UL27Xuncp+4iGD/4J3dO9nDNhwbeU+OA2GHj3Viygz1EIIGcJ//AUxfhIGp0= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR0701MB1722; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 20:5fOEMxslYbrxb3wcJoB4gZ/NfRE3pxqA+UNL4t3InbAtwevuDHVZdPY4Y58jJ1qJRVP2sWJiy6weG1hFtvDwyJDpxwZg5/LQk97y9EHLBZaClNUASroMcPGacJSIC9KlBrk+IEjW016+rNoulLnIuGmN9aeJvQMQbaLMEs7jAgBK54owY1K6rDiizwlNjQ6P0lQBogSAtGLcjeXmdS/26nwFEyYlvo4wNskZfaLX1DcOt/FUZcE00FKvQUoWUXZRFABGIapULK+LPCtoEjE8sZmUp1ZPP8smyJW5A5qMU6GMWaJw1u2UZ1/EGa8DUa7DoblVbAXZDZ7x4Qkzc4Uqx1r+U3H/juF5+MFpSw7A6BxqSHtt8GSTF8ujt0P7y0G/wfjNSHM1XBvokWpmAlMdfhHyxL0gXcsDhaittHbffQSsiFBH8ykDfpcMr6UP7wnF+t3eIj+Aabpu2Fs6AftKWuP+n+908ohJQLvtc1XbefT/sHH0V0j7E3e6iA2OsNPdRIT1jbC4WunuAhIQw+0VioxFPjGiycPcYXkyoOfc7xbPhdRrSN3gvOjXT58MKvQwTx+85Zw11GX95yqO+7o+AyvXKZIeTxyV4Un7W65F0wA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:BY1PR0701MB1722; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0701MB1722; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 4:nGzo11THJnDWEQoBkDdvr4WFDdWlcKKR7xJ2O/NZJ7O5bsTZfe7z9Fqujb3COLSRbuSAs1eaNQnnA3X30eTlqgISySk0tMYwadbOUix7W2YwbKDJwgk+sNl+2lfItPf8HhoE1Qr8Fb2u7wASmPvzf6kds+yoJul+WBJn5wtgQN+VoBKqfOcq4bKgUf0zLufbvFFrZsusGK7Ph5WyJzoUJSby169SvuLqezZ9sfcItgKJl8ri10WOcGKyE+YXZzAn+T0BarAUZQQCsxFs9vlyaVbE6ZN5g7lwpfBnMYJ342dmIpiL4k9YnC6CAZEhVvqTl9lz9F+d6/N7aSxYupnTKflcP7KPJ1qjhEhUFwd2DFG/ZghaPBwjYe+Fb4ZaENyd X-Forefront-PRVS: 0957AD37A0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(6069001)(33646002)(42186005)(50466002)(92566002)(50226002)(229853001)(2351001)(5003940100001)(189998001)(2950100001)(110136002)(77096005)(36756003)(47776003)(5004730100002)(48376002)(575784001)(2906002)(586003)(8676002)(5008740100001)(19580395003)(66066001)(4326007)(76176999)(81166006)(3846002)(6116002)(50986999)(19580405001)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0701MB1722; H:localhost.localdomain.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 23:32Mxl1OBZIopG8e58TqBR2j2KMhgbr/s833V+CQTyi+JzoOlYANuAnWxx6yTbK9IIYBM7CIe2Kp6CsDrkMOiy5j339FTnOWwfhpDtkmG0gld7IqEuSqOF+sRdSOadutQZCGH44AVOOPFC6OWPjTqsRl6r2iQfb/8ZVZTwv5OoAmgdjISDF04QWDGcqcdaXOzMEDeIfLsD8LEqSn662j2tqL7Y79fDgBlpDdORRg1G1/DXPbfrpA+kEvnwPrgYBgLZtvhz82pyOJ5f9tFC/evIRoY68LYezAgRs3D9OKXeGWRR5TnEqXfZRdVXgNSYAXBzc1LvbPIyZWVyX5//fTqMM+7Iqip3VqHUijjH892Uc0itIs+FKVGiV044B4PwgPPcBCaxyGpwLTY2Dc9DX9n6JuAwo84MkSWx1Fkl294pm25VL+bCPX1t5yYPF03xgUTrfCDZMltNlYnaQRTVDJO7rUwbrkrfewCr5B+WSSZhtXHXVVogDD86WxV/Vyk8c0XFCYZtzbPvmRL3EdMneeChIoTK7bgT96iR/ec3sF7vk9eZ93BmOIUaRe2IARwGaJCo4sQhTQ++w+CFdSDwLO4l6Ax/NEgQBfcVAlLuv36rTVXVoFyYWOMrrlESllavHEJIiYRj/nF4ROpk5PkwZRlo4jup+0W5aiR2yAmCLZJ0wNmKBAPKSFcYjsXDSPMnlTcWnThuwE0AWRO8CDWxkjtawSfBlBPjUdKAPpDBao9Ow+NJvJILsmCdiCK1YSrxR3P8U7xDhtqtgGUKdwuqp3+4bQn4jxjMpiWnzbOE0yaoWIVo0LrCodpbkMMLF9+Wbun/XMAcTQB8WCOcLSo+1ku+NCZTXyWeyT6inMOWFM9gXpXJ0bHKSAtARysHLOmvkPgz/IA+ol8d5Wa/A9OKZ3PrQ== X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 5:4hN3gLjPce3EFN562B13jfhLnAlbBudI3NVZB134rGhRZ9ULoUIEwcfKmYDmN5Dv5iw58NU6DrKmPy70nZbjMFMKkpwmuFxXGia7EVH1JJmUPV0XYrkYQ1g0DMk/kysKl0YZEAGr2e1UjqNozsYDOA==; 24:KtTVq5ck6+KGUmzma3lu/wpkq28+U8AsBYgKOOw10iUhdMoE6cELlchhp86nkBJF4ADLbA7o1kCZCesNkUUtw3y4KaWwQJ80slg2yzEE+OM=; 7:RatDHehhZ9paUKkIBVDc56k51dV6db1v5w+Kl1cN5mhaUCWSzf2TuYQqxZJgLKNX1NhNy2iCaFDo9Yj/td9quC+jBb3DKlJR4T9hmzlaDet6tlKNlJ60IGfNcX3ld3gH9Q9hfA9eXt9d31VGozkaUU++hR04SqlVKFedO2MMcf/TGOg9A56k+aT6g3/ClX7A SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 May 2016 16:55:24.9790 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0701MB1722 Subject: [dpdk-dev] [PATCH v2 15/20] thunderx/nicvf: add rx queue start and stop support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 May 2016 16:55:32 -0000 Signed-off-by: Jerin Jacob Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki --- drivers/net/thunderx/nicvf_ethdev.c | 175 ++++++++++++++++++++++++++++++++++++ drivers/net/thunderx/nicvf_rxtx.c | 18 ++++ drivers/net/thunderx/nicvf_rxtx.h | 1 + 3 files changed, 194 insertions(+) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 56e0b5c..6a3c01b 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -88,6 +88,8 @@ static int nicvf_dev_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf); static int nicvf_dev_rss_hash_conf_get(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf); +static int nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx); +static int nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx); static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, @@ -616,6 +618,54 @@ nicvf_tx_queue_reset(struct nicvf_txq *txq) txq->xmit_bufs = 0; } + +static inline int +nicvf_configure_cpi(struct rte_eth_dev *dev) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + uint16_t qidx, qcnt; + int ret; + + /* Count started rx queues */ + for (qidx = qcnt = 0; qidx < nic->eth_dev->data->nb_rx_queues; qidx++) + if (dev->data->rx_queue_state[qidx] == + RTE_ETH_QUEUE_STATE_STARTED) + qcnt++; + + nic->cpi_alg = CPI_ALG_NONE; + ret = nicvf_mbox_config_cpi(nic, qcnt); + if (ret) + PMD_INIT_LOG(ERR, "Failed to configure CPI %d", ret); + + return ret; +} + +static int +nicvf_configure_rss_reta(struct rte_eth_dev *dev) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + unsigned int idx, qmap_size; + uint8_t qmap[RTE_MAX_QUEUES_PER_PORT]; + uint8_t default_reta[NIC_MAX_RSS_IDR_TBL_SIZE]; + + if (nic->cpi_alg != CPI_ALG_NONE) + return -EINVAL; + + /* Prepare queue map */ + for (idx = 0, qmap_size = 0; idx < dev->data->nb_rx_queues; idx++) { + if (dev->data->rx_queue_state[idx] == + RTE_ETH_QUEUE_STATE_STARTED) + qmap[qmap_size++] = idx; + } + + /* Update default RSS RETA */ + for (idx = 0; idx < NIC_MAX_RSS_IDR_TBL_SIZE; idx++) + default_reta[idx] = qmap[idx % qmap_size]; + + return nicvf_rss_reta_update(nic, default_reta, + NIC_MAX_RSS_IDR_TBL_SIZE); +} + static void nicvf_dev_tx_queue_release(void *sq) { @@ -741,6 +791,33 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, return 0; } +static inline void +nicvf_rx_queue_release_mbufs(struct nicvf_rxq *rxq) +{ + uint32_t rxq_cnt; + uint32_t nb_pkts, released_pkts = 0; + uint32_t refill_cnt = 0; + struct rte_eth_dev *dev = rxq->nic->eth_dev; + struct rte_mbuf *rx_pkts[NICVF_MAX_RX_FREE_THRESH]; + + if (dev->rx_pkt_burst == NULL) + return; + + while ((rxq_cnt = nicvf_dev_rx_queue_count(dev, rxq->queue_id))) { + nb_pkts = dev->rx_pkt_burst(rxq, rx_pkts, + NICVF_MAX_RX_FREE_THRESH); + PMD_DRV_LOG(INFO, "nb_pkts=%d rxq_cnt=%d", nb_pkts, rxq_cnt); + while (nb_pkts) { + rte_pktmbuf_free_seg(rx_pkts[--nb_pkts]); + released_pkts++; + } + } + + refill_cnt += nicvf_dev_rbdr_refill(dev, rxq->queue_id); + PMD_DRV_LOG(INFO, "free_cnt=%d refill_cnt=%d", + released_pkts, refill_cnt); +} + static void nicvf_rx_queue_reset(struct nicvf_rxq *rxq) { @@ -749,6 +826,69 @@ nicvf_rx_queue_reset(struct nicvf_rxq *rxq) rxq->recv_buffers = 0; } +static inline int +nicvf_start_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + struct nicvf_rxq *rxq; + int ret; + + if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + /* Update rbdr pointer to all rxq */ + rxq = dev->data->rx_queues[qidx]; + rxq->shared_rbdr = nic->rbdr; + + ret = nicvf_qset_rq_config(nic, qidx, rxq); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to configure rq %d %d", qidx, ret); + goto config_rq_error; + } + ret = nicvf_qset_cq_config(nic, qidx, rxq); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to configure cq %d %d", qidx, ret); + goto config_cq_error; + } + + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + return 0; + +config_cq_error: + nicvf_qset_cq_reclaim(nic, qidx); +config_rq_error: + nicvf_qset_rq_reclaim(nic, qidx); + return ret; +} + +static inline int +nicvf_stop_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + struct nicvf_rxq *rxq; + int ret, other_error; + + if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + ret = nicvf_qset_rq_reclaim(nic, qidx); + if (ret) + PMD_INIT_LOG(ERR, "Failed to reclaim rq %d %d", qidx, ret); + + other_error = ret; + rxq = dev->data->rx_queues[qidx]; + nicvf_rx_queue_release_mbufs(rxq); + nicvf_rx_queue_reset(rxq); + + ret = nicvf_qset_cq_reclaim(nic, qidx); + if (ret) + PMD_INIT_LOG(ERR, "Failed to reclaim cq %d %d", qidx, ret); + + other_error |= ret; + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + return other_error; +} + static void nicvf_dev_rx_queue_release(void *rx_queue) { @@ -761,6 +901,39 @@ nicvf_dev_rx_queue_release(void *rx_queue) } static int +nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) +{ + int ret; + + if (qidx >= nicvf_pmd_priv(dev)->eth_dev->data->nb_rx_queues) + return -EINVAL; + + ret = nicvf_start_rx_queue(dev, qidx); + if (ret) + return ret; + + ret = nicvf_configure_cpi(dev); + if (ret) + return ret; + + return nicvf_configure_rss_reta(dev); +} + +static int +nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx) +{ + int ret; + + if (qidx >= nicvf_pmd_priv(dev)->eth_dev->data->nb_rx_queues) + return -EINVAL; + + ret = nicvf_stop_rx_queue(dev, qidx); + ret |= nicvf_configure_cpi(dev); + ret |= nicvf_configure_rss_reta(dev); + return ret; +} + +static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, @@ -987,6 +1160,8 @@ static const struct eth_dev_ops nicvf_eth_dev_ops = { .reta_query = nicvf_dev_reta_query, .rss_hash_update = nicvf_dev_rss_hash_update, .rss_hash_conf_get = nicvf_dev_rss_hash_conf_get, + .rx_queue_start = nicvf_dev_rx_queue_start, + .rx_queue_stop = nicvf_dev_rx_queue_stop, .rx_queue_setup = nicvf_dev_rx_queue_setup, .rx_queue_release = nicvf_dev_rx_queue_release, .rx_queue_count = nicvf_dev_rx_queue_count, diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 8031685..e8c605d 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -580,3 +580,21 @@ nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx) rxq = (struct nicvf_rxq *)dev->data->rx_queues[queue_idx]; return nicvf_addr_read(rxq->cq_status) & NICVF_CQ_CQE_COUNT_MASK; } + +uint32_t +nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx) +{ + struct nicvf_rxq *rxq; + uint32_t to_process; + uint32_t rx_free; + + rxq = (struct nicvf_rxq *)dev->data->rx_queues[queue_idx]; + to_process = rxq->recv_buffers; + while (rxq->recv_buffers > 0) { + rx_free = RTE_MIN(rxq->recv_buffers, NICVF_MAX_RX_FREE_THRESH); + rxq->recv_buffers -= nicvf_fill_rbdr(rxq, rx_free); + } + + assert(rxq->recv_buffers == 0); + return to_process; +} diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h index 44cef06..3484928 100644 --- a/drivers/net/thunderx/nicvf_rxtx.h +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -85,6 +85,7 @@ fill_sq_desc_gather(union sq_entry_t *entry, struct rte_mbuf *pkt) #endif uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx); +uint32_t nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx); uint16_t nicvf_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts, uint16_t pkts); uint16_t nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, -- 2.5.5