From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0065.outbound.protection.outlook.com [157.56.111.65]) by dpdk.org (Postfix) with ESMTP id 48A16D0DF for ; Fri, 17 Jun 2016 15:32:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=zv8C4/zIcidDT6wuB6guajYCUNSmOurvvXIfwZ916+o=; b=ioTIBGtwtNx1NazXxN5gnuFs6xk4WGrfVo5b+2boRbqrX5zuACXE8oV/Sns1kbq8SeXzxf3l+tX70CRrDuN7Yfnwg5NS9uV8siGciV/nweb//yK/J8wF/J7opKbdZnkxqeh+r9IEtzdm/ZHaFA5WU3/IgD6PS5x1fuTEQZxDQJ4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain.localdomain (171.48.17.70) by BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17) with Microsoft SMTP Server (TLS) id 15.1.517.8; Fri, 17 Jun 2016 13:32:41 +0000 From: Jerin Jacob To: CC: , , , Jerin Jacob , Maciej Czekaj , Kamil Rytarowski , Zyta Szpak , Slawomir Rosek , Radoslaw Biernacki Date: Fri, 17 Jun 2016 18:59:50 +0530 Message-ID: <1466170194-28393-24-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1466170194-28393-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1465931200-23912-1-git-send-email-jerin.jacob@caviumnetworks.com> <1466170194-28393-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [171.48.17.70] X-ClientProxiedBy: BM1PR01CA0028.INDPRD01.PROD.OUTLOOK.COM (10.163.198.163) To BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17) X-MS-Office365-Filtering-Correlation-Id: 7a394d2f-9efb-48d5-8008-08d396b3de16 X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 2:FzinHZDBcSYmvHW9osU8u9lBgEQzTTeI7pafqeH5whvZ2q+UANWoVho9HWJUglyf2yeyASsL1oWxAA90rXfRf09kZTDYbCCoAgsPjcbCmx8A/HSQLNJzV4H5/lhw8oqPPih07diCWCB3w52dtWAj/m92PpKK7+3wuQutEfxxumPmyJmdx5PSqsY85eDGzmyC; 3:SzVivMKbif72yndDoElZ8ewS7aO5BhuWO5dW0vZG1aRHbluMp8heHQuuIkMmaez9dE4koSELQn1O4pIIfv3h5rBQQ6bPIV/0BqoQec9kOrKcuTD8dogSqXdQR+0mgfEp; 25:4pAdg1mOV2Nh3aOjHg37Gq4wqsT0tEeBYwdQB4GRH7znqbS4MXudDMny76pXS/kxAW5IFIkULP/H9NtcWbI3yfifQjaxuXGx2h1QUt7IwCnMj79MZUx4Pkc6Cg6Ud8c+pChMTGMs7xlcUZJnhn01rNbhkejbDn2GuJ0cehhqEeRWfo+dYBQFGTPAJGMlL2R3PFhGFa/wt+iMvlUWcOirAFqsyAMdbiCkV9HotJ9L7adFa8VDNzRTuektaDLY1ugxLJ2Uq+0wIMknxNFBFmCexEh6JDMKO8n0F3fTji3wVgR31mqjPsca/z2O13Q/5qNfcizgA+l/MEyPHdbJlEO771/u3BZLZLTbz4G8gdHChvjjvLsVTj8WEOaRxbE/+ls0zSW7pzZetcgh1G+BayYDh9QZPqpWSwDdpa6i7YEKBvE= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1718; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 20:MpI3JnjhNImSFodzTZ87Yk5PQP26JinpSW13qdFGmLYztfUvoSKnJHeK/wfJGoICCEGNFgljYcSUGgAOb52+X6uk2Ffx+1FAS/l+84WKXSUFsEOvSE1kglvwtVWeF7F1kKzuhwLokyLAHOcwuNhQ4fThridlJnOTOZS8WZPJxsdm72OPxTV3++YwW7yTb0iL/JH+UruVhwfpeBFxidlwDRRu3GrOOdJTCRS1AnYNZEYXN4JE3VR+izY+67M8aYobqj/y48CBxytJ92Tvq9RND5aWzPmAKDXs5EH4uSAMXb5ywFR2JcXLLoYqwudrMXa6PLK2rZOvUs5MemcycWKQdTkqiTRhb/NHEj1jF1gsRFcnPUlWhBUfO33/dRDtYWjnoWJFxuqNnqV+kouLFf2ZRQbDpK8Enm0koN2kXgIHvRN+zKiHkRYmM1U9gxn3n4uKHgC8bETGWwpoLQKx4YNfTe5m/jYqWWg5tsmH8YUnYd1A1f4RYmr68lSubVm+/dtpkstX04RTYcr3Yxo6TeK/2DDlDRG4Cinxj1+Ugat4YsCyfYa6LQAc/vDo9qRAIjM/NyCzj10Bhd5rqR/2rvw3PRJqPoNgNz4uIzRJrNtI7g0= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:BN3PR0701MB1718; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1718; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 4:3FA7z9qS7BHImrMnvW81aGPUywCLQSSlio5NqHuWSN7uXTUj0EuOiRrAOlqM3YEYzhXx+pnZgSnj8E4kwP9kL15dT6+t1dWhS2ZG6V94CNgDiZexFjfsuVg/VK+QKp9YvgQ5X13Zw0JMfqoaN61zy1ZCjyD2TDseN5A91l0EbgeKodesACtz1/wZ9XRJ98x5w2Y7icAaVoEMl+WoGuPaBjTDLEt65qVTgsJ4MTkcjMZ6FJbpljJu4o489FMrGpGlivG0lbdDqQ9+tESfAi+a4sOz1AQj71qZBWHDLM74ck9VtoCqC7CXvQNfC4Mo2UIg6cy37DQ8d+31tNZn42IquB11+6rUiN1fz1U38I+VrYRwyLSkPGRhsYaQ6HB2WRcc X-Forefront-PRVS: 09760A0505 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(199003)(189002)(48376002)(19580395003)(68736007)(2351001)(47776003)(229853001)(97736004)(19580405001)(586003)(50986999)(110136002)(36756003)(3846002)(66066001)(92566002)(105586002)(189998001)(76176999)(33646002)(42186005)(81156014)(5004730100002)(101416001)(50466002)(2950100001)(5003940100001)(4326007)(5008740100001)(50226002)(2906002)(6116002)(77096005)(81166006)(8676002)(106356001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1718; H:localhost.localdomain.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; CAT:NONE; LANG:en; CAT:NONE; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1718; 23:mkt6n3qud8DmZ8AeSxI8W63JCTzn1jQi6PRl0K1?= =?us-ascii?Q?SKxrrtuo/RONM0SjBDIywafHZePTb1LC6hk2UKAbfIpHpA/5+RtwdoQsW8oB?= =?us-ascii?Q?AudbiqDsJTv3zkAde6O1c0ndye8ogP2aCX7dCdfION0P2g54KdeJp2CL+cte?= =?us-ascii?Q?rzSii+0QHU8u09uMvHJMZrVHWk27vuSNZbDWy8T7C3GoHNrNtsTcH5NAhhyK?= =?us-ascii?Q?jStjSSGUrmGeqIFOq7+bhuDdzWWKakqaAW+51m1uEBeUhTne0/zkaR8R2qSk?= =?us-ascii?Q?vFEwy5AUPmsZ1Ar9fqmmYg+7UJoj5Y83kEDsug8HcPoCLXnYQXTBq63i6Egl?= =?us-ascii?Q?kKyHUp1BT6hn34/MjlcCCmx//FELomkCEC0CzEBwgHtkttJ6F0MmsvZoMGbG?= =?us-ascii?Q?BnUaJczWwS81zYJYuXeC1dYLVZ7jq4DFSwf9WAP/UoGkW1hGLO8myRUAxqbd?= =?us-ascii?Q?mbp6Nmk5owl684ywM64hE0em7RggGyJsow6EIhw1ph1iRHy1GiqwfB4HIt0v?= =?us-ascii?Q?fO0+SJtAFYSV4C6V4gDH58Mx7/Lfdpjd+4JRONbX0CQ6e4jCCGsEen2EV8M9?= =?us-ascii?Q?8d9WnjqPHP1hV1Bdt+gyhvLMnVLfis4a1SiyU5cS1MhoutZQ70AaAHeaIJOV?= =?us-ascii?Q?OVzwLhWmNfLuQ+HFRWmMNS0+DwZoYgR5jxD2RQIZxfiXEphvmCdS8lqDpBog?= =?us-ascii?Q?a6QkzTfU8Od9H4KZRtv1fvPt4LrnW1MTFQ58aPSgwP1utKmrKX8W6YRbemfd?= =?us-ascii?Q?ccJsxna8Uo3dDbKRnD0TyzvU/mBiu6HERUHUnUKbbedupqXwSIA26N4ATHcq?= =?us-ascii?Q?GTtBtXviWqCwKq81d3NsqtDYI0KbMvZptPMSG1Vz45acbikS8MPBb+PI+0bj?= =?us-ascii?Q?OxqXWqmjSiptDrjnRH6UQGmloVV6v0pixJYZ3RcN5VkKnvJ1v5iCALfTchET?= =?us-ascii?Q?NrA1Dx1AELkXJvgRLFr0keVQSkPPbLESzVS5QkfaYvuqRcqy5uJUG2E+r/Zs?= =?us-ascii?Q?lgM6ReVc5hZaxHI7MvONikKatQHKTZrbtKLtV80C5s+uHRs5HjJAj+POaOXS?= =?us-ascii?Q?6xFDRXRm/pWhEaMeuDYX03GO0scVf?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 6:kfHb6g7JyXLePbSw7byUK2bIxQizCbvfKtMjD0qvc/yMjpYlPdXh66LLyZvpmGsUF0oHJkwzh+fuJve2gtsHrH7UtDznHArmquACuHWSpDpZau4uo7HJavjx0xUeKtfYdOW0Pg+8WhBBuH18ntR7R/KVfZOZ10HrCHHlqHksx/XCg5Q62LAIi/bPu6wYXLfTlswQcHG0L9CyrMbLgXrxWxngb6Inmq2Gar8wA4Og6ecMd4lg46LLLMS5hrUxuK7P8DBEmeF5wr6e37icm54gj+RQM72anGvfKsCpsTQBEh8=; 5:B9UPp/pUbnsMto4+ylJOKtQcubsGQqeC/ZnntiMS3c99D4iuZt/d0hA7g3Yp0YRC5eQ6YbVthQym8yyVUv81/3uTTjvaxfLfQvvnglSY5BEXeR70g48ALmYim62DdYwwHOiN8YD5jkf7YFunhHveUA==; 24:7NVqmYrOZiO4jG6okoU+A95sd1KByd+p4brAZJ6RNPQundHWMmaUb/6ZS2qF1L+9W2PmKd5UO10kD+g+2aZw3OFUWq/2MlGg2fBvmskGhUA=; 7:mILLeHbqx4tqTe5xBMRuOF49O8LSUq37QWPQxam5USB6RkaF+oriKzbt0G0Cg7r63jOV+EVCSZPWCVuAzdNO4Odu9ryrvDI+a2ThVlg+IyqwBANEqThK1IfxF8dKZcIVYA3UxPU/UjoBJpi+/AC0eqia6VAqj6B9HHHdakGlFu2ee+9Mx4fF1vYR15xKjNrZZ3+6YdGkz8PMLL9Zr6Lzew== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jun 2016 13:32:41.6824 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1718 Subject: [dpdk-dev] [PATCH v6 23/27] net/thunderx: add Rx queue start and stop support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jun 2016 13:32:47 -0000 Signed-off-by: Jerin Jacob Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki --- drivers/net/thunderx/nicvf_ethdev.c | 167 ++++++++++++++++++++++++++++++++++++ drivers/net/thunderx/nicvf_rxtx.c | 18 ++++ drivers/net/thunderx/nicvf_rxtx.h | 1 + 3 files changed, 186 insertions(+) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 33d5fba..ed69147 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -562,6 +562,54 @@ nicvf_tx_queue_reset(struct nicvf_txq *txq) txq->xmit_bufs = 0; } + +static inline int +nicvf_configure_cpi(struct rte_eth_dev *dev) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + uint16_t qidx, qcnt; + int ret; + + /* Count started rx queues */ + for (qidx = qcnt = 0; qidx < nic->eth_dev->data->nb_rx_queues; qidx++) + if (dev->data->rx_queue_state[qidx] == + RTE_ETH_QUEUE_STATE_STARTED) + qcnt++; + + nic->cpi_alg = CPI_ALG_NONE; + ret = nicvf_mbox_config_cpi(nic, qcnt); + if (ret) + PMD_INIT_LOG(ERR, "Failed to configure CPI %d", ret); + + return ret; +} + +static int +nicvf_configure_rss_reta(struct rte_eth_dev *dev) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + unsigned int idx, qmap_size; + uint8_t qmap[RTE_MAX_QUEUES_PER_PORT]; + uint8_t default_reta[NIC_MAX_RSS_IDR_TBL_SIZE]; + + if (nic->cpi_alg != CPI_ALG_NONE) + return -EINVAL; + + /* Prepare queue map */ + for (idx = 0, qmap_size = 0; idx < dev->data->nb_rx_queues; idx++) { + if (dev->data->rx_queue_state[idx] == + RTE_ETH_QUEUE_STATE_STARTED) + qmap[qmap_size++] = idx; + } + + /* Update default RSS RETA */ + for (idx = 0; idx < NIC_MAX_RSS_IDR_TBL_SIZE; idx++) + default_reta[idx] = qmap[idx % qmap_size]; + + return nicvf_rss_reta_update(nic, default_reta, + NIC_MAX_RSS_IDR_TBL_SIZE); +} + static void nicvf_dev_tx_queue_release(void *sq) { @@ -687,6 +735,33 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, return 0; } +static inline void +nicvf_rx_queue_release_mbufs(struct nicvf_rxq *rxq) +{ + uint32_t rxq_cnt; + uint32_t nb_pkts, released_pkts = 0; + uint32_t refill_cnt = 0; + struct rte_eth_dev *dev = rxq->nic->eth_dev; + struct rte_mbuf *rx_pkts[NICVF_MAX_RX_FREE_THRESH]; + + if (dev->rx_pkt_burst == NULL) + return; + + while ((rxq_cnt = nicvf_dev_rx_queue_count(dev, rxq->queue_id))) { + nb_pkts = dev->rx_pkt_burst(rxq, rx_pkts, + NICVF_MAX_RX_FREE_THRESH); + PMD_DRV_LOG(INFO, "nb_pkts=%d rxq_cnt=%d", nb_pkts, rxq_cnt); + while (nb_pkts) { + rte_pktmbuf_free_seg(rx_pkts[--nb_pkts]); + released_pkts++; + } + } + + refill_cnt += nicvf_dev_rbdr_refill(dev, rxq->queue_id); + PMD_DRV_LOG(INFO, "free_cnt=%d refill_cnt=%d", + released_pkts, refill_cnt); +} + static void nicvf_rx_queue_reset(struct nicvf_rxq *rxq) { @@ -695,6 +770,69 @@ nicvf_rx_queue_reset(struct nicvf_rxq *rxq) rxq->recv_buffers = 0; } +static inline int +nicvf_start_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + struct nicvf_rxq *rxq; + int ret; + + if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + /* Update rbdr pointer to all rxq */ + rxq = dev->data->rx_queues[qidx]; + rxq->shared_rbdr = nic->rbdr; + + ret = nicvf_qset_rq_config(nic, qidx, rxq); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to configure rq %d %d", qidx, ret); + goto config_rq_error; + } + ret = nicvf_qset_cq_config(nic, qidx, rxq); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to configure cq %d %d", qidx, ret); + goto config_cq_error; + } + + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + return 0; + +config_cq_error: + nicvf_qset_cq_reclaim(nic, qidx); +config_rq_error: + nicvf_qset_rq_reclaim(nic, qidx); + return ret; +} + +static inline int +nicvf_stop_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +{ + struct nicvf *nic = nicvf_pmd_priv(dev); + struct nicvf_rxq *rxq; + int ret, other_error; + + if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + ret = nicvf_qset_rq_reclaim(nic, qidx); + if (ret) + PMD_INIT_LOG(ERR, "Failed to reclaim rq %d %d", qidx, ret); + + other_error = ret; + rxq = dev->data->rx_queues[qidx]; + nicvf_rx_queue_release_mbufs(rxq); + nicvf_rx_queue_reset(rxq); + + ret = nicvf_qset_cq_reclaim(nic, qidx); + if (ret) + PMD_INIT_LOG(ERR, "Failed to reclaim cq %d %d", qidx, ret); + + other_error |= ret; + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + return other_error; +} + static void nicvf_dev_rx_queue_release(void *rx_queue) { @@ -707,6 +845,33 @@ nicvf_dev_rx_queue_release(void *rx_queue) } static int +nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) +{ + int ret; + + ret = nicvf_start_rx_queue(dev, qidx); + if (ret) + return ret; + + ret = nicvf_configure_cpi(dev); + if (ret) + return ret; + + return nicvf_configure_rss_reta(dev); +} + +static int +nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx) +{ + int ret; + + ret = nicvf_stop_rx_queue(dev, qidx); + ret |= nicvf_configure_cpi(dev); + ret |= nicvf_configure_rss_reta(dev); + return ret; +} + +static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, uint16_t nb_desc, unsigned int socket_id, const struct rte_eth_rxconf *rx_conf, @@ -933,6 +1098,8 @@ static const struct eth_dev_ops nicvf_eth_dev_ops = { .reta_query = nicvf_dev_reta_query, .rss_hash_update = nicvf_dev_rss_hash_update, .rss_hash_conf_get = nicvf_dev_rss_hash_conf_get, + .rx_queue_start = nicvf_dev_rx_queue_start, + .rx_queue_stop = nicvf_dev_rx_queue_stop, .rx_queue_setup = nicvf_dev_rx_queue_setup, .rx_queue_release = nicvf_dev_rx_queue_release, .rx_queue_count = nicvf_dev_rx_queue_count, diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 1c6d6a8..eb51a72 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -579,3 +579,21 @@ nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx) rxq = dev->data->rx_queues[queue_idx]; return nicvf_addr_read(rxq->cq_status) & NICVF_CQ_CQE_COUNT_MASK; } + +uint32_t +nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx) +{ + struct nicvf_rxq *rxq; + uint32_t to_process; + uint32_t rx_free; + + rxq = dev->data->rx_queues[queue_idx]; + to_process = rxq->recv_buffers; + while (rxq->recv_buffers > 0) { + rx_free = RTE_MIN(rxq->recv_buffers, NICVF_MAX_RX_FREE_THRESH); + rxq->recv_buffers -= nicvf_fill_rbdr(rxq, rx_free); + } + + assert(rxq->recv_buffers == 0); + return to_process; +} diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h index ded87f3..9dad8a5 100644 --- a/drivers/net/thunderx/nicvf_rxtx.h +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -85,6 +85,7 @@ fill_sq_desc_gather(union sq_entry_t *entry, struct rte_mbuf *pkt) #endif uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx); +uint32_t nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx); uint16_t nicvf_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts, uint16_t pkts); uint16_t nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, -- 2.5.5