From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0054.outbound.protection.outlook.com [65.55.169.54]) by dpdk.org (Postfix) with ESMTP id 4CFDF5A52 for ; Sat, 7 May 2016 17:18:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=r81GNkc2u9CN+ZEXoI0KF7FFD7OiYZWE+eptwMYmRso=; b=JNrIkBBXYm7tefSQ6HrBxuQ+gqqRIJP3lCvO8/kP3mLczDheOFXMjMoB+PklY/4q+9oCzBUb2HIlFQRzC1qnUA9wFnlYdLM3HKi/KlCF9mBmuL2mRQmBXXtBHFL8s0qBHyBLjdNcgV/bYYQTo1XXXh8ZLhb+R8lgzDKn/p8BGMw= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain.localdomain (122.167.10.92) by BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17) with Microsoft SMTP Server (TLS) id 15.1.492.11; Sat, 7 May 2016 15:18:09 +0000 From: Jerin Jacob To: CC: , , Jerin Jacob , Maciej Czekaj , Kamil Rytarowski , Zyta Szpak , Slawomir Rosek , Radoslaw Biernacki Date: Sat, 7 May 2016 20:46:25 +0530 Message-ID: <1462634198-2289-8-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1462634198-2289-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1462634198-2289-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [122.167.10.92] X-ClientProxiedBy: PN1PR01CA0050.INDPRD01.PROD.OUTLOOK.COM (10.164.136.150) To BN3PR0701MB1718.namprd07.prod.outlook.com (10.163.39.17) X-MS-Office365-Filtering-Correlation-Id: a7258ff2-e1b0-4c89-da41-08d3768acf1b X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 2:KUGCsBlHdk1qG2SNHrpldmU43Qn5UTSx12tx4Z7o01U/lZxacyHUOJ36rNdXB12TUmdzEKLwlaAnJdKG8t4m4wmXETVkfyVDuI9wzt1dJNKsbod3uK9iuBuUeBEhUqZfSrF15jinAfkObD3I8BvNhXkkuveyrsq/SYVgFbUILg6ph2EolnGvCq2Q2IxiJbd4; 3:Qvqpg3e8hPYzVWHNxmHbcC+dKEwEmAlVtg2//ieJEQU3pbzpqnBWeTA0wuboaOzdUB55rwClVdExUoD5XF7r05umATMIQTydlUiteJgtgXz1lVZnm23IVqiv56tzXHYf X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1718; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 25:5dWbwCU0G50XS80K7ToZxBiLaAE/SuagwH0p19LItgIbhj2FsXVfVqR2QjmPWPopvDTUOMgyLDN+IfgaYPPmlaiQ+ICkfZp1XxTP3cDqpR4TAf4Ku7w7szWVfJ6OcNvEYFD1U4wKFC+R4yY1IQjL9r0/sTtHJb5GZa5jSERStvkOJFsQE3XRmE3uhdyAEy9KQXTtqu+s8wDAr9/Mozx3Jxf6sDJlZ5nWMCjb6w2LM2JxZ4RGAKqri19Hs2yz/XdSZvPfJA1X/f9RtNLUEV27VmpuNAkhsV435QsP2YYY/8m+SH6MUdyfL7xdmlyiPZvkFAm28CkTEeNsDAAruodLdII4ZOoD5zXww01NZ+SQuklfXkA/cqgKuF5VEwQyvrrZ7eGxT91SWuTXgMu0J9AeUmoKYQzIW41bWqHFWoj6q0hVSEPX4YE07G/+/Onec1mdT9FR27Mwb24LmUxfCe0t8mZkNBR7MEfPVxj8FwVpxleAEQ+3Fa+tyjkiOqizBekqn8apg6PJm3F9jpi1JkNS3w5NmkiLSn+mUlIitzG2F6eBeV2plUfEqIMa8U+kCKmTg1mFZsYYmDR5nZg93ynsrqpv4NQvR9yaO5x4e2GMUz2QIwQE9DBH4+RSNfnj+yEIQ8E4r60lmeTw1aF8RFMBsU/thmjEUcjmPqrg2mPk7QR6xXrLbj3ZZ6MFuL76/WFV2jsFaZuJoCWVa/6edD5BtRopgluODhpEllF2xgmIhWA= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 20:/vqkDtAUzbQt3pz9+ZdxMcgZ7dFSnJsayyUMNjsGPBKo1JZUDZZeI4W2RUVdUk8bCgZRcniETOq/hU3LKmKNZ6T4Hkc89i0f1AbKZ+Dx9NpVJRim6/nHS8tW2widYbrN1MvkPnmxiTefyYWJ+smQ1y6OoxnVg3tAowXScwZd2Md/LiFLU3yR0UPq2F7BOEVb16ZJS+zOGzznu2IJ1wwZnVRI3YOrH41ex7dt9CeLnWLXUzL02ESGrzSuq2ZSHerC1nGLSo35J5wKqQOmFQtRt/zZDzBKbuB3Rcef7dzpVbcXZPTIXPVGm4jP1mbKsxtxTkdg/Yzh8ypKHDGqDCW+1Q5qeB0NHD33mCIZzIDykpyj194laKz0oLgE2Ff7YaOl5dzG9GZMmVzvL6fpSU3YNhngVzTcVDLRdwCJUpWFInzPz3EN4bS96MKv07IFnM6zs+0XYSlgypBxWoA7nN05Nf39llyUdUbH/FRL8TeI9aVjqRbqnZOR9ExQW9nsfE0Ih8gXG2uvO/xpqOt8ftfN1KLcp+TKe3oFu/yZ2cbzEIIcQo+Zv75nCS7Tj56bUvkPtQsuVOMIZU1hKVOnsluyl71104h5HCdphIqyww0Gt+g= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:BN3PR0701MB1718; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1718; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 4:N5blhgULD13RQt/vA7sb+/+ZQN2GVFGpP6TE1Rh2ydDV6lB4s16gGsRaL0lqE7/PXOYywCfEaFYhHpwQl3PhJaHsyLyvyNk5u7U2gTf2VXOQwUWEs3+bSCcpjLkkFH8CO8lH+f7UxcMz45lZklKA48VbYiyOVGSc/NhOy1kLehEP3q5/nEGjU8b6mM/aeMOdEiXud6FaP1CUlPiBGGHoz44EejDoSNFnEYzLY5oIgs9W1h1FATlKSNMSpO9RlBcozHT0z/MPOE0wywxr1xdOVDlQp580sESWkoYaSCcsHw6wUAAVHtmOipM9cMumCkkyjmQwkE7NSrJODcB6Ao8Z7R8Zu1peWTT/IO4wS3DsV+rbGN7qq1beEHAvduM1jctL X-Forefront-PRVS: 09352FD734 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6069001)(50226002)(77096005)(2950100001)(48376002)(81166005)(50466002)(66066001)(47776003)(92566002)(5003940100001)(4326007)(2906002)(110136002)(50986999)(2351001)(33646002)(586003)(3846002)(6116002)(229853001)(5004730100002)(76176999)(19580395003)(19580405001)(189998001)(5008740100001)(42186005)(36756003)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1718; H:localhost.localdomain.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 23:uXMY2VySd+CJNksJvzlr729MACQEX4mLz5iBNPJ09Mme+M9E+KMw/Ufzw4Ful1VLgyGW3QAF5hRO/HD5NJBKi0M+7kSrrpgBBdZebjyAt14m5pWS9lQWDoMV9o+OgGS3RDyKEhyeuVfvuWGv2qobjgsI3cqURFLfz+pWK7fllBr1w9M9fplUMENVcuk8fqWX20T6a+/APBWHlj7Wrl9gxw03yvdWYkUx72KtLK2ROQp2aTvcDjvq+1z/6Rdx48yt+UtSgI2mY5Hqfzj1CDcKhy528YeRqHIqYuSXSDrtX6wTxdtJ/cNB6rd2X7kCn3YqWZGH4TuAFzN2j+SBJq1xcZ0dgAYx9D4WXI4PDVGH843smzyNVV3xv+DyVszIJrsQiYx6ZtgWrFmtn54j8K4MYNmZiwd1mA4+dml07bhuB82Lmup2FaRtv8lSqv/NQjw4pZgfumL+q551n+SATsMFVl3nl3+ic+wCWEM2mY1MtzrztrGmkE4crrIYjL8qRUIxHE30+V1DBIQVvlRFeD1LmMApipj5//8ausrHKbae4GAmg10rwER6KZCEYX6uwT7DpJCQxb+tweBiqQX6LK5tuS7O2FJ6gE4P7/GVAdPRPAkZb8T2jufcTiOwgjCkzxapehPk17OrUSzp/DX8s9QTCx0cEOSNB8LdYLZ3BrnP0N7UBx5Ie73Pws0XDCY6cYiUBSj2RQsHsyS4aGvXBm93CBxL/iXPVO8R3igcljvBMFPitmhx9/tpj5GCN2Pp7muV1TanyBk83xvKVKTbpJxwxuAVxzFBfftAa7M+I/vwsoTR24WgJo4wQf93TK6aa/InHqNHdvwvuDiz5wf+YkNJGzb7t0kAR4w2cPREJDAoCP8wyjGFQ8feZBgr4++SEdBm X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1718; 5:Ou8e8uZ0VtduI7jisie/X7W4lIIzlthSqxwsrVEtcOK9jYHf9086vNXNPRsEhrkwl9PjmBuNC05VbkSZnkFORZ5Sdwt+UY6xhmhsdXFSYFkOuaso3qzbqCTTp0pRW3ydhEOpsX6kBzMuIhYScwxCPQ==; 24:vCzwFDoy6Pv457YjpV6fy5lwU28LEFnxC1QcOo7qTXWCwA9nwh3rq9a/BCIR12gpqZFth2MJLsxbcCtq+N/fDHxSOUqq0d9W3hYWQcAAdsk=; 7:G7Mb/nfI3ZXz7UxtvqMcH/nGTZUwZ5GI5RRDasdxlBTCuFGPqUts2ArjuTwi3AsDD8FANmjRfIJfsFzM8b2ybvQw2AcLeSDXRPBcO83hKRwkVsbZvf4ILIaiyWDs71Q4F4ElbV6lNGYwfPyeEusFnxH+kdj2XrlnePIsl0qkgKPlDKIzEE+rEoMZRDsDx26o SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2016 15:18:09.6331 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1718 Subject: [dpdk-dev] [PATCH 07/20] thunderx/nicvf: add rx_queue_setup/release support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 May 2016 15:18:15 -0000 Signed-off-by: Jerin Jacob Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki --- drivers/net/thunderx/nicvf_ethdev.c | 141 ++++++++++++++++++++++++++++++++++++ drivers/net/thunderx/nicvf_ethdev.h | 3 + 2 files changed, 144 insertions(+) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 1269672..3b94168 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -73,6 +73,11 @@ static int nicvf_dev_configure(struct rte_eth_dev *dev); static int nicvf_dev_link_update(struct rte_eth_dev *dev, int wait_to_complete); static void nicvf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info); +static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); +static void nicvf_dev_rx_queue_release(void *rx_queue); static int nicvf_dev_get_reg_length(struct rte_eth_dev *dev); static int nicvf_dev_get_regs(struct rte_eth_dev *dev, struct rte_dev_reg_info *regs); @@ -198,6 +203,140 @@ nicvf_dev_get_regs(struct rte_eth_dev *dev, struct rte_dev_reg_info *regs) return -ENOTSUP; } +static int +nicvf_qset_cq_alloc(struct nicvf *nic, struct nicvf_rxq *rxq, uint16_t qidx, + uint32_t desc_cnt) +{ + const struct rte_memzone *rz; + uint32_t ring_size = desc_cnt * sizeof(union cq_entry_t); + + rz = rte_eth_dma_zone_reserve(nic->eth_dev, "cq_ring", qidx, ring_size, + NICVF_CQ_BASE_ALIGN_BYTES, nic->node); + if (rz == NULL) { + PMD_INIT_LOG(ERR, "Failed allocate mem for cq hw ring"); + return -ENOMEM; + } + + memset(rz->addr, 0, ring_size); + + rxq->phys = rz->phys_addr; + rxq->desc = rz->addr; + rxq->qlen_mask = desc_cnt - 1; + + return 0; +} + +static void +nicvf_rx_queue_reset(struct nicvf_rxq *rxq) +{ + rxq->head = 0; + rxq->available_space = 0; + rxq->recv_buffers = 0; +} + +static void +nicvf_dev_rx_queue_release(void *rx_queue) +{ + struct nicvf_rxq *rxq = rx_queue; + + PMD_INIT_FUNC_TRACE(); + + if (rxq) + rte_free(rxq); +} + +static int +nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + uint16_t rx_free_thresh; + struct nicvf_rxq *rxq; + struct nicvf *nic = nicvf_pmd_priv(dev); + + PMD_INIT_FUNC_TRACE(); + + /* Socked id check */ + if (socket_id != (unsigned int)SOCKET_ID_ANY && socket_id != nic->node) + PMD_DRV_LOG(WARNING, "socket_id expected %d, configured %d", + socket_id, nic->node); + + /* Mempool memory should be contiguous */ + if (mp->pg_num != 1) { + PMD_INIT_LOG(ERR, "Non contiguous mempool, check huge page sz"); + return -EINVAL; + } + + /* Rx deferred start is not supported */ + if (rx_conf->rx_deferred_start) { + PMD_INIT_LOG(ERR, "Rx deferred start not supported"); + return -EINVAL; + } + + /* Roundup nb_desc to available qsize and validate max number of desc */ + nb_desc = nicvf_qsize_cq_roundup(nb_desc); + if (nb_desc == 0) { + PMD_INIT_LOG(ERR, "Value nb_desc beyond available hw cq qsize"); + return -EINVAL; + } + + /* Check rx_free_thresh upper bound */ + rx_free_thresh = (uint16_t)((rx_conf->rx_free_thresh) ? + rx_conf->rx_free_thresh : + DEFAULT_RX_FREE_THRESH); + if (rx_free_thresh > MAX_RX_FREE_THRESH || + rx_free_thresh >= nb_desc * .75) { + PMD_INIT_LOG(ERR, "rx_free_thresh greater than expected %d", + rx_free_thresh); + return -EINVAL; + } + + /* Free memory prior to re-allocation if needed */ + if (dev->data->rx_queues[qidx] != NULL) { + PMD_RX_LOG(DEBUG, "Freeing memory prior to re-allocation %d", + qidx); + nicvf_dev_rx_queue_release(dev->data->rx_queues[qidx]); + dev->data->rx_queues[qidx] = NULL; + } + + /* Allocate rxq memory */ + rxq = rte_zmalloc_socket("ethdev rx queue", sizeof(struct nicvf_rxq), + RTE_CACHE_LINE_SIZE, nic->node); + if (rxq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate rxq=%d", qidx); + return -ENOMEM; + } + + rxq->nic = nic; + rxq->pool = mp; + rxq->queue_id = qidx; + rxq->port_id = dev->data->port_id; + rxq->rx_free_thresh = rx_free_thresh; + rxq->rx_drop_en = rx_conf->rx_drop_en; + rxq->cq_status = nicvf_qset_base(nic, qidx) + NIC_QSET_CQ_0_7_STATUS; + rxq->cq_door = nicvf_qset_base(nic, qidx) + NIC_QSET_CQ_0_7_DOOR; + rxq->precharge_cnt = 0; + rxq->rbptr_offset = NICVF_CQE_RBPTR_WORD; + + /* Alloc completion queue */ + if (nicvf_qset_cq_alloc(nic, rxq, rxq->queue_id, nb_desc)) { + PMD_INIT_LOG(ERR, "failed to allocate cq %u", rxq->queue_id); + nicvf_dev_rx_queue_release(rxq); + return -ENOMEM; + } + + nicvf_rx_queue_reset(rxq); + + PMD_RX_LOG(DEBUG, "[%d] rxq=%p pool=%s nb_desc=(%d/%d) phy=%" PRIx64, + qidx, rxq, mp->name, nb_desc, + rte_mempool_count(mp), rxq->phys); + + dev->data->rx_queues[qidx] = rxq; + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + static void nicvf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { @@ -324,6 +463,8 @@ static const struct eth_dev_ops nicvf_eth_dev_ops = { .dev_configure = nicvf_dev_configure, .link_update = nicvf_dev_link_update, .dev_infos_get = nicvf_dev_info_get, + .rx_queue_setup = nicvf_dev_rx_queue_setup, + .rx_queue_release = nicvf_dev_rx_queue_release, .get_reg_length = nicvf_dev_get_reg_length, .get_reg = nicvf_dev_get_regs, }; diff --git a/drivers/net/thunderx/nicvf_ethdev.h b/drivers/net/thunderx/nicvf_ethdev.h index da6fdcf..8ffea8d 100644 --- a/drivers/net/thunderx/nicvf_ethdev.h +++ b/drivers/net/thunderx/nicvf_ethdev.h @@ -60,6 +60,9 @@ #define DEFAULT_TX_FREE_THRESH 224 #define DEFAULT_TX_FREE_MPOOL_THRESH 16 +#define MAX_RX_FREE_THRESH 1024 +#define MAX_TX_FREE_THRESH 1024 + static inline struct nicvf* nicvf_pmd_priv(struct rte_eth_dev *eth_dev) { -- 2.1.0