From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0092.outbound.protection.outlook.com [157.56.111.92]) by dpdk.org (Postfix) with ESMTP id 4D0489AD0 for ; Tue, 7 Jun 2016 18:41:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=AX9WODor0DhpqbRUOHxCKSL+VscCJr0zxaGo+c///VE=; b=TIX1IpkLP5GMoV0dLg3uwU9kAtyWqfTWHtro8T/A1vFnVPji5ZlciQUAeT4ZCH8hHg39jkAIoIg1A8mQ7zB4jm3F8ZiXo6n6veeQ0BTKtC6RrFI7XGr+yP3K7H1xogwT5eQBk37HReTvn8TIraxo6zn/63lNl4kq8CwfnRKrKFc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.caveonetworks.com (111.93.218.67) by BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) with Microsoft SMTP Server (TLS) id 15.1.511.8; Tue, 7 Jun 2016 16:41:43 +0000 From: Jerin Jacob To: CC: , , Jerin Jacob , Maciej Czekaj , Kamil Rytarowski , Zyta Szpak , Slawomir Rosek , Radoslaw Biernacki Date: Tue, 7 Jun 2016 22:10:19 +0530 Message-ID: <1465317632-11471-8-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1465317632-11471-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1464540424-12631-1-git-send-email-jerin.jacob@caviumnetworks.com> <1465317632-11471-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MAXPR01CA0054.INDPRD01.PROD.OUTLOOK.COM (10.164.146.154) To BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) X-MS-Office365-Filtering-Correlation-Id: 342b212c-68f4-4ee8-f7d7-08d38ef29da1 X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 2:1yjLQdcVnG5P6FUjygOauOtSFupHnQpMTM+GKYhpoeeMKWDduBZbl7wH29delbrKKlSM8bhs6ASe22/jfHeLP7pOkID7ZY49A8qai0dApNksYinuRJzdq90xGyTvi5YNDEd7xJDtJ7mbZOZF3DJveFc8DKYmVQl6gWcRSSINeWpdw9XDeZ/oQU8rhlVYhaMi; 3:uWd+roDm+3j0Ia697G5KOB/z40lRG/A4ssUIEtKrp/d/1KeqDXuew4UYkIFwwCqZRC2gAJhoj6ZwLNw/YDO2ds/tJEOECTLHORY+3HrOZoHaLXVWVO2MxZbnfFHrw7WL; 25:wswx715M/9CEhp9JKfKrdpNkchZCFxcfPJ+n6za/ahyjBZpKhdXY+4LNhQ8pUE9EeGSTFYFxFXHdoeH02G4PJ6lPx2Hmjvqy/SxNC2jomoU7FnoZBqhCh8x/hO2fYlA023GKtAFsRCZLeoNENNJEiospzv/zihAr1mcl3pa+sht2OdpUyjCc9uwgetYI+6AbufihZuS2uhCYIe2LgLjPRzPhgUIXVTbOaMlbzVNJLKR4GkBq7gPOtpc6XhNfPLjXNn5fakP2HCBhvpHCJafpqyFkoCAHsdxmnsGaxfqjhUsXHlH6Oxg2Y6keVoRCc6znHgP6qLQpXJpM/HF5EgRHb2KANPeKCYkEciI4wD6p4F36BXISDNtG92jQE0JTzoHnwbmfVcvPJjN4mOsbEUX7i6ieNBsTrvADUZ+pR/LcTxY= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 20:Dq07h3ccC1jjHHfBpnGHfWU7KoKfjlYsTEXxELucx0aoik6cN7c77fNfzuypMvDZwC38hTIP5PBpf9QpT2QMQGuYMhuxN653U7o4FFgRCLA6LhjRUlvWyHvXOjVowhX4fo4KufNbNM66odP6n58WAPnOLrS0Y8aktTG4lbshPauutQ3VdHUqlCYAuZNMVrYDIJjcfuZd9MfebGG9WmHMcVeAPCdXJtUkVOOupfW7r5ZWPGG4ZxKFx5X4alLU2cSXf5s+Qbh9x1/i+Pt+2HdT2jXcTWMRm9E7/xACz/c07ryv6BfEGd8uxrFDoLyp3PghDpRzv+8Ou25TPzngH+ZvJdCqXi+lqs7YeVGbL8UtY5Vz/HO3nfpRwxzrqTFajXc731Nt1hZnhIQQZamGSL+K7UCT3V/E1p3gZyT1hjCAF9OaFp7x91nQjY/goH+3Fm2/2YkCXdq4MPptgGJP8aYcsZu4MqVlHSJS/eOu/RMm7T2u6blH/uSxeIIvYrsDckrlvac6O6GNpfqVwACM82lFo5AAgsDcGcNax0dgXHA/0P7LxwvbrHliWHZUkQ9OpQtXq+ysIQAtrYAOx1FAwdQTRjzOyBYGxsle1AVCJOGXK0E= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:BN3PR0701MB1719; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 4:ZtMtApTDYhpgYszUVlCYyhk434Fo5KuZHeu4q3PqtrCKPDFMhUkKuBnBeiTN+FFrWZxmBP/POyMdjZ6dd4uYjVbYfcInp1cA7JwIrN7M0yV+Yfjdh6HMj7Ya8iX3XxZNRK5RNw21Iq/aF+JeSQJXB2vHztxYIC4afcfpod/i/ipHOYKpcOM+e1JKY/RqV1B1saa8+XTV9Kt9A/Vws5o8mk7e35gUrpjeL7GGmiFhy0PN7GfqAgn7jzSDXf7tZL1xWrQShn6BW0HpjAle+R+8YWHOz826VEs2/1pLd1i+Y8qG2NtX/l3TV0QZxvgXAmnzlcywCP3lCngpOOaNdEDmMMlwlTCAfPv3XzyLU5VnEhAnyYewLI04c3XFjWUZtY1H X-Forefront-PRVS: 09669DB681 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(199003)(189002)(81166006)(8676002)(81156014)(47776003)(66066001)(36756003)(92566002)(2950100001)(586003)(2351001)(48376002)(76506005)(42186005)(50466002)(4326007)(77096005)(3846002)(6116002)(5003940100001)(53416004)(5004730100002)(105586002)(97736004)(2906002)(5008740100001)(106356001)(189998001)(19580395003)(69596002)(101416001)(110136002)(19580405001)(68736007)(33646002)(76176999)(50986999)(229853001)(50226002)(5009440100003)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1719; H:localhost.caveonetworks.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1719; 23:DPvw/R48FHbVVT1mVRAWMboXzApPLUHrNRMRRW1?= =?us-ascii?Q?DVOcC9WaU+LyMpOZgga01Lo2jb6vHDBymEhgVORRKCBnPFHCJ0MBIxuMnY7f?= =?us-ascii?Q?Rjpd3Uej4Wg1Ue0ZrTFufKjRuBMqAynzKU4N6iDooZI09LUhFCrI+eITgPOi?= =?us-ascii?Q?oko2kGUCgGT9JeQ84FMZpf6HQgxd4j+5GWMndeG8UQZvrq+kZdXA5MW1Afg9?= =?us-ascii?Q?VjfYbgNl6+/MmxqpycbY0cbwFkLFz+taCRsi3EpE3EMjo3w019pbd4S3AdnJ?= =?us-ascii?Q?ILKRMFYTwlhX9GW0nBcmOPO9Y3ymv/Uj3mkmVUyfjFJsCeAdWSP8lWFFSthb?= =?us-ascii?Q?mXTyArLfEjOpXG4q6m508zDGq8or7U1YpjuG6PW89Otm8lxDe65wz48nntpH?= =?us-ascii?Q?71xYjrYL5dZkQOw1v1GBwMVPnkvz4MfsrX8bUwuWr97YQYx3+Z4RI+flWaj/?= =?us-ascii?Q?rtbMbYlhSdnLUy2DK5wxBk6xhcC9QVzcdy3KgwiZpuDbZVedrOEN/noWDUfb?= =?us-ascii?Q?GBmdSI5jhmLEpZeUlyMyRoR5R+LTI6DpJKjWM2AjEd8EUQJdzNiG+/GM9SFB?= =?us-ascii?Q?pczey/EhJsWMz1qMbvWBDcQTgxESPuVrBFpIspKYAJflXDLWKSOkDnfXn+vT?= =?us-ascii?Q?VmMuM4oAKkcPBSSY581ilWAnLJjTIH4n6UWfvvPSxELFjXcOBC8E4nFQIwwV?= =?us-ascii?Q?B1fJPfXG7xaCS9LDFNTbyRQJq8h5ahh3jJYT6jUMkObPFD9fMpBjWEM2M6KU?= =?us-ascii?Q?EoT/YdjbU37rRmOmQ9YwKIXckN6jKkJlTi0NB5SJeEofondEXVvCThNVpURr?= =?us-ascii?Q?T++pfU3oADk1NcOfjDozoKBQ+HUxDBNsXDagMzM2GIKXP3B9IZidD7MLVX0z?= =?us-ascii?Q?Qm0ozCEZ6kPhOH2NqlTwrIR+nE8VYnq82xAPJGmn31NBkvDmhR3uDrTLCfD1?= =?us-ascii?Q?usHcs3z3lNGL6SaMQPkWIsg2jV9HzkKPp/PmVCmyQsE//vgoZ5IQAb07VXfA?= =?us-ascii?Q?zWc5eeesmmIiNakG1V0jmNOmyc8QIJonttOzK7LX/Fok+qT4DAqqcSQSvUX6?= =?us-ascii?Q?mP9qEIHMqjPDrUBS32GvaqepD2WBST6/6aEXkKSYcDyPM4tumM6gCg+2iRxo?= =?us-ascii?Q?4sAe5T7ye3NrPrCX3Jv1z+IAAOynHbsqRwrX5tIKFN2lSk8WTNz7ubA=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 5:VIj3GiuhfBRMJKGGVHbaNJj9jNQ5TsK+NXCfPciT+HIS8RN45oVbsLjnv1sjCik+nYg0t5aJZAnHLRETVvUoUFgdOcqBb0F6zxxB6gMVtcc7OFkWmEhowY6Nm1kAztKsZzlrNyg0adfQlbdnltb7ew==; 24:L4aXkeryhrh8AOBf/pC3AAe56qDBQUl6+DBYg/oK4YZvXpR7KY50T77Unr8q31J72TzTqntQhCAbAd9m5jwFBSe1Etc0zAIUzo0w9Wx7Opg=; 7:KA17MO3aAfl0LitmxgROztczdkIWtkIS+Ysh4AR3ZCfc4qpV1MeXi+/QedGjOt1k+vcPxYa7cawvIOAUt/P9AQpSDD90sg27c8yCF3yLJE/icK6a22BqXg+Wwqnw+//RadpewIP4xrWpebVMPuf0hrS6jPkG6gSHxuT1hQlX285iBowyRbiM2Lr0MnTxKNhx3keJIsQQE3Z5afgsARo1NE9pAenynjn3hff09gRXKhY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2016 16:41:43.8628 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1719 Subject: [dpdk-dev] [PATCH v3 07/20] thunderx/nicvf: add rx_queue_setup/release support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jun 2016 16:41:48 -0000 Signed-off-by: Jerin Jacob Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki --- drivers/net/thunderx/nicvf_ethdev.c | 141 ++++++++++++++++++++++++++++++++++++ drivers/net/thunderx/nicvf_ethdev.h | 2 + 2 files changed, 143 insertions(+) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 5d91cfd..6150ef0 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -73,6 +73,11 @@ static int nicvf_dev_configure(struct rte_eth_dev *dev); static int nicvf_dev_link_update(struct rte_eth_dev *dev, int wait_to_complete); static void nicvf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info); +static int nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); +static void nicvf_dev_rx_queue_release(void *rx_queue); static int nicvf_dev_get_reg_length(struct rte_eth_dev *dev); static int nicvf_dev_get_regs(struct rte_eth_dev *dev, struct rte_dev_reg_info *regs); @@ -176,6 +181,140 @@ nicvf_dev_get_regs(struct rte_eth_dev *dev, struct rte_dev_reg_info *regs) return -ENOTSUP; } +static int +nicvf_qset_cq_alloc(struct nicvf *nic, struct nicvf_rxq *rxq, uint16_t qidx, + uint32_t desc_cnt) +{ + const struct rte_memzone *rz; + uint32_t ring_size = desc_cnt * sizeof(union cq_entry_t); + + rz = rte_eth_dma_zone_reserve(nic->eth_dev, "cq_ring", qidx, ring_size, + NICVF_CQ_BASE_ALIGN_BYTES, nic->node); + if (rz == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate mem for cq hw ring"); + return -ENOMEM; + } + + memset(rz->addr, 0, ring_size); + + rxq->phys = rz->phys_addr; + rxq->desc = rz->addr; + rxq->qlen_mask = desc_cnt - 1; + + return 0; +} + +static void +nicvf_rx_queue_reset(struct nicvf_rxq *rxq) +{ + rxq->head = 0; + rxq->available_space = 0; + rxq->recv_buffers = 0; +} + +static void +nicvf_dev_rx_queue_release(void *rx_queue) +{ + struct nicvf_rxq *rxq = rx_queue; + + PMD_INIT_FUNC_TRACE(); + + if (rxq) + rte_free(rxq); +} + +static int +nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, + uint16_t nb_desc, unsigned int socket_id, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + uint16_t rx_free_thresh; + struct nicvf_rxq *rxq; + struct nicvf *nic = nicvf_pmd_priv(dev); + + PMD_INIT_FUNC_TRACE(); + + /* Socket id check */ + if (socket_id != (unsigned int)SOCKET_ID_ANY && socket_id != nic->node) + PMD_DRV_LOG(WARNING, "socket_id expected %d, configured %d", + socket_id, nic->node); + + /* Mempool memory should be contiguous */ + if (mp->nb_mem_chunks != 1) { + PMD_INIT_LOG(ERR, "Non contiguous mempool, check huge page sz"); + return -EINVAL; + } + + /* Rx deferred start is not supported */ + if (rx_conf->rx_deferred_start) { + PMD_INIT_LOG(ERR, "Rx deferred start not supported"); + return -EINVAL; + } + + /* Roundup nb_desc to available qsize and validate max number of desc */ + nb_desc = nicvf_qsize_cq_roundup(nb_desc); + if (nb_desc == 0) { + PMD_INIT_LOG(ERR, "Value nb_desc beyond available hw cq qsize"); + return -EINVAL; + } + + /* Check rx_free_thresh upper bound */ + rx_free_thresh = (uint16_t)((rx_conf->rx_free_thresh) ? + rx_conf->rx_free_thresh : + NICVF_DEFAULT_RX_FREE_THRESH); + if (rx_free_thresh > NICVF_MAX_RX_FREE_THRESH || + rx_free_thresh >= nb_desc * .75) { + PMD_INIT_LOG(ERR, "rx_free_thresh greater than expected %d", + rx_free_thresh); + return -EINVAL; + } + + /* Free memory prior to re-allocation if needed */ + if (dev->data->rx_queues[qidx] != NULL) { + PMD_RX_LOG(DEBUG, "Freeing memory prior to re-allocation %d", + qidx); + nicvf_dev_rx_queue_release(dev->data->rx_queues[qidx]); + dev->data->rx_queues[qidx] = NULL; + } + + /* Allocate rxq memory */ + rxq = rte_zmalloc_socket("ethdev rx queue", sizeof(struct nicvf_rxq), + RTE_CACHE_LINE_SIZE, nic->node); + if (rxq == NULL) { + PMD_INIT_LOG(ERR, "Failed to allocate rxq=%d", qidx); + return -ENOMEM; + } + + rxq->nic = nic; + rxq->pool = mp; + rxq->queue_id = qidx; + rxq->port_id = dev->data->port_id; + rxq->rx_free_thresh = rx_free_thresh; + rxq->rx_drop_en = rx_conf->rx_drop_en; + rxq->cq_status = nicvf_qset_base(nic, qidx) + NIC_QSET_CQ_0_7_STATUS; + rxq->cq_door = nicvf_qset_base(nic, qidx) + NIC_QSET_CQ_0_7_DOOR; + rxq->precharge_cnt = 0; + rxq->rbptr_offset = NICVF_CQE_RBPTR_WORD; + + /* Alloc completion queue */ + if (nicvf_qset_cq_alloc(nic, rxq, rxq->queue_id, nb_desc)) { + PMD_INIT_LOG(ERR, "failed to allocate cq %u", rxq->queue_id); + nicvf_dev_rx_queue_release(rxq); + return -ENOMEM; + } + + nicvf_rx_queue_reset(rxq); + + PMD_RX_LOG(DEBUG, "[%d] rxq=%p pool=%s nb_desc=(%d/%d) phy=%" PRIx64, + qidx, rxq, mp->name, nb_desc, + rte_mempool_count(mp), rxq->phys); + + dev->data->rx_queues[qidx] = rxq; + dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + static void nicvf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { @@ -302,6 +441,8 @@ static const struct eth_dev_ops nicvf_eth_dev_ops = { .dev_configure = nicvf_dev_configure, .link_update = nicvf_dev_link_update, .dev_infos_get = nicvf_dev_info_get, + .rx_queue_setup = nicvf_dev_rx_queue_setup, + .rx_queue_release = nicvf_dev_rx_queue_release, .get_reg_length = nicvf_dev_get_reg_length, .get_reg = nicvf_dev_get_regs, }; diff --git a/drivers/net/thunderx/nicvf_ethdev.h b/drivers/net/thunderx/nicvf_ethdev.h index e31657d..afb875a 100644 --- a/drivers/net/thunderx/nicvf_ethdev.h +++ b/drivers/net/thunderx/nicvf_ethdev.h @@ -59,6 +59,8 @@ #define NICVF_DEFAULT_RX_FREE_THRESH 224 #define NICVF_DEFAULT_TX_FREE_THRESH 224 #define NICVF_TX_FREE_MPOOL_THRESH 16 +#define NICVF_MAX_RX_FREE_THRESH 1024 +#define NICVF_MAX_TX_FREE_THRESH 1024 static inline struct nicvf * nicvf_pmd_priv(struct rte_eth_dev *eth_dev) -- 2.5.5