From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0046.outbound.protection.outlook.com [104.47.42.46]) by dpdk.org (Postfix) with ESMTP id 1CFE35939 for ; Fri, 30 Sep 2016 14:07:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pxQ4mDuTacOIZoIuKJ6OFiOuwySz/d9a/n4qedKIrwU=; b=O4ptD9yMojPrC9uwmf5sCLKur5njmE63J8gBb+3XCn+4h+1kNNwcRnNECQk96tCWL83na/BkKJQg84Th0cfWFS/xWMiZh4LBq+eY5tK1feuswvxT88LoeOvu6DNIhwYcMOzeW5tGMOmynPZw7/EeEfYWi8Nuy1LWKQwbUwIDRA4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Kamil.Rytarowski@cavium.com; Received: from cavium1.semihalf.local (31.172.191.173) by CY4PR07MB3063.namprd07.prod.outlook.com (10.172.116.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Fri, 30 Sep 2016 12:06:58 +0000 From: Kamil Rytarowski To: CC: , , , , , , , Kamil Rytarowski Date: Fri, 30 Sep 2016 14:05:49 +0200 Message-ID: <1475237154-25388-11-git-send-email-krytarowski@caviumnetworks.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1475237154-25388-1-git-send-email-krytarowski@caviumnetworks.com> References: <1472230448-17490-1-git-send-email-krytarowski@caviumnetworks.com> <1475237154-25388-1-git-send-email-krytarowski@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [31.172.191.173] X-ClientProxiedBy: VI1PR07CA0001.eurprd07.prod.outlook.com (10.163.160.139) To CY4PR07MB3063.namprd07.prod.outlook.com (10.172.116.144) X-MS-Office365-Filtering-Correlation-Id: dab2ecab-5ddf-4fe3-1e4b-08d3e92a48b7 X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 2:Dk/0bqyAo+YXdyJE9BDq2Z0tHQx+DdxcjzAVmywErrG6/QKuvcUi/DCRaAsoqTarO0nmTiTBaNHRg+w4gKYVHHUo68FzyRsY3gbh3UPYgIk8tLh0236qKadEWgi1SZ8vH1oyQArAVxMJh2Ta8hIi6NHDtxfgmWHIKqrJIlV/IfOiyD4zETPejVCYrCvEM+8P; 3:eZBjMRDkxa3a0/bpoas49eyL+JnXB19ZOzzGmnYqQCFz9rQTwJs2K5SS2iMq2R1i2tI4TSzo1Qy18u9+Cv6YtZ10/m9s/EhRe1TYQmlo1FZl8XdOXuIERex11PWPU3HX; 25:P+BvjUJAx9hPleIQsNAzI8ACie/0NVqJSO8/e749fznMX9Eryt8xcaVx8OAr5v8nvHwmO9gXxlclQ+AT7O+3j1sUBOWhHC0PQVgOJsSGJmFfco62mOXr9j9Q1WO83T6wSeG4RQArb4MtaKix2D/qfSUnj6l2CiVIm0LfEb7lrJzQ74yas7Z4tg9wK82a38hBGHfqhzdcg6XClSnYh6lhQiB0BdpfCllapfFXajAeAJua10oqvLVkjSrlJrJnqQeTPSO7xUraIzHjZgUN50aT0KXXvtjjOgDj3FKyDj89Jbz201HXzdvVYN15p206OG1TQnjDbo57MtpvAcn6dshlVa/GoMwn0GULAtOGf1WsNAFcjoC48p3PEvChgEiz0mBb/mmZnFmVCgv7QzENjQ/wozd+5xMzqZkLbQqrjGO2pHw= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR07MB3063; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 31:eRvVLiPLMtVrL1OjuhC74POPGNw8/6HvUz783YJdJFYFrSjDqHhQwg+SyT6UYL3ga8QhkqRaOXwyJeiSf6QT3EwK8QVDcEnl/7BJnImXJ0NJXmd7LhqsW5O5VT3DcbD8b6/jor+hW3vF9slwSuunCiYUfk2Xyw92/knOZ+4r+d3srOitqjpt8Urvd/prBPA35++0NHa4lpjuNnKwVXIGmOWWzbMRfywpQ1+0BrWDONM=; 20:tOKIy9tZUSrqX4GZUGJBhzqyjxrl4PjAMpbZVCdnF5Lga+u8GAm2z2TgM1S5Yd1QxbK1ELyaLnSR3V+1S7DgkgUoocTfNLATpSaKUS2g5ojDKXKECuhqa4csrQQvubtHuJCyFrYM0F/REdvvQ3mfGoDV65U6aneJ3Z+XO76gmsFtOWvcl4t4qgcy1FgAxkTlQCBxRHmDfx1e7XuZ1u7BOzYCmkIVARoAz5Mq+vy0LqQVtDCLihj/fwJ/m8Q5sPmIZqvP3Mju+pfT8AwrJlSF9UKfz8LNbA44BhMjrg2kIu50f4STu6mlIKSKokvSw2h+8tTGiQEvndC398WWletT+YlozaO6blNtPJzknit4uQHoYiZNRfgK3Ky4hlrPaTYu1l0dbGTt2lPabN5+BIz+mW5Tr7O65ecwSrFhLrRz/M76CVQGKQfohrjLK8eWMRMPDR/2iaTLPj7juX/OiSiY49B681Rvx0KkicRnKChdvz26WLvgLw8wEKaHXrGx0dX9OhDTtw9oOXClKK8TtMfCSuFpACxDA7tv+wJgzhRWCE00E9D6XY1anvLNAE08/9ZRz/gAQlbXX4vKs7Zjtc+bhvEQOGCo08bE0kdL7kFIKxY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:CY4PR07MB3063; BCL:0; PCL:0; RULEID:; SRVR:CY4PR07MB3063; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 4:x7bgaT+D/urW5ye98Mqvmlzu1RAffsPtEhfKDnKCI8NBQyW6UVKf4fKjTOgGectMRhYp1luy8Llg0dIckIPNLIzxZgdKOKX1NFmLTrLgiEdOsTcEzrf58ZRy27pa9fw5C2gdGG3Z1JxYCxRmaWkEylPtY0xjetEB/3SMOLO41+P3H+vDeNtYr9tqCXL0+HwHv3RjLHC4KgMyI7kgFU78ssZgsy1Fu3sMTQIN6ZL361Fx6OP5J4IINlKkMQxxkLwxY9Wm68VOJB2f8edJc609pHnLjeY0182YFT8LjYraLbqnA7P6gVopVu/bJbMLkRxr3y67CidvheROF+6OjYlcNPINdbYft+FXGnG+FJe1vtbiNqmiP4hC6GpzsPItRkzljlZTs9X71BVIry5ERJv7uw== X-Forefront-PRVS: 008184426E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(92566002)(3846002)(66066001)(189998001)(47776003)(42882006)(36756003)(105586002)(19580395003)(97736004)(6916009)(6666003)(68736007)(42186005)(19580405001)(50986999)(50226002)(77096005)(76176999)(5003940100001)(6116002)(7736002)(2950100002)(101416001)(586003)(106356001)(7846002)(5660300001)(4001430100002)(305945005)(33646002)(229853001)(110136003)(8676002)(4326007)(81166006)(5890100001)(107886002)(81156014)(50466002)(2906002)(48376002)(2351001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3063; H:cavium1.semihalf.local; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3063; 23:MVBIv39RwKy0jApPdopNLDp4QkHnxP7RuqNuKx0er?= =?us-ascii?Q?oT97iMxRR0s6F+VF8L+75OHUTa/JFAi/dDwj10Uzeey0WQCxbpil/BIdbjr3?= =?us-ascii?Q?awtmCycmeiH04Q3XYqL9M1VrLyfjB3gziXdrVM8EBxfy8A7EAbox8koe4shS?= =?us-ascii?Q?xKikg+OOAGBrQLDjuLt5c0ws/uY4F0vrN1GTV3mm0jjMn1foeDAk+dZX1k3x?= =?us-ascii?Q?bUOSysuAxi492zMbUolFvGdGDVriLqZjXfwRi77rbE9YxuypvBKdSZa2pkjB?= =?us-ascii?Q?JRO6RcEICgm+H+cz+b2i+SAbTNrlgnTwr+PFJ0rAGT/P0IkwPKQAgT+y6EmR?= =?us-ascii?Q?eB+wPt7PozOtNS4aqmG6a2xZGhPGH1bhle8ZCqvxCwyGVcQAGBNSSaryt2Fs?= =?us-ascii?Q?3zQJ0u+LEC6owSCdLzk5JxsZuq34h7RdgGY6ke6j8z3VAwOlWr/wmQjfzLcL?= =?us-ascii?Q?4UOjUyiPXNILI4asoIKPTFjda4TnItzxf5xj+Hf4hLvJ6KSSK9TvCiVeantu?= =?us-ascii?Q?aTLgh2oXjzA3dLwczNsbMVuP79LiP0P8lqeXQrJnoW07bqlpix4nuYDuT6Mt?= =?us-ascii?Q?1tDIJUR0mjMvoq0fXPkMDK5GJNbYP9jyJE583Eg3gte+qdwYhKHZmKdwND6J?= =?us-ascii?Q?6D4zaxdu66Jkdn0snCqHUk294YFaZ7906AyNxplCUVEk9tA97Ne8REd6O+n2?= =?us-ascii?Q?9xNdRNVR/py0l6EPpPTCqDTSRO7/lc4zdJHj7ebUaoquW8iPQ7CNKElIucN4?= =?us-ascii?Q?iSmrmFhH+iEQPNg5fzxdyUapDdgKiv5AgQ0yRpeoFnpSYdmkdgeXirBDl7nv?= =?us-ascii?Q?x2zh+wwUKgNoT4/FlNnGoZzXufl7C5RqLs928yVIP2CnnZwrkIrr6xy3FFFS?= =?us-ascii?Q?LUd2cQP4UGBQEgkmD2DQRuKYFnAANiIbMTHNhuYk/KQYm/5my9o4wbPXJhzs?= =?us-ascii?Q?NKdZEKKhoKj6DJYlUM0rLiuyz/CBsoSZsd9Zt04NiAWkEYVIiUI3UvyGMT6r?= =?us-ascii?Q?b4Lccr6JNydf9YvKUXwKUxt4vmGjusP9B6ieKk86HTb2jQ3N/LCxx+9eQ3fl?= =?us-ascii?Q?RTSnUT0pklRtKmsXqVlEWWAul2bDcosH+Hz7V69i3h7fhextLZp/nhyEWHBT?= =?us-ascii?Q?c34IZaxxD5QN0ffyIuRjiQhvHEandquIJdfQdA3Rf3fg1FRLCQM8A8lBxflp?= =?us-ascii?Q?wbAud8s47dX7BtyJSpVKA1oKWww8XiA+9J/?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 6:QlUPt2B6soO1E69aFy/j/yqk+kdeXwWoAyraUIYIdswmQf/LY+ZpM0dW/lWnnxWYbwz3tg4Q+518eBs+DHHMi3OhZTyNqcSHvKalSy2jmik8Bc+cBIo6nk/abEKpYQTstX7Sp4gIztChvuXz3Yob0+TpC/tnhAI//blh7cyP5K8pUsxIDpEdrYKNF07HZdoMPcG6nsi+ZISCq1RDROHH7ZAR7iF9UtOSw17LJjKYrIo7LBNTlOqkUZLxDsv0wvOcJl71GDq2jXhs0BkUJgV/F0Bi+Ja0TrVQH2I1XMQQ6ms=; 5:Sx7fbvpamI3vn9yg+XMO1Yp8PxtHcCn/MSMD1KilxUtP70g6OVFnWRalqZzHN0T7eCajnJ0BlMzircl/okfx40DlhnU+5usoGEKTRmO+jTCmOk3hEAES8IZTwiR6w04mLsnKlOe9MHjJhUVBV/vZMw==; 24:sQ33eVlFga+nvHu2533NqpcGIczzanpoUGYoHshub0FnxUruIX7EDOYDzoYxyHpWVMjIujUJ/CrAdcbUAuuauXCNfLlSc1i51aHn15XL4AQ=; 7:XdhuPfzO34OIaCpkFiUWVE//N341t4rzo2g8WkMvargD4ON3hWZ0QlArUna2lcMiigDqcPieb20URuzJV3HN/uAGRfXdG8yQI0MGvnJzYscmclnroL91lN8EuxxiuBlzmbwZQVoyzjVFiFYRzosnysGFlWXZIIimfbbyxZCRFfjSCVOzzmIXXab/1PICsemaN+3JmyxAqa1WPWXdFu6NsSAEsepSjEst2A5/xJmq/JFZTgvFOqJOGWJgGGiZRVWh7VoqJwG+CTt5LOT6D5uWB9NkASWQHWOUh7oRhBedl+T46+rTgk/Zmd9fnyAsaghk SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2016 12:06:58.9986 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3063 Subject: [dpdk-dev] [PATCH v2 10/15] net/thunderx: add secondary qset support in device start X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 12:07:03 -0000 From: Kamil Rytarowski Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki Signed-off-by: Jerin Jacob --- drivers/net/thunderx/nicvf_ethdev.c | 266 +++++++++++++++++++++++------------- 1 file changed, 172 insertions(+), 94 deletions(-) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 7ca7d07..f7d5188 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -653,27 +653,33 @@ nicvf_tx_queue_reset(struct nicvf_txq *txq) } static inline int -nicvf_start_tx_queue(struct rte_eth_dev *dev, uint16_t qidx) +nicvf_vf_start_tx_queue(struct rte_eth_dev *dev, struct nicvf *nic, + uint16_t qidx) { struct nicvf_txq *txq; int ret; - if (dev->data->tx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + assert(qidx < MAX_SND_QUEUES_PER_QS); + + if (dev->data->tx_queue_state[nicvf_netdev_qidx(nic, qidx)] == + RTE_ETH_QUEUE_STATE_STARTED) return 0; - txq = dev->data->tx_queues[qidx]; + txq = dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)]; txq->pool = NULL; - ret = nicvf_qset_sq_config(nicvf_pmd_priv(dev), qidx, txq); + ret = nicvf_qset_sq_config(nic, qidx, txq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure sq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure sq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_sq_error; } - dev->data->tx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + dev->data->tx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STARTED; return ret; config_sq_error: - nicvf_qset_sq_reclaim(nicvf_pmd_priv(dev), qidx); + nicvf_qset_sq_reclaim(nic, qidx); return ret; } @@ -977,31 +983,37 @@ nicvf_rx_queue_reset(struct nicvf_rxq *rxq) } static inline int -nicvf_start_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +nicvf_vf_start_rx_queue(struct rte_eth_dev *dev, struct nicvf *nic, + uint16_t qidx) { - struct nicvf *nic = nicvf_pmd_priv(dev); struct nicvf_rxq *rxq; int ret; - if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + assert(qidx < MAX_RCV_QUEUES_PER_QS); + + if (dev->data->rx_queue_state[nicvf_netdev_qidx(nic, qidx)] == + RTE_ETH_QUEUE_STATE_STARTED) return 0; /* Update rbdr pointer to all rxq */ - rxq = dev->data->rx_queues[qidx]; + rxq = dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)]; rxq->shared_rbdr = nic->rbdr; ret = nicvf_qset_rq_config(nic, qidx, rxq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure rq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure rq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_rq_error; } ret = nicvf_qset_cq_config(nic, qidx, rxq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure cq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure cq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_cq_error; } - dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + dev->data->rx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STARTED; return 0; config_cq_error: @@ -1054,9 +1066,15 @@ nicvf_dev_rx_queue_release(void *rx_queue) static int nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) { + struct nicvf *nic = nicvf_pmd_priv(dev); int ret; - ret = nicvf_start_rx_queue(dev, qidx); + if (qidx >= MAX_RCV_QUEUES_PER_QS) + nic = nic->snicvf[(qidx / MAX_RCV_QUEUES_PER_QS - 1)]; + + qidx = qidx % MAX_RCV_QUEUES_PER_QS; + + ret = nicvf_vf_start_rx_queue(dev, nic, qidx); if (ret) return ret; @@ -1087,7 +1105,14 @@ nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx) static int nicvf_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) { - return nicvf_start_tx_queue(dev, qidx); + struct nicvf *nic = nicvf_pmd_priv(dev); + + if (qidx >= MAX_SND_QUEUES_PER_QS) + nic = nic->snicvf[(qidx / MAX_SND_QUEUES_PER_QS - 1)]; + + qidx = qidx % MAX_SND_QUEUES_PER_QS; + + return nicvf_vf_start_tx_queue(dev, nic, qidx); } static int @@ -1274,25 +1299,25 @@ rbdr_rte_mempool_get(void *dev, void *opaque) } static int -nicvf_dev_start(struct rte_eth_dev *dev) +nicvf_vf_start(struct rte_eth_dev *dev, struct nicvf *nic, uint32_t rbdrsz) { int ret; uint16_t qidx; - uint32_t buffsz = 0, rbdrsz = 0; uint32_t total_rxq_desc, nb_rbdr_desc, exp_buffs; uint64_t mbuf_phys_off = 0; struct nicvf_rxq *rxq; - struct rte_pktmbuf_pool_private *mbp_priv; struct rte_mbuf *mbuf; - struct nicvf *nic = nicvf_pmd_priv(dev); - struct rte_eth_rxmode *rx_conf = &dev->data->dev_conf.rxmode; - uint16_t mtu; + uint16_t rx_start, rx_end; + uint16_t tx_start, tx_end; PMD_INIT_FUNC_TRACE(); /* Userspace process exited without proper shutdown in last run */ if (nicvf_qset_rbdr_active(nic, 0)) - nicvf_dev_stop(dev); + nicvf_vf_stop(dev, nic, false); + + /* Get queue ranges for this VF */ + nicvf_rx_range(dev, nic, &rx_start, &rx_end); /* * Thunderx nicvf PMD can support more than one pool per port only when @@ -1307,32 +1332,15 @@ nicvf_dev_start(struct rte_eth_dev *dev) * */ - /* Validate RBDR buff size */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { - rxq = dev->data->rx_queues[qidx]; - mbp_priv = rte_mempool_get_priv(rxq->pool); - buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; - if (buffsz % 128) { - PMD_INIT_LOG(ERR, "rxbuf size must be multiply of 128"); - return -EINVAL; - } - if (rbdrsz == 0) - rbdrsz = buffsz; - if (rbdrsz != buffsz) { - PMD_INIT_LOG(ERR, "buffsz not same, qid=%d (%d/%d)", - qidx, rbdrsz, buffsz); - return -EINVAL; - } - } - /* Validate mempool attributes */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = dev->data->rx_queues[qidx]; rxq->mbuf_phys_off = nicvf_mempool_phy_offset(rxq->pool); mbuf = rte_pktmbuf_alloc(rxq->pool); if (mbuf == NULL) { - PMD_INIT_LOG(ERR, "Failed allocate mbuf qid=%d pool=%s", - qidx, rxq->pool->name); + PMD_INIT_LOG(ERR, "Failed allocate mbuf VF%d qid=%d " + "pool=%s", + nic->vf_id, qidx, rxq->pool->name); return -ENOMEM; } rxq->mbuf_phys_off -= nicvf_mbuff_meta_length(mbuf); @@ -1342,15 +1350,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) if (mbuf_phys_off == 0) mbuf_phys_off = rxq->mbuf_phys_off; if (mbuf_phys_off != rxq->mbuf_phys_off) { - PMD_INIT_LOG(ERR, "pool params not same,%s %" PRIx64, - rxq->pool->name, mbuf_phys_off); + PMD_INIT_LOG(ERR, "pool params not same,%s VF%d %" + PRIx64, rxq->pool->name, nic->vf_id, + mbuf_phys_off); return -EINVAL; } } /* Check the level of buffers in the pool */ total_rxq_desc = 0; - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = dev->data->rx_queues[qidx]; /* Count total numbers of rxq descs */ total_rxq_desc += rxq->qlen_mask + 1; @@ -1368,14 +1377,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) /* Check RBDR desc overflow */ ret = nicvf_qsize_rbdr_roundup(total_rxq_desc); if (ret == 0) { - PMD_INIT_LOG(ERR, "Reached RBDR desc limit, reduce nr desc"); + PMD_INIT_LOG(ERR, "Reached RBDR desc limit, reduce nr desc " + "VF%d", nic->vf_id); return -ENOMEM; } /* Enable qset */ ret = nicvf_qset_config(nic); if (ret) { - PMD_INIT_LOG(ERR, "Failed to enable qset %d", ret); + PMD_INIT_LOG(ERR, "Failed to enable qset %d VF%d", ret, + nic->vf_id); return ret; } @@ -1383,14 +1394,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) nb_rbdr_desc = nicvf_qsize_rbdr_roundup(total_rxq_desc); ret = nicvf_qset_rbdr_alloc(dev, nic, nb_rbdr_desc, rbdrsz); if (ret) { - PMD_INIT_LOG(ERR, "Failed to allocate memory for rbdr alloc"); + PMD_INIT_LOG(ERR, "Failed to allocate memory for rbdr alloc " + "VF%d", nic->vf_id); goto qset_reclaim; } /* Enable and configure RBDR registers */ ret = nicvf_qset_rbdr_config(nic, 0); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure rbdr %d", ret); + PMD_INIT_LOG(ERR, "Failed to configure rbdr %d VF%d", ret, + nic->vf_id); goto qset_rbdr_free; } @@ -1398,52 +1411,127 @@ nicvf_dev_start(struct rte_eth_dev *dev) ret = nicvf_qset_rbdr_precharge(dev, nic, 0, rbdr_rte_mempool_get, total_rxq_desc); if (ret) { - PMD_INIT_LOG(ERR, "Failed to fill rbdr %d", ret); + PMD_INIT_LOG(ERR, "Failed to fill rbdr %d VF%d", ret, + nic->vf_id); goto qset_rbdr_reclaim; } - PMD_DRV_LOG(INFO, "Filled %d out of %d entries in RBDR", - nic->rbdr->tail, nb_rbdr_desc); + PMD_DRV_LOG(INFO, "Filled %d out of %d entries in RBDR VF%d", + nic->rbdr->tail, nb_rbdr_desc, nic->vf_id); + + /* Configure VLAN Strip */ + nicvf_vlan_hw_strip(nic, dev->data->dev_conf.rxmode.hw_vlan_strip); + + /* Get queue ranges for this VF */ + nicvf_tx_range(dev, nic, &tx_start, &tx_end); + + /* Configure TX queues */ + for (qidx = tx_start; qidx <= tx_end; qidx++) { + ret = nicvf_vf_start_tx_queue(dev, nic, + qidx % MAX_SND_QUEUES_PER_QS); + if (ret) + goto start_txq_error; + } /* Configure RX queues */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { - ret = nicvf_start_rx_queue(dev, qidx); + for (qidx = rx_start; qidx <= rx_end; qidx++) { + ret = nicvf_vf_start_rx_queue(dev, nic, + qidx % MAX_RCV_QUEUES_PER_QS); if (ret) goto start_rxq_error; } - /* Configure VLAN Strip */ - nicvf_vlan_hw_strip(nic, dev->data->dev_conf.rxmode.hw_vlan_strip); - - /* Configure TX queues */ - for (qidx = 0; qidx < dev->data->nb_tx_queues; qidx++) { - ret = nicvf_start_tx_queue(dev, qidx); + if (!nic->sqs_mode) { + /* Configure CPI algorithm */ + ret = nicvf_configure_cpi(dev); if (ret) goto start_txq_error; + + ret = nicvf_mbox_get_rss_size(nic); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to get rss table size"); + goto qset_rss_error; + } + + /* Configure RSS */ + ret = nicvf_configure_rss(dev); + if (ret) + goto qset_rss_error; } - /* Configure CPI algorithm */ - ret = nicvf_configure_cpi(dev); - if (ret) - goto start_txq_error; + /* Done; Let PF make the BGX's RX and TX switches to ON position */ + nicvf_mbox_cfg_done(nic); + return 0; - /* Configure RSS */ - ret = nicvf_configure_rss(dev); - if (ret) - goto qset_rss_error; +qset_rss_error: + nicvf_rss_term(nic); +start_rxq_error: + for (qidx = rx_start; qidx <= rx_end; qidx++) + nicvf_vf_stop_rx_queue(dev, nic, qidx % MAX_RCV_QUEUES_PER_QS); +start_txq_error: + for (qidx = tx_start; qidx <= tx_end; qidx++) + nicvf_vf_stop_tx_queue(dev, nic, qidx % MAX_SND_QUEUES_PER_QS); +qset_rbdr_reclaim: + nicvf_qset_rbdr_reclaim(nic, 0); + nicvf_rbdr_release_mbufs(dev, nic); +qset_rbdr_free: + if (nic->rbdr) { + rte_free(nic->rbdr); + nic->rbdr = NULL; + } +qset_reclaim: + nicvf_qset_reclaim(nic); + return ret; +} + +static int +nicvf_dev_start(struct rte_eth_dev *dev) +{ + uint16_t qidx; + int ret; + size_t i; + struct nicvf *nic = nicvf_pmd_priv(dev); + struct rte_eth_rxmode *rx_conf = &dev->data->dev_conf.rxmode; + uint16_t mtu; + uint32_t buffsz = 0, rbdrsz = 0; + struct rte_pktmbuf_pool_private *mbp_priv; + struct nicvf_rxq *rxq; + + PMD_INIT_FUNC_TRACE(); + + /* This function must be called for a primary device */ + assert_primary(nic); + + /* Validate RBDR buff size */ + for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + rxq = dev->data->rx_queues[qidx]; + mbp_priv = rte_mempool_get_priv(rxq->pool); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + if (buffsz % 128) { + PMD_INIT_LOG(ERR, "rxbuf size must be multiply of 128"); + return -EINVAL; + } + if (rbdrsz == 0) + rbdrsz = buffsz; + if (rbdrsz != buffsz) { + PMD_INIT_LOG(ERR, "buffsz not same, qidx=%d (%d/%d)", + qidx, rbdrsz, buffsz); + return -EINVAL; + } + } /* Configure loopback */ ret = nicvf_loopback_config(nic, dev->data->dev_conf.lpbk_mode); if (ret) { PMD_INIT_LOG(ERR, "Failed to configure loopback %d", ret); - goto qset_rss_error; + return ret; } /* Reset all statistics counters attached to this port */ ret = nicvf_mbox_reset_stat_counters(nic, 0x3FFF, 0x1F, 0xFFFF, 0xFFFF); if (ret) { PMD_INIT_LOG(ERR, "Failed to reset stat counters %d", ret); - goto qset_rss_error; + return ret; } /* Setup scatter mode if needed by jumbo */ @@ -1464,33 +1552,23 @@ nicvf_dev_start(struct rte_eth_dev *dev) return -EBUSY; } + ret = nicvf_vf_start(dev, nic, rbdrsz); + if (ret != 0) + return ret; + + for (i = 0; i < nic->sqs_count; i++) { + assert(nic->snicvf[i]); + + ret = nicvf_vf_start(dev, nic->snicvf[i], rbdrsz); + if (ret != 0) + return ret; + } + /* Configure callbacks based on scatter mode */ nicvf_set_tx_function(dev); nicvf_set_rx_function(dev); - /* Done; Let PF make the BGX's RX and TX switches to ON position */ - nicvf_mbox_cfg_done(nic); return 0; - -qset_rss_error: - nicvf_rss_term(nic); -start_txq_error: - for (qidx = 0; qidx < dev->data->nb_tx_queues; qidx++) - nicvf_vf_stop_tx_queue(dev, nic, qidx); -start_rxq_error: - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) - nicvf_vf_stop_rx_queue(dev, nic, qidx); -qset_rbdr_reclaim: - nicvf_qset_rbdr_reclaim(nic, 0); - nicvf_rbdr_release_mbufs(dev, nic); -qset_rbdr_free: - if (nic->rbdr) { - rte_free(nic->rbdr); - nic->rbdr = NULL; - } -qset_reclaim: - nicvf_qset_reclaim(nic); - return ret; } static void -- 1.9.1