From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0051.outbound.protection.outlook.com [104.47.40.51]) by dpdk.org (Postfix) with ESMTP id 4111E5AB0 for ; Fri, 26 Aug 2016 18:55:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mnl2jDggWfsSiij/oPl3ZWx2dXWLmqCRGAtInCOvdo0=; b=VqNTPb0z5dzLlME9CvRyUAq5OMuV59CgyfZi6g8/ckZ4gijDfI7NHHTBJTE65HkLhIwDN/Je/X8+D/7lO37wfN08x0zsbsZcmkZ8sIVWCSsG3fbF6wz9bRcdZNXr8A6Nbqece3f+LDU2Vq0yH7NWh50Tuq93eYE/awv0pMWXXBk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Kamil.Rytarowski@cavium.com; Received: from cavium1.semihalf.local (31.172.191.173) by CY4PR07MB3064.namprd07.prod.outlook.com (10.172.116.145) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.549.15; Fri, 26 Aug 2016 16:55:13 +0000 From: Kamil Rytarowski To: CC: , , , , , Kamil Rytarowski Date: Fri, 26 Aug 2016 18:54:05 +0200 Message-ID: <1472230448-17490-11-git-send-email-krytarowski@caviumnetworks.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1472230448-17490-1-git-send-email-krytarowski@caviumnetworks.com> References: <1472230448-17490-1-git-send-email-krytarowski@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [31.172.191.173] X-ClientProxiedBy: HE1PR0201CA0008.eurprd02.prod.outlook.com (10.165.229.18) To CY4PR07MB3064.namprd07.prod.outlook.com (10.172.116.145) X-MS-Office365-Filtering-Correlation-Id: 764c6ff0-881f-4e2f-168e-08d3cdd1c064 X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3064; 2:bsyZ7D4pP5cXTS78KNUkTX1WS99m+8KQGyN/ZAigHSl5NJL9E55eN5FNX/SKLHyBUVotbpol7OQDINdoHnKFyabrdG81vlYEaznSDUJ3nVJ9L//IUFeb31c5zPxVq9/szkteY1cr4H5C/65BrN9qCjCcX4AUaUwDUvNSbGKDJGLDPDZBQpzxSEMLeXGv3lCS; 3:T/YtwMJBB/PbILuXe4He6oGikbDZGC31myHKkyceKkaLrELrO7SMSGrgbITsGVqARffSFXwCFPJOmJoTYefyaGdfBc3Cv3iBDUb7YLHE8F+Pph5m2kaqZp76H26+kRQa X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR07MB3064; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3064; 25:Z7VU/Nz9734SGnDHId3BfwsqXJN81YwtV7Wd64vDgjUwinz10S1/TgezbB+akETQ3YcfDoVz4nUXMcLWKEcUb++CSK8K5QK5dQkaLQg9jypynSOFx6TMVj+SG9NnKLIS3U6q4UFjKtEVagYGR+kZ+Hzwu3PZMUzjn7ZL18XGhFlppcYZIy2fSLxQlm16zIlCi2Gm3220Eschx9yXGB7dlBEyBJnyV+KfbvNvB22Qij+0C6Z1oMkUEOYB+82sE5oaUN5TXyZ5K4gFw5sN2fc+BRc6FM/LmFzwnsVURq+XmsumQaLx4+Ok9PCi5ZtdErc2D0TmIyRAguZm08h/aG38y7O82PPS6aTna+dBnrYrF3LbwCie0MLUz/h8unp4GBN0t6X3Z0w48TcuIlrv0am5pCEKvlgvp2dPtVhIVvD4a21j7Fw3QZ9ObH2KWxzh+ZqiR7vgwoVDwYJ0qkc9q+tbd2v8KM4MjrSWPU6y4Bn8x69o/nkEO8iD6GJV+Acref1YoZP4OpA8D4kFlUSaJddGUY7eJ13kPEhy8wjR7WPjiFlIDnW6lfhKqDwYYQRPkgt0vp+W6lxfSKfEM58Zt6yfR3XdIZLC+rB7By60GTIExK1RR2XTOC6BwsPaO3ny2tEXsr6TAfFUrnMVnwzQfnMsUguez/XLpz7BXpTatmdaDHJXnejia24DNRYpyLsmD7cki4mMGN+a8g5gh2B7buqC7y4ST1RXCWpgi2ruzWdAtj4GcDO7DAWAgEUAxsBQ4J3c X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3064; 31:lWzBPVVzd6lOJUEJHPP1eA6n8qoV3OF7aVj4mMOYu3EbonDJgA7FoEsawXI2rF4tQE8ho5Zfv6QrYAUyknsmoceza5/4rAqqbBMtSZ+mCveGxCm2EzjE3XhRF5Tp5eSbHz8ORYoJ/w1/5UrOZt3vuBpUYpDyLrA7FGNTuFNx0kb0JvwY/vhAf1dMaQHoXzUWVfva8aLa5bWEfIA2cYK4eL4iKmz6zfjSNZJi/VSpkw4=; 20:XJfHIDBZfz3z+Z4bqSzUDHKeo3ZgpGv54TPf9nFvtLqAyntbnxA4a7cfewRdMJjZ41lFWeDdpY382Ww4wlP0pseknB7m1R6meF+ej1tHbZliieoSX7wdI9SVyr/4QjHFoolsQ+9Z6P/k3Gl7xC/pZoEuDh0I13xtY82BWBihzmZMvokYugUsYoztQLB0u+O5soRjYHXK7PXkZkiOb+HLQUeRp+1J5g4zquno/odrzoOzp9Cv208QDLZXK4T8A+RMnZKRQHlVqvG8j1dd1o7bh+y6EdTSmBfOr1e4080L9jnN9IPcuxn51S2H1zugeqiwXLVjPwb35VL1K4CEsQkpkmuzrl7od3F30W43N13rSTpFHt3jFvCMkdQ8pvVJSObKxt/xZ9psQRsllfErumxbOJJ4u/BBMgjXNBkl6BDkzAxlVo9u8S1rfqtVuVsy/Q5nx0WNfQhgzAVvUAH5C3BhCKrWuCAF7CZAGwVVexELV10BkshJl88l2PUUPglykD7IGqwvegi+A5KFfzP3jUNxxaSeKHDnQsVenUnWXZH+buQM1horn0AkzIF7/Q8LNAk5hzW84PvSpD+YWhnSevOsRee5mj6LY+FDnJqxcADqBss= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001); SRVR:CY4PR07MB3064; BCL:0; PCL:0; RULEID:; SRVR:CY4PR07MB3064; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3064; 4:+N2P1t/IfK3oszon01jYRSJiFbHrbpR1bKFkY9DQogqt8nnq73jQbL2yTcqU75yGFv1YfZZBuNlkkS6pUuVBAtJvyiHMLji4hgTx02CoEV7ZsnUN9tB7mIjTgboOif91o5xenSZVLTrfkKd+oEal5ph4mvUlxn+bG9ONK4WopMY5ZjCPLX245jG98O7r2bgjlgKsmCNZF+MAuCGimF0e/ccIEYdTA2k1KNli627ny5O74VnCyExvey7sM4Nei6g5fDsAD/c9H3pyYeQ5Icfdlb5cAG5rBWS8OptpBVOFd/Jadvqg2DKyk55AMsMIn7++zcxTUTXxuk5fhZcfw3yo5EzMqxLEXiUSDubmnpv8hmpye4ls0ZtVgoYFFVy2K/A64k7bxaLkSKLlNzOpZ/J9jg== X-Forefront-PRVS: 00462943DE X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(81156014)(36756003)(77096005)(189998001)(2906002)(229853001)(2351001)(105586002)(8676002)(101416001)(92566002)(5660300001)(81166006)(6116002)(4001430100002)(2950100001)(7736002)(47776003)(4326007)(66066001)(586003)(50466002)(106356001)(50226002)(5003940100001)(19580405001)(19580395003)(305945005)(50986999)(110136002)(76176999)(7846002)(48376002)(68736007)(42186005)(5890100001)(3846002)(107886002)(33646002)(97736004); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3064; H:cavium1.semihalf.local; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3064; 23:5KPGchWyygfG/YCzIJUGk89RjrZPzs33zHDZ6/FeN?= =?us-ascii?Q?ObnVm/dfWYAJBkS9vbRHx5er0RbZBUeEeP7iWSfd8EEcwV1PeYb2RMBJF+Lk?= =?us-ascii?Q?ZCzz6g3+XXCJG64y1p2jwHPYbFWj9ZtgVfVMeUn6wlHFKDlrb7JzfSJ5Z+JH?= =?us-ascii?Q?USUF0zo5siQp7fkRsWPMkgZGSzUiIr+XI1HUj6p0+piZiOm+qL+iGmX2i/aJ?= =?us-ascii?Q?SzuBG4YtPNiZ3QQlDZJLWxeu6EkXV1f5obXEIRkATMxGn2MMwO7l/OM0asgK?= =?us-ascii?Q?tx0+oaP6Uj9LghHRsNOtJzTH3jjvzRfFDp6zI+mcmK/qm/V4C5ggB3VKOHMp?= =?us-ascii?Q?HbLxsZH/TA3JI+I0++lomihgLyEbuyVl38kbhvbvg0FK3wYn2xZp+K4AWo3/?= =?us-ascii?Q?WflEFDSwtMVGNB4E2L0MclwjQgmulnATg3gf9Tixg870TTaMbq4uhAsVDdyy?= =?us-ascii?Q?t479SLVIbLyodIVtKMmHuY81D+VgMSJnqlma/+wI8o2TffDWpdcXIzO2LTqM?= =?us-ascii?Q?bqy2yWNcb89FerztCydfof9gjdtooOKnL6cMKhDb311NjL1jdOsNUyDDUtDx?= =?us-ascii?Q?FUdUOhu1G0lmum2iIPIFklAFmI8aSvRJ8/EmRDKumZmVeiej9wLLbkJ7+kAc?= =?us-ascii?Q?2AFPKfLOD31T+KlAhKERRNueJMAjkVB+Cm2RQVGOWGgDxthcYJgdBnUSDlvH?= =?us-ascii?Q?UIa2nCXQ6SsEIUaOK6666enCjCHuFkHjOFbFtPtR39EbRNqmroXpIaPGG579?= =?us-ascii?Q?GgswGJ00KWiV+zw/Zp0ckolWNxnW8uy7FBNtU+bNhSDITmUhzdUDaYB+NF1O?= =?us-ascii?Q?Me46WW+WvH7C5YAj1Hdu8PlRGc1EcXG1eDdtYb/OXJaHgmuD6o2Nroz8kpBx?= =?us-ascii?Q?o7iAGONyWit7sGNlor/807ObncfgU922wG1t/sucijkbfhMVNAuRlI+dlVcf?= =?us-ascii?Q?M+fqI07Ao8nQz65J4obYDrs8lWyKwOKtuqyaGdL/1G9kBZ5NnUWdycag0LRQ?= =?us-ascii?Q?jMrZFouAS+oYyKKfVbK2rJqlfX9r4zAhEl6ZVl/Uos6x06FBQUBNI0/POpTh?= =?us-ascii?Q?JYMRSZa31X3l4dhNLiGup/fIgLDO8IgPcdowGHGyLDh/2W3/XXu0OZnmnf3M?= =?us-ascii?Q?HRUSiXRYj0v5RmtAiB9vf6Gi//KtTO3WQh+rOBnyv7JuF4FCOnO7A=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3064; 6:XqXpcwFkni/JKJTJCSiVsempyk8nkptb3Ymb1MgEsWE7M+VNjEuBSg9tCmqD/sGBEr89lQaCv32d1DqHyhWb/kOl18mx78W17wkEUTQObL8Kgm7XjOHMXeE/M3YehpHHrAQ8UZookBOPK7sxkX4zC0D1UXAPomQjwFCuhj1K5HWLThZVRXggLM/z74g/LIBEtoF+VKuEH55SIN+n4gTsHRcLiUXoHIYlqs26eD1km0i0pDf9BLEICO/DnO0q6nTUcqCUf6HH1iYTdnGH0uemUp/236NNZxHTZE/dTA6+Phc=; 5:ZTWoUeKOEqlSCur+Iy3jQuSFNxn+olS9GjPy5kbONI2QW4ciQoJtiHjDeXWDVubYz5LWzJBMZvjILbDkwzxBV+5iVI+BCBZIuj1lB9Iz2MRUqM3ARBjX5MHH2wlnXoVD3NqqrdZ9etkCE8MEsuFHDg==; 24:0U6GMlHqIRPIj2+8GvVkpiMe0dDAixC8FZM9sD6HBtw+fzT3MeJVt8MnDLgEJkJuzQbA6RkXN5lH5JGyOBWbKdvzoOLu3202jEgLKskfZD0=; 7:7HN3gH5W0pAZ7m2L0+J3rOwTetTlwICHdQQ+m8ONZ5l3ir+OMPgEOnN5rzIoCKWAmIpooLgbcGspFLt2w8V1XbuIB+0LdNLuEMGc0oxU5cOsNZRhBBysvqjVh6lWDQvfGeHYaYCPNit3cVKA070HN7Ek+3E+p5Dydv9n3KyJBXbKnube2bAjlnfriEmxQS+NklHxiuEZvYwD4CJYP43Qo7bq0rEoWB0naGkRLORn++fWMuaGbejyqFmhk43u+88Y SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2016 16:55:13.0450 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3064 Subject: [dpdk-dev] [PATCH 10/13] net/thunderx: add secondary qset support in device start X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2016 16:55:18 -0000 From: Kamil Rytarowski Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki Signed-off-by: Jerin Jacob --- drivers/net/thunderx/nicvf_ethdev.c | 266 +++++++++++++++++++++++------------- 1 file changed, 172 insertions(+), 94 deletions(-) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index c9c5b78..8510675 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -653,27 +653,33 @@ nicvf_tx_queue_reset(struct nicvf_txq *txq) } static inline int -nicvf_start_tx_queue(struct rte_eth_dev *dev, uint16_t qidx) +nicvf_vf_start_tx_queue(struct rte_eth_dev *dev, struct nicvf *nic, + uint16_t qidx) { struct nicvf_txq *txq; int ret; - if (dev->data->tx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + assert(qidx < MAX_SND_QUEUES_PER_QS); + + if (dev->data->tx_queue_state[nicvf_netdev_qidx(nic, qidx)] == + RTE_ETH_QUEUE_STATE_STARTED) return 0; - txq = dev->data->tx_queues[qidx]; + txq = dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)]; txq->pool = NULL; - ret = nicvf_qset_sq_config(nicvf_pmd_priv(dev), qidx, txq); + ret = nicvf_qset_sq_config(nic, qidx, txq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure sq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure sq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_sq_error; } - dev->data->tx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + dev->data->tx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STARTED; return ret; config_sq_error: - nicvf_qset_sq_reclaim(nicvf_pmd_priv(dev), qidx); + nicvf_qset_sq_reclaim(nic, qidx); return ret; } @@ -977,31 +983,37 @@ nicvf_rx_queue_reset(struct nicvf_rxq *rxq) } static inline int -nicvf_start_rx_queue(struct rte_eth_dev *dev, uint16_t qidx) +nicvf_vf_start_rx_queue(struct rte_eth_dev *dev, struct nicvf *nic, + uint16_t qidx) { - struct nicvf *nic = nicvf_pmd_priv(dev); struct nicvf_rxq *rxq; int ret; - if (dev->data->rx_queue_state[qidx] == RTE_ETH_QUEUE_STATE_STARTED) + assert(qidx < MAX_RCV_QUEUES_PER_QS); + + if (dev->data->rx_queue_state[nicvf_netdev_qidx(nic, qidx)] == + RTE_ETH_QUEUE_STATE_STARTED) return 0; /* Update rbdr pointer to all rxq */ - rxq = dev->data->rx_queues[qidx]; + rxq = dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)]; rxq->shared_rbdr = nic->rbdr; ret = nicvf_qset_rq_config(nic, qidx, rxq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure rq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure rq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_rq_error; } ret = nicvf_qset_cq_config(nic, qidx, rxq); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure cq %d %d", qidx, ret); + PMD_INIT_LOG(ERR, "Failed to configure cq VF%d %d %d", + nic->vf_id, qidx, ret); goto config_cq_error; } - dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STARTED; + dev->data->rx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STARTED; return 0; config_cq_error: @@ -1054,9 +1066,15 @@ nicvf_dev_rx_queue_release(void *rx_queue) static int nicvf_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) { + struct nicvf *nic = nicvf_pmd_priv(dev); int ret; - ret = nicvf_start_rx_queue(dev, qidx); + if (qidx >= MAX_RCV_QUEUES_PER_QS) + nic = nic->snicvf[(qidx / MAX_RCV_QUEUES_PER_QS - 1)]; + + qidx = qidx % MAX_RCV_QUEUES_PER_QS; + + ret = nicvf_vf_start_rx_queue(dev, nic, qidx); if (ret) return ret; @@ -1087,7 +1105,14 @@ nicvf_dev_rx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx) static int nicvf_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t qidx) { - return nicvf_start_tx_queue(dev, qidx); + struct nicvf *nic = nicvf_pmd_priv(dev); + + if (qidx >= MAX_SND_QUEUES_PER_QS) + nic = nic->snicvf[(qidx / MAX_SND_QUEUES_PER_QS - 1)]; + + qidx = qidx % MAX_SND_QUEUES_PER_QS; + + return nicvf_vf_start_tx_queue(dev, nic, qidx); } static int @@ -1267,25 +1292,25 @@ rbdr_rte_mempool_get(void *dev, void *opaque) } static int -nicvf_dev_start(struct rte_eth_dev *dev) +nicvf_vf_start(struct rte_eth_dev *dev, struct nicvf *nic, uint32_t rbdrsz) { int ret; uint16_t qidx; - uint32_t buffsz = 0, rbdrsz = 0; uint32_t total_rxq_desc, nb_rbdr_desc, exp_buffs; uint64_t mbuf_phys_off = 0; struct nicvf_rxq *rxq; - struct rte_pktmbuf_pool_private *mbp_priv; struct rte_mbuf *mbuf; - struct nicvf *nic = nicvf_pmd_priv(dev); - struct rte_eth_rxmode *rx_conf = &dev->data->dev_conf.rxmode; - uint16_t mtu; + uint16_t rx_start, rx_end; + uint16_t tx_start, tx_end; PMD_INIT_FUNC_TRACE(); /* Userspace process exited without proper shutdown in last run */ if (nicvf_qset_rbdr_active(nic, 0)) - nicvf_dev_stop(dev); + nicvf_vf_stop(dev, nic, false); + + /* Get queue ranges for this VF */ + nicvf_rx_range(dev, nic, &rx_start, &rx_end); /* * Thunderx nicvf PMD can support more than one pool per port only when @@ -1300,32 +1325,15 @@ nicvf_dev_start(struct rte_eth_dev *dev) * */ - /* Validate RBDR buff size */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { - rxq = dev->data->rx_queues[qidx]; - mbp_priv = rte_mempool_get_priv(rxq->pool); - buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; - if (buffsz % 128) { - PMD_INIT_LOG(ERR, "rxbuf size must be multiply of 128"); - return -EINVAL; - } - if (rbdrsz == 0) - rbdrsz = buffsz; - if (rbdrsz != buffsz) { - PMD_INIT_LOG(ERR, "buffsz not same, qid=%d (%d/%d)", - qidx, rbdrsz, buffsz); - return -EINVAL; - } - } - /* Validate mempool attributes */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = dev->data->rx_queues[qidx]; rxq->mbuf_phys_off = nicvf_mempool_phy_offset(rxq->pool); mbuf = rte_pktmbuf_alloc(rxq->pool); if (mbuf == NULL) { - PMD_INIT_LOG(ERR, "Failed allocate mbuf qid=%d pool=%s", - qidx, rxq->pool->name); + PMD_INIT_LOG(ERR, "Failed allocate mbuf VF%d qid=%d " + "pool=%s", + nic->vf_id, qidx, rxq->pool->name); return -ENOMEM; } rxq->mbuf_phys_off -= nicvf_mbuff_meta_length(mbuf); @@ -1335,15 +1343,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) if (mbuf_phys_off == 0) mbuf_phys_off = rxq->mbuf_phys_off; if (mbuf_phys_off != rxq->mbuf_phys_off) { - PMD_INIT_LOG(ERR, "pool params not same,%s %" PRIx64, - rxq->pool->name, mbuf_phys_off); + PMD_INIT_LOG(ERR, "pool params not same,%s VF%d %" + PRIx64, rxq->pool->name, nic->vf_id, + mbuf_phys_off); return -EINVAL; } } /* Check the level of buffers in the pool */ total_rxq_desc = 0; - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = dev->data->rx_queues[qidx]; /* Count total numbers of rxq descs */ total_rxq_desc += rxq->qlen_mask + 1; @@ -1361,14 +1370,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) /* Check RBDR desc overflow */ ret = nicvf_qsize_rbdr_roundup(total_rxq_desc); if (ret == 0) { - PMD_INIT_LOG(ERR, "Reached RBDR desc limit, reduce nr desc"); + PMD_INIT_LOG(ERR, "Reached RBDR desc limit, reduce nr desc " + "VF%d", nic->vf_id); return -ENOMEM; } /* Enable qset */ ret = nicvf_qset_config(nic); if (ret) { - PMD_INIT_LOG(ERR, "Failed to enable qset %d", ret); + PMD_INIT_LOG(ERR, "Failed to enable qset %d VF%d", ret, + nic->vf_id); return ret; } @@ -1376,14 +1387,16 @@ nicvf_dev_start(struct rte_eth_dev *dev) nb_rbdr_desc = nicvf_qsize_rbdr_roundup(total_rxq_desc); ret = nicvf_qset_rbdr_alloc(dev, nic, nb_rbdr_desc, rbdrsz); if (ret) { - PMD_INIT_LOG(ERR, "Failed to allocate memory for rbdr alloc"); + PMD_INIT_LOG(ERR, "Failed to allocate memory for rbdr alloc " + "VF%d", nic->vf_id); goto qset_reclaim; } /* Enable and configure RBDR registers */ ret = nicvf_qset_rbdr_config(nic, 0); if (ret) { - PMD_INIT_LOG(ERR, "Failed to configure rbdr %d", ret); + PMD_INIT_LOG(ERR, "Failed to configure rbdr %d VF%d", ret, + nic->vf_id); goto qset_rbdr_free; } @@ -1391,52 +1404,127 @@ nicvf_dev_start(struct rte_eth_dev *dev) ret = nicvf_qset_rbdr_precharge(dev, nic, 0, rbdr_rte_mempool_get, total_rxq_desc); if (ret) { - PMD_INIT_LOG(ERR, "Failed to fill rbdr %d", ret); + PMD_INIT_LOG(ERR, "Failed to fill rbdr %d VF%d", ret, + nic->vf_id); goto qset_rbdr_reclaim; } - PMD_DRV_LOG(INFO, "Filled %d out of %d entries in RBDR", - nic->rbdr->tail, nb_rbdr_desc); + PMD_DRV_LOG(INFO, "Filled %d out of %d entries in RBDR VF%d", + nic->rbdr->tail, nb_rbdr_desc, nic->vf_id); + + /* Configure VLAN Strip */ + nicvf_vlan_hw_strip(nic, dev->data->dev_conf.rxmode.hw_vlan_strip); + + /* Get queue ranges for this VF */ + nicvf_tx_range(dev, nic, &tx_start, &tx_end); + + /* Configure TX queues */ + for (qidx = tx_start; qidx <= tx_end; qidx++) { + ret = nicvf_vf_start_tx_queue(dev, nic, + qidx % MAX_SND_QUEUES_PER_QS); + if (ret) + goto start_txq_error; + } /* Configure RX queues */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { - ret = nicvf_start_rx_queue(dev, qidx); + for (qidx = rx_start; qidx <= rx_end; qidx++) { + ret = nicvf_vf_start_rx_queue(dev, nic, + qidx % MAX_RCV_QUEUES_PER_QS); if (ret) goto start_rxq_error; } - /* Configure VLAN Strip */ - nicvf_vlan_hw_strip(nic, dev->data->dev_conf.rxmode.hw_vlan_strip); - - /* Configure TX queues */ - for (qidx = 0; qidx < dev->data->nb_tx_queues; qidx++) { - ret = nicvf_start_tx_queue(dev, qidx); + if (!nic->sqs_mode) { + /* Configure CPI algorithm */ + ret = nicvf_configure_cpi(dev); if (ret) goto start_txq_error; + + ret = nicvf_mbox_get_rss_size(nic); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to get rss table size"); + goto qset_rss_error; + } + + /* Configure RSS */ + ret = nicvf_configure_rss(dev); + if (ret) + goto qset_rss_error; } - /* Configure CPI algorithm */ - ret = nicvf_configure_cpi(dev); - if (ret) - goto start_txq_error; + /* Done; Let PF make the BGX's RX and TX switches to ON position */ + nicvf_mbox_cfg_done(nic); + return 0; - /* Configure RSS */ - ret = nicvf_configure_rss(dev); - if (ret) - goto qset_rss_error; +qset_rss_error: + nicvf_rss_term(nic); +start_rxq_error: + for (qidx = rx_start; qidx <= rx_end; qidx++) + nicvf_vf_stop_rx_queue(dev, nic, qidx % MAX_RCV_QUEUES_PER_QS); +start_txq_error: + for (qidx = tx_start; qidx <= tx_end; qidx++) + nicvf_vf_stop_tx_queue(dev, nic, qidx % MAX_SND_QUEUES_PER_QS); +qset_rbdr_reclaim: + nicvf_qset_rbdr_reclaim(nic, 0); + nicvf_rbdr_release_mbufs(dev, nic); +qset_rbdr_free: + if (nic->rbdr) { + rte_free(nic->rbdr); + nic->rbdr = NULL; + } +qset_reclaim: + nicvf_qset_reclaim(nic); + return ret; +} + +static int +nicvf_dev_start(struct rte_eth_dev *dev) +{ + uint16_t qidx; + int ret; + size_t i; + struct nicvf *nic = nicvf_pmd_priv(dev); + struct rte_eth_rxmode *rx_conf = &dev->data->dev_conf.rxmode; + uint16_t mtu; + uint32_t buffsz = 0, rbdrsz = 0; + struct rte_pktmbuf_pool_private *mbp_priv; + struct nicvf_rxq *rxq; + + PMD_INIT_FUNC_TRACE(); + + /* This function must be called for a primary device */ + assert_if_secondary(nic); + + /* Validate RBDR buff size */ + for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + rxq = dev->data->rx_queues[qidx]; + mbp_priv = rte_mempool_get_priv(rxq->pool); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + if (buffsz % 128) { + PMD_INIT_LOG(ERR, "rxbuf size must be multiply of 128"); + return -EINVAL; + } + if (rbdrsz == 0) + rbdrsz = buffsz; + if (rbdrsz != buffsz) { + PMD_INIT_LOG(ERR, "buffsz not same, qidx=%d (%d/%d)", + qidx, rbdrsz, buffsz); + return -EINVAL; + } + } /* Configure loopback */ ret = nicvf_loopback_config(nic, dev->data->dev_conf.lpbk_mode); if (ret) { PMD_INIT_LOG(ERR, "Failed to configure loopback %d", ret); - goto qset_rss_error; + return ret; } /* Reset all statistics counters attached to this port */ ret = nicvf_mbox_reset_stat_counters(nic, 0x3FFF, 0x1F, 0xFFFF, 0xFFFF); if (ret) { PMD_INIT_LOG(ERR, "Failed to reset stat counters %d", ret); - goto qset_rss_error; + return ret; } /* Setup scatter mode if needed by jumbo */ @@ -1457,33 +1545,23 @@ nicvf_dev_start(struct rte_eth_dev *dev) return -EBUSY; } + ret = nicvf_vf_start(dev, nic, rbdrsz); + if (ret != 0) + return ret; + + for (i = 0; i < nic->sqs_count; i++) { + assert(nic->snicvf[i]); + + ret = nicvf_vf_start(dev, nic->snicvf[i], rbdrsz); + if (ret != 0) + return ret; + } + /* Configure callbacks based on scatter mode */ nicvf_set_tx_function(dev); nicvf_set_rx_function(dev); - /* Done; Let PF make the BGX's RX and TX switches to ON position */ - nicvf_mbox_cfg_done(nic); return 0; - -qset_rss_error: - nicvf_rss_term(nic); -start_txq_error: - for (qidx = 0; qidx < dev->data->nb_tx_queues; qidx++) - nicvf_vf_stop_tx_queue(dev, nic, qidx); -start_rxq_error: - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) - nicvf_vf_stop_rx_queue(dev, nic, qidx); -qset_rbdr_reclaim: - nicvf_qset_rbdr_reclaim(nic, 0); - nicvf_rbdr_release_mbufs(dev, nic); -qset_rbdr_free: - if (nic->rbdr) { - rte_free(nic->rbdr); - nic->rbdr = NULL; - } -qset_reclaim: - nicvf_qset_reclaim(nic); - return ret; } static void -- 1.9.1