From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0048.outbound.protection.outlook.com [104.47.42.48]) by dpdk.org (Postfix) with ESMTP id 399056932 for ; Fri, 30 Sep 2016 14:07:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=IwyRyQSsFTUnujpZWPjxISC2pESOVAtGVW0fgIkGlf8=; b=jJKcHwSZCF30ld200Wii1RsHGH7vwItPaQGHM5WcUvAQ668BxufwWgDIfEmc2ib+6jVosCa3KkMK6R6t58H63YnCyOn9nwHEjj5Ml61BCdxolTKcCJROS06a662GTm+dmFWE4TH55VmerDPGQSB1nLHsabbEssM44kadMACKchk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Kamil.Rytarowski@cavium.com; Received: from cavium1.semihalf.local (31.172.191.173) by CY4PR07MB3063.namprd07.prod.outlook.com (10.172.116.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.549.15; Fri, 30 Sep 2016 12:07:03 +0000 From: Kamil Rytarowski To: CC: , , , , , , , Kamil Rytarowski Date: Fri, 30 Sep 2016 14:05:51 +0200 Message-ID: <1475237154-25388-13-git-send-email-krytarowski@caviumnetworks.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1475237154-25388-1-git-send-email-krytarowski@caviumnetworks.com> References: <1472230448-17490-1-git-send-email-krytarowski@caviumnetworks.com> <1475237154-25388-1-git-send-email-krytarowski@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [31.172.191.173] X-ClientProxiedBy: VI1PR07CA0001.eurprd07.prod.outlook.com (10.163.160.139) To CY4PR07MB3063.namprd07.prod.outlook.com (10.172.116.144) X-MS-Office365-Filtering-Correlation-Id: 0bbc82e5-65e4-4db9-b60f-08d3e92a4b3b X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 2:Cq/oJF+XBI9PNDNDr1yB+cLqelJZhdH0u+3dbjivzeGXAgo4Pde9nd0+FfrED8QEDnbPKsEBjtXBcHXeZH83rBqcpf1VxKIk74ia6xOIVO6d0hxjFYg01UxT6i9QlJYpqI+y6Lf5ssEWiRS2KsYdENbe2x3ohIxWRQpjfRAvf4o+cLEcEHqeUHQ9mwsBdjCZ; 3:45/EvqfW6uVYUTvvMR7fzohTOXYAQvJlFuA4o4BsDiFGXmgRpzH0vVguK8mXmzZZlfnAK777OYSd/lKe2lwin7x4ZpsAHa/eAKf1P1dWvsUWlyMt14V+pVB/os/4fAI+; 25:AgXB4mEMxJzJWWTbPVpgavVI9vhxq6oUEzh0Abv+kBLPauvTGu4sy3KwFxROPaMaKpg3f4ye9odUsK85sm3sT60/8gj3uxuna49bd0ZmiX9A7CcgBloMOvTrczMkuGOjUAjruE8Ss4N1pnxzvi0a+Fu29hgK/An0Vn+3lED32LFBY4FeMBXu0Tp0tnLVrW7uzRvWOU4FkyEqYO4xfuyNqWiuA7f/LnYrxl0qkRLLXLn3O76nRhkeykqaXy2D16xUz2227dkyJKQohM+lBKwvebYBJC5yYiaiWwB1p/PXIE0KQgQxrgJ54HQWlNc5jo29MNr0Rbg4a4fq6WIsp9wvm5mW+Xm3bbLy43njGNzh170C+NdSkOKffyRWQqUbZTH9W+nDHMLdFvOgDVF34seSJ9HY4+XUBoKmK9ERgLjhIZk= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR07MB3063; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 31:JWUuCb48gY+qCsf0Nh31nUbfy1/uW7JLR3pLyEz5nndF5SXvC5Kj50M79BDCNV/PRw0Wi6rJjKcxV/KuN7R5BYkhx+orWXqFq4wNS8+yzy9agE8ccnEY6yV1vkuZgnefk2izwQXPPEQQtRXvdvk/A2kOV+BNk7GnNdF9mugVkdK2PPb824JGiLymjxIsU5nuXIDjBVQSVJ/pDHhZwR6w8KhDtj23CgWcI0n5n6AtTLk=; 20:807bCYxFBrpmw+pkoSi2fMu5jxblILjEHitQ0bbO2Uu1OLPOi+l181y4h0jB29iqvgp7qYqSutYPsd5r9c4rY0n5hbOZNJ6RzVh6x/pF2iUqnrlK51kYtUezcsroPBq0oi4VW2dP3NTeoZT+8HfK0pCjxyIOcyhB8wy/IXWYPa2MmtJhqAJrHnknoS2OgQGze+ucdzv88gIC0dUBm03IsEkSfxU1QCEf9t6Aphg9t6jbDIHb8PBS1mETpRryUjdeZryebeK+3tO3jbJhcRkAVKKZVWa9LOjWWFiI8TMHMaf0xlZl2JbF3IYiJzyOQJzZcAZV/ItrOeaz3GjUMJJu5tHnBljhFbpNV6/TKyt0f2pacHQ3pBkTqw1JXYKNErpvZYxnlaMScKUs24EypCO/Yp3QBsv01rsxl+VjsOPaYpvcha765BT2CKVT4QU+J0JTQfJMGT9DL/GLOe2btkjeuIPgbwhQXuG9dOlD3bxXgOEIbWRGKbGxqfPjW8IXCJHum661/NowBtOovkZxQ8ac/dbC+Z6T299qSyWmMvTfxPClWoSsHbGsoHH8iwSjkbyHKmAB1EuTTJ6EIjEOsaTWNM7/s5a42oWJmIYRNdx3a+M= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:CY4PR07MB3063; BCL:0; PCL:0; RULEID:; SRVR:CY4PR07MB3063; X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 4:181yXpoOYewhrlcGFjg1k9O+wVmw5QlDu1DIdTXjVJTNbPzELbBqCGSjJTbYz++YhywK5JMM323gHvi7UakzvG71BF9jNDoy6EZHqjG/h2uoefVVB0vvEN+j0YtRW/t8yDFTAWGkEqHsr/Cs1Wl5i6TS9Zlep0UOsjRBl6V9H+KNJmvZW05gy9MMVivDjy/FCvbzm9s039U2SOY/1CBvzchqUvkOjvVDF9C6/aVbrvKh5QQHJhzSqKkO/3KCOvrmRaQtfYLfnViyjP5MvQveOLFYLu322icg+OB6mlk2AAs/nnwvvX4Ti3mZjG8Nb+LACn38Ora/fzxYvinpJYyH4Rnc85KW3LuP150wEr18p+J3zkvU2rTjjrnYmgpMLykOYWioUWbflNUD+NgiXsaqiA== X-Forefront-PRVS: 008184426E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(92566002)(3846002)(66066001)(189998001)(47776003)(42882006)(36756003)(105586002)(19580395003)(97736004)(6916009)(6666003)(68736007)(42186005)(19580405001)(50986999)(50226002)(77096005)(76176999)(5003940100001)(6116002)(7736002)(2950100002)(101416001)(586003)(106356001)(7846002)(5660300001)(4001430100002)(305945005)(33646002)(229853001)(110136003)(8676002)(4326007)(81166006)(107886002)(81156014)(50466002)(2906002)(48376002)(2351001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR07MB3063; H:cavium1.semihalf.local; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR07MB3063; 23:o+lz2Imk4oiZtB0iXah6cLvvk8ksuSHt13qEXBV9S?= =?us-ascii?Q?mok28qiJwJpcfGfIq8YRivYfKd3Apt89DGQYEMR8Ab06X36/WBXR10TtIXXn?= =?us-ascii?Q?qNASNFNrq2Y8Kdza55DxXNgejthzPlVRwrGX5ducQZJrTJtfSqeY7aJaPOgk?= =?us-ascii?Q?pTQY8q2+dUwDXrbNAZBXU7tGcR5A9eOpyTIul5BS53fY/jcWXnOgfCM8tQQH?= =?us-ascii?Q?b/GDxwHYelzbZ3zM3H3JDsqM6rsejHyIOaiImn/a7QP7A8nxTGSpLNQEt+Ht?= =?us-ascii?Q?QpUZjqsq1uWLYgdQbT46tOIBwpEfEC2G1+2gSoUL7bHbbdPd+EI+v+GN4Z17?= =?us-ascii?Q?j/pMyLwctO0wJFbjtNZYsTRfCkm0ZMwNtZNeAWaZZ7EXsJy1mdnH/f/aZnGL?= =?us-ascii?Q?pAQpLFdNRJbhx3anI+YuWYS9LhN7UlYfTxCxUpDNtTmueYn8DaNjPtmws224?= =?us-ascii?Q?+cTtwR2PgwkvQxRLe6aXAT9n7iEhGg0F1ILgJgKKa03f5UcDDyjcTgHSEhiM?= =?us-ascii?Q?2iQ7PH1P+PLZ+01gKCuyi1hdY63H5CTU3BeJiqNrzxF0L+QfhOKB+l3c6mjI?= =?us-ascii?Q?pbJM+BU/6fCQIWrzlibN5i1ixt16UZ7UXodgNhmIZHvYUsmaYS0XMuUzyLmb?= =?us-ascii?Q?EZAUfEMLONJ72s9lRYOhYOYijeYTvm7lvt8VsZYC8toDgFRJ1ZML2+SOUKV4?= =?us-ascii?Q?YQ7KZmKS7S7ae3gin2qTOIFZU1Ud7L/V46hegkE0r9DRI31+9t7UpRjot8fX?= =?us-ascii?Q?TnJbiE2BWGUiCMs3QhquADTjDSPhuSRYYCFyJyOzdKjzZgTIF2fCEZ4Yif72?= =?us-ascii?Q?J7bAR2E7KFnUOJ2m75bKptyplDJezV83ZACx/1Ttn0xTHt6b7X4bzPXzZfeL?= =?us-ascii?Q?oLmF9zq8XJyxAMzoaSuLGu01IBPPP6gp7qvXWOR+3e2nIl2LQOmb7cAg2i9i?= =?us-ascii?Q?B9fVPVHQ+e/ILCVnghPN8pTOAnAr4Yr+zmPas9mS6ek9vfrU+t/iw5tfotCT?= =?us-ascii?Q?rwI/U9btyS2kt3KrRRBAECpiJG++GSSoX6lR3wlDMJeXu+zBrzqtkIN9rhZ2?= =?us-ascii?Q?ILan5zyT1FU5zsFdef9OKPKhBuuFOXD+lsetwa8VEhtpH0aOIQj4EeCajsb7?= =?us-ascii?Q?HLXjh+0BYYIK6YNYl8tVh8VlmQvHX+/YC2ep72q+2Q6J4POwl9ekuHaYzNKR?= =?us-ascii?Q?/d9GHSzSafhMeQ=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR07MB3063; 6:cUMkueAHJ5XAHCJZWDkq5lIpQ7le46DN841pS4wm9YJ9zAHBw6gn0Yqa/+i2Y9XdW5ewbEejLNTA2TrQMSuxSS7a4u6du048ovZRBh2QkBEYCji2NVy7CSNinpAMWELYETZvdUOr6mu2+EbgFiaj/ONqxIZB8ScYWTqu5PjotD8qXT2lLFGHaQJjPd2UOB014PsqZSyP6Qh+nA8hYWPYslW3hdqEiUKdj/5phL8khCr0Kx5HSZ6S8zT1IZ9nkF3AHmXGl4uUtwVjSTauh4Itu960hZApZEHIGjUx23oPYws=; 5:RNRAEq2L5KsrE7c5qVxALTCqcZxUDrMaEa2/KN8g5Xbilxy5i+z5IxFTf2BF1w9JORKx/H8Mj5em4NsZZ5QQpsuBIpbvGWT/yBNTTtv32ZibgeTZHek1Uiiine/1l+QpwvxH2LpKBy8gUGqvEZ5PGA==; 24:GnMJr5sJaEZPo7NsxsmfdCwRHaKthrBuX1W/Cp3T6pvmi5halOMF4C2KmqLEfrP27Zc5jH1PARc6/ui4y5jrc3MofOMQss0E5wamEEuOgxQ=; 7:fDGEIYPUW0N5NxAVg0oPg/q4bG6Vk7Gxsd3qO25bL0PUbm+Uo3TtVZfNYXM0817r9PpHbc5OPvv8knKWDE/nYs90MequuKVUiq9YjNWAXN6aPnylHdAccLH1GYRWHIhSN/kYbe2T4BB/Az+qs4s8eUT0AZs292GmD8S2DA7RcMLjPbtYZXJzP/jCS7XUTw8xbJlxo9ygcarIdS3wgD/N3dPuFRXtiBsjKJswbEuC2x+/oT94PmXD5U2CoGMcm9JXFApI7Ub++E3SmcnwnIFdHGD3u4lMCrNQjVRMRL8XwL5lvoQW2wxUijyPoe1SBgMc SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2016 12:07:03.2240 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR07MB3063 Subject: [dpdk-dev] [PATCH v2 12/15] net/thunderx: add final bits for secondary queue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2016 12:07:07 -0000 From: Kamil Rytarowski Summary: - add secondary qset support in device stats - add support for releasing mbufs from RBDR for >8 queues - add support for releasing mbufs from RX queues for >8 queues - support >8 queues in tx_queue_setup - support >8 queues in rx_queue_setup - support up to 96 queues per device (dev_info->max_rx_queues) - add secondary qset support in rbdr_rte_mempool_get - support >8 queues in multiprocess mode (do not reconfigure VFs) - setup periodic alarm accordingly for type of VFs: * primary VF - handle events on queues and link status * secondary VF - handle events on queues - initialize hardware capabilities in secondary qsets Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki Signed-off-by: Jerin Jacob --- drivers/net/thunderx/nicvf_ethdev.c | 200 ++++++++++++++++++++++++++++-------- 1 file changed, 157 insertions(+), 43 deletions(-) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 5f12b91..9dde75d 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -118,7 +118,7 @@ nicvf_interrupt(void *arg) nicvf_interrupt, dev); } -static void __rte_unused +static void nicvf_vf_interrupt(void *arg) { struct nicvf *nic = arg; @@ -236,9 +236,15 @@ nicvf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) struct nicvf_hw_tx_qstats tx_qstats; struct nicvf_hw_stats port_stats; struct nicvf *nic = nicvf_pmd_priv(dev); + uint16_t rx_start, rx_end; + uint16_t tx_start, tx_end; + size_t i; + + /* RX queue indices for the first VF */ + nicvf_rx_range(dev, nic, &rx_start, &rx_end); /* Reading per RX ring stats */ - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { if (qidx == RTE_ETHDEV_QUEUE_STAT_CNTRS) break; @@ -247,8 +253,11 @@ nicvf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->q_ipackets[qidx] = rx_qstats.q_rx_packets; } + /* TX queue indices for the first VF */ + nicvf_tx_range(dev, nic, &tx_start, &tx_end); + /* Reading per TX ring stats */ - for (qidx = 0; qidx < dev->data->nb_tx_queues; qidx++) { + for (qidx = tx_start; qidx <= tx_end; qidx++) { if (qidx == RTE_ETHDEV_QUEUE_STAT_CNTRS) break; @@ -257,6 +266,40 @@ nicvf_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) stats->q_opackets[qidx] = tx_qstats.q_tx_packets; } + for (i = 0; i < nic->sqs_count; i++) { + struct nicvf *snic = nic->snicvf[i]; + + if (snic == NULL) + break; + + /* RX queue indices for a secondary VF */ + nicvf_rx_range(dev, snic, &rx_start, &rx_end); + + /* Reading per RX ring stats */ + for (qidx = rx_start; qidx <= rx_end; qidx++) { + if (qidx == RTE_ETHDEV_QUEUE_STAT_CNTRS) + break; + + nicvf_hw_get_rx_qstats(snic, &rx_qstats, + qidx % MAX_RCV_QUEUES_PER_QS); + stats->q_ibytes[qidx] = rx_qstats.q_rx_bytes; + stats->q_ipackets[qidx] = rx_qstats.q_rx_packets; + } + + /* TX queue indices for a secondary VF */ + nicvf_tx_range(dev, snic, &tx_start, &tx_end); + /* Reading per TX ring stats */ + for (qidx = tx_start; qidx <= tx_end; qidx++) { + if (qidx == RTE_ETHDEV_QUEUE_STAT_CNTRS) + break; + + nicvf_hw_get_tx_qstats(snic, &tx_qstats, + qidx % MAX_SND_QUEUES_PER_QS); + stats->q_obytes[qidx] = tx_qstats.q_tx_bytes; + stats->q_opackets[qidx] = tx_qstats.q_tx_packets; + } + } + nicvf_hw_get_stats(nic, &port_stats); stats->ibytes = port_stats.rx_bytes; stats->ipackets = port_stats.rx_ucast_frames; @@ -322,13 +365,36 @@ nicvf_dev_stats_reset(struct rte_eth_dev *dev) int i; uint16_t rxqs = 0, txqs = 0; struct nicvf *nic = nicvf_pmd_priv(dev); + uint16_t rx_start, rx_end; + uint16_t tx_start, tx_end; - for (i = 0; i < dev->data->nb_rx_queues; i++) + /* Reset all primary nic counters */ + nicvf_rx_range(dev, nic, &rx_start, &rx_end); + for (i = rx_start; i <= rx_end; i++) rxqs |= (0x3 << (i * 2)); - for (i = 0; i < dev->data->nb_tx_queues; i++) + + nicvf_tx_range(dev, nic, &tx_start, &tx_end); + for (i = tx_start; i <= tx_end; i++) txqs |= (0x3 << (i * 2)); nicvf_mbox_reset_stat_counters(nic, 0x3FFF, 0x1F, rxqs, txqs); + + /* Reset secondary nic queue counters */ + for (i = 0; i < nic->sqs_count; i++) { + struct nicvf *snic = nic->snicvf[i]; + if (snic == NULL) + break; + + nicvf_rx_range(dev, snic, &rx_start, &rx_end); + for (i = rx_start; i <= rx_end; i++) + rxqs |= (0x3 << ((i % MAX_CMP_QUEUES_PER_QS) * 2)); + + nicvf_tx_range(dev, snic, &tx_start, &tx_end); + for (i = tx_start; i <= tx_end; i++) + txqs |= (0x3 << ((i % MAX_SND_QUEUES_PER_QS) * 2)); + + nicvf_mbox_reset_stat_counters(snic, 0, 0, rxqs, txqs); + } } /* Promiscuous mode enabled by default in LMAC to VF 1:1 map configuration */ @@ -596,14 +662,18 @@ nicvf_qset_rbdr_alloc(struct rte_eth_dev *dev, struct nicvf *nic, } static void -nicvf_rbdr_release_mbuf(struct rte_eth_dev *dev, struct nicvf *nic __rte_unused, +nicvf_rbdr_release_mbuf(struct rte_eth_dev *dev, struct nicvf *nic, nicvf_phys_addr_t phy) { uint16_t qidx; void *obj; struct nicvf_rxq *rxq; + uint16_t rx_start, rx_end; - for (qidx = 0; qidx < dev->data->nb_rx_queues; qidx++) { + /* Get queue ranges for this VF */ + nicvf_rx_range(dev, nic, &rx_start, &rx_end); + + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = dev->data->rx_queues[qidx]; if (rxq->precharge_cnt) { obj = (void *)nicvf_mbuff_phy2virt(phy, @@ -861,6 +931,11 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, PMD_INIT_FUNC_TRACE(); + if (qidx >= MAX_SND_QUEUES_PER_QS) + nic = nic->snicvf[qidx / MAX_SND_QUEUES_PER_QS - 1]; + + qidx = qidx % MAX_SND_QUEUES_PER_QS; + /* Socket id check */ if (socket_id != (unsigned int)SOCKET_ID_ANY && socket_id != nic->node) PMD_DRV_LOG(WARNING, "socket_id expected %d, configured %d", @@ -895,18 +970,20 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, } /* Free memory prior to re-allocation if needed. */ - if (dev->data->tx_queues[qidx] != NULL) { + if (dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)] != NULL) { PMD_TX_LOG(DEBUG, "Freeing memory prior to re-allocation %d", - qidx); - nicvf_dev_tx_queue_release(dev->data->tx_queues[qidx]); - dev->data->tx_queues[qidx] = NULL; + nicvf_netdev_qidx(nic, qidx)); + nicvf_dev_tx_queue_release( + dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)]); + dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)] = NULL; } /* Allocating tx queue data structure */ txq = rte_zmalloc_socket("ethdev TX queue", sizeof(struct nicvf_txq), RTE_CACHE_LINE_SIZE, nic->node); if (txq == NULL) { - PMD_INIT_LOG(ERR, "Failed to allocate txq=%d", qidx); + PMD_INIT_LOG(ERR, "Failed to allocate txq=%d", + nicvf_netdev_qidx(nic, qidx)); return -ENOMEM; } @@ -949,10 +1026,12 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, nicvf_tx_queue_reset(txq); PMD_TX_LOG(DEBUG, "[%d] txq=%p nb_desc=%d desc=%p phys=0x%" PRIx64, - qidx, txq, nb_desc, txq->desc, txq->phys); + nicvf_netdev_qidx(nic, qidx), txq, nb_desc, txq->desc, + txq->phys); - dev->data->tx_queues[qidx] = txq; - dev->data->tx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + dev->data->tx_queues[nicvf_netdev_qidx(nic, qidx)] = txq; + dev->data->tx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STOPPED; return 0; } @@ -967,7 +1046,8 @@ nicvf_rx_queue_release_mbufs(struct rte_eth_dev *dev, struct nicvf_rxq *rxq) if (dev->rx_pkt_burst == NULL) return; - while ((rxq_cnt = nicvf_dev_rx_queue_count(dev, rxq->queue_id))) { + while ((rxq_cnt = nicvf_dev_rx_queue_count(dev, + nicvf_netdev_qidx(rxq->nic, rxq->queue_id)))) { nb_pkts = dev->rx_pkt_burst(rxq, rx_pkts, NICVF_MAX_RX_FREE_THRESH); PMD_DRV_LOG(INFO, "nb_pkts=%d rxq_cnt=%d", nb_pkts, rxq_cnt); @@ -977,7 +1057,10 @@ nicvf_rx_queue_release_mbufs(struct rte_eth_dev *dev, struct nicvf_rxq *rxq) } } - refill_cnt += nicvf_dev_rbdr_refill(dev, rxq->queue_id); + + refill_cnt += nicvf_dev_rbdr_refill(dev, + nicvf_netdev_qidx(rxq->nic, rxq->queue_id)); + PMD_DRV_LOG(INFO, "free_cnt=%d refill_cnt=%d", released_pkts, refill_cnt); } @@ -1149,6 +1232,11 @@ nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, PMD_INIT_FUNC_TRACE(); + if (qidx >= MAX_RCV_QUEUES_PER_QS) + nic = nic->snicvf[qidx / MAX_RCV_QUEUES_PER_QS - 1]; + + qidx = qidx % MAX_RCV_QUEUES_PER_QS; + /* Socket id check */ if (socket_id != (unsigned int)SOCKET_ID_ANY && socket_id != nic->node) PMD_DRV_LOG(WARNING, "socket_id expected %d, configured %d", @@ -1191,18 +1279,20 @@ nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, } /* Free memory prior to re-allocation if needed */ - if (dev->data->rx_queues[qidx] != NULL) { + if (dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)] != NULL) { PMD_RX_LOG(DEBUG, "Freeing memory prior to re-allocation %d", - qidx); - nicvf_dev_rx_queue_release(dev->data->rx_queues[qidx]); - dev->data->rx_queues[qidx] = NULL; + nicvf_netdev_qidx(nic, qidx)); + nicvf_dev_rx_queue_release( + dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)]); + dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)] = NULL; } /* Allocate rxq memory */ rxq = rte_zmalloc_socket("ethdev rx queue", sizeof(struct nicvf_rxq), RTE_CACHE_LINE_SIZE, nic->node); if (rxq == NULL) { - PMD_INIT_LOG(ERR, "Failed to allocate rxq=%d", qidx); + PMD_INIT_LOG(ERR, "Failed to allocate rxq=%d", + nicvf_netdev_qidx(nic, qidx)); return -ENOMEM; } @@ -1232,11 +1322,12 @@ nicvf_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, nicvf_rx_queue_reset(rxq); PMD_RX_LOG(DEBUG, "[%d] rxq=%p pool=%s nb_desc=(%d/%d) phy=%" PRIx64, - qidx, rxq, mp->name, nb_desc, + nicvf_netdev_qidx(nic, qidx), rxq, mp->name, nb_desc, rte_mempool_avail_count(mp), rxq->phys); - dev->data->rx_queues[qidx] = rxq; - dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; + dev->data->rx_queues[nicvf_netdev_qidx(nic, qidx)] = rxq; + dev->data->rx_queue_state[nicvf_netdev_qidx(nic, qidx)] = + RTE_ETH_QUEUE_STATE_STOPPED; return 0; } @@ -1249,8 +1340,10 @@ nicvf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->min_rx_bufsize = ETHER_MIN_MTU; dev_info->max_rx_pktlen = NIC_HW_MAX_FRS; - dev_info->max_rx_queues = (uint16_t)MAX_RCV_QUEUES_PER_QS; - dev_info->max_tx_queues = (uint16_t)MAX_SND_QUEUES_PER_QS; + dev_info->max_rx_queues = + (uint16_t)MAX_RCV_QUEUES_PER_QS * (MAX_SQS_PER_VF + 1); + dev_info->max_tx_queues = + (uint16_t)MAX_SND_QUEUES_PER_QS * (MAX_SQS_PER_VF + 1); dev_info->max_mac_addrs = 1; dev_info->max_vfs = dev->pci_dev->max_vfs; @@ -1291,9 +1384,13 @@ rbdr_rte_mempool_get(void *dev, void *opaque) uintptr_t mbuf; struct nicvf_rxq *rxq; struct rte_eth_dev *eth_dev = (struct rte_eth_dev *)dev; - struct nicvf *nic __rte_unused = (struct nicvf *)opaque; + struct nicvf *nic = (struct nicvf *)opaque; + uint16_t rx_start, rx_end; + + /* Get queue ranges for this VF */ + nicvf_rx_range(eth_dev, nic, &rx_start, &rx_end); - for (qidx = 0; qidx < eth_dev->data->nb_rx_queues; qidx++) { + for (qidx = rx_start; qidx <= rx_end; qidx++) { rxq = eth_dev->data->rx_queues[qidx]; /* Maintain equal buffer count across all pools */ if (rxq->precharge_cnt >= rxq->qlen_mask) @@ -1856,10 +1953,16 @@ nicvf_eth_dev_init(struct rte_eth_dev *eth_dev) /* For secondary processes, the primary has done all the work */ if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - /* Setup callbacks for secondary process */ - nicvf_set_tx_function(eth_dev); - nicvf_set_rx_function(eth_dev); - return 0; + if (nic) { + /* Setup callbacks for secondary process */ + nicvf_set_tx_function(eth_dev); + nicvf_set_rx_function(eth_dev); + return 0; + } else { + /* If nic == NULL than it is secondary function + * so ethdev need to be released by caller */ + return ENOTSUP; + } } pci_dev = eth_dev->pci_dev; @@ -1904,11 +2007,28 @@ nicvf_eth_dev_init(struct rte_eth_dev *eth_dev) ); } + ret = nicvf_base_init(nic); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to execute nicvf_base_init"); + goto malloc_fail; + } + if (nic->sqs_mode) { - PMD_INIT_LOG(INFO, "Unsupported SQS VF detected, Detaching..."); - /* Detach port by returning Positive error number */ - ret = ENOTSUP; - goto alarm_fail; + /* Push nic to stack of secondary vfs */ + nicvf_svf_push(nic); + + /* Steal nic pointer from the device for further reuse */ + eth_dev->data->dev_private = NULL; + + nicvf_periodic_alarm_stop(nicvf_interrupt, eth_dev); + ret = nicvf_periodic_alarm_start(nicvf_vf_interrupt, nic); + if (ret) { + PMD_INIT_LOG(ERR, "Failed to start period alarm"); + goto fail; + } + + /* Detach port by returning postive error number */ + return ENOTSUP; } eth_dev->data->mac_addrs = rte_zmalloc("mac_addr", ETHER_ADDR_LEN, 0); @@ -1929,12 +2049,6 @@ nicvf_eth_dev_init(struct rte_eth_dev *eth_dev) goto malloc_fail; } - ret = nicvf_base_init(nic); - if (ret) { - PMD_INIT_LOG(ERR, "Failed to execute nicvf_base_init"); - goto malloc_fail; - } - PMD_INIT_LOG(INFO, "Port %d (%x:%x) mac=%02x:%02x:%02x:%02x:%02x:%02x", eth_dev->data->port_id, nic->vendor_id, nic->device_id, nic->mac_addr[0], nic->mac_addr[1], nic->mac_addr[2], -- 1.9.1