From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0055.outbound.protection.outlook.com [104.47.42.55]) by dpdk.org (Postfix) with ESMTP id 6D6D55F29 for ; Fri, 7 Sep 2018 17:23:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=AQUANTIA1COM.onmicrosoft.com; s=selector1-aquantia-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wTbTOi0TnZcsLqtdJHsXaMbGz1N7gNDbjsKay7KP4Dw=; b=MnLyFL7Iic0hydP7VnqbaSzncOMPQNq3g+a2xrOJoaimOiFToKWohrO6k6DLRylgA/ulklcvbIYVfCYP+ujPNGP2pRj8YVRHmkfJVYoTArcNLb4/1tYqoNoNN5Ga0Z1kUpD525uo8bmJgboB1WFjlLf+J2A5SJL5y+8DtCOZKEM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Igor.Russkikh@aquantia.com; Received: from ubuntubox.rdc.aquantia.com (95.79.108.179) by BLUPR0701MB1652.namprd07.prod.outlook.com (2a01:111:e400:58c6::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1122.16; Fri, 7 Sep 2018 15:23:41 +0000 From: Igor Russkikh To: dev@dpdk.org Cc: pavel.belous@aquantia.com, Nadezhda.Krupnina@aquantia.com, igor.russkikh@aquantia.com, Simon.Edelhaus@aquantia.com, Corey Melton , Ashish Kumar Date: Fri, 7 Sep 2018 18:21:59 +0300 Message-Id: <1536333719-32155-22-git-send-email-igor.russkikh@aquantia.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536333719-32155-1-git-send-email-igor.russkikh@aquantia.com> References: <1536333719-32155-1-git-send-email-igor.russkikh@aquantia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [95.79.108.179] X-ClientProxiedBy: BN6PR20CA0060.namprd20.prod.outlook.com (2603:10b6:404:151::22) To BLUPR0701MB1652.namprd07.prod.outlook.com (2a01:111:e400:58c6::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5ed4e362-2b70-4c80-47cb-08d614d5e523 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BLUPR0701MB1652; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1652; 3:PWWAYFfkz4MpCSoZpSbgAuTlP3L0RSv5VUSqkHtOWK3qbhiPvZCSC5sZRBLF8Fb5mrE24rZuPQbgNQCfxQ2lCrq0HrdjLr0tHyWo+45YZ8a48g46KqIlJ7AHm+McmDFJqDRrb1aZ6jXlp5kRv2fm1rR4qIfOU3SAMMp4S4Ow98OzkqPR+msc96thTuueg6r+huMU81bWOtM5f6My1cNursOG5HZNrD/IwfAH13uLkew9ZtU+KOr6d9KtgyBGyA4L; 25:7ZA6d0a9RoDPlBNSTHl0ZYwoNDS2u8vRb/O9MZJXVDIlCNKi8em2afruuBdMt/52+mEG9fdd3nOM8Xp/AGxzV1FkgBRmqXklOfu4QM8nlGavYAL0pt7nr/4wq4cKqVy1vgmdMY09lpOD06kHbiVgzBVh6fiyNzLGmf1DVHG1XQGKkegsu0qxipTJl0foU0069HQ/StmbEqRXI/97HX7fSIhivzLJ2DLify+nwI++M0Y5vADMFjHMBtuLj72DhjMzLzSsY2PqQkjOq4MX9Cc1NUvQaB+HGMTzP32oNWi3dmOPHc9z6JtHLszYEXgneRhM7zAeEFOM8ka74In1J7FIKQ==; 31:7pXVn+77AvQx1708Kp+JAc3iDR1XZoYeQ3KF5vEya616FVfsVTav/q/gjdDJ8MzFWKX0pNtA3cJIgDosSEOKbmzjIbejuHxjpuzMYmx2IS8/i9r1NUE+O2yzJupfns8JacRJkpPm8D3sFvlTz9uoDy9EMmzL/V0Ra6a8NAdfUqfLmqPrOXZYaHDodCU/k01avQf781rjxpn+5ffojVbXM8fi2hNYJ6rDEGMQmKwCYs4= X-MS-TrafficTypeDiagnostic: BLUPR0701MB1652: X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1652; 20:M9bwyaVSsrr2LQNDgq/sqltslFHFTgBKpvxtXNDeh4sqdviHUggOW0jKKWn2oA7R2OZUSmDjvRRLXlvK1e91XaSAjlSVDZiNXrFPZ3DmGDbVmbGinTmlth084pjaHe0LvFT3/xL/v++MAs9W7p6wDCDbP6y7t8dUHkdgFcUymLGdDOBomMX0CNpi+4tbQ8LxvUoYtKmDPHwnOl6hO51zoNCxBrrOcNEcTmiRhJt3nugomGKOFqUEGagj4HVb4Gy6KzA0wWKwrzjbX/6DwRzLW0JIU9cmDsZGUmxthYdnHqVIAYLkLCiEY/sstwFWC8P1lRuehwEauQoqc2YIOhxg4auNPQe8VyNNFWMk1a7FUlJERCW+EbMOl218x5tmI8dfaKFuiZmGGiOur91+hSBklDIUpotJMp8XWWKQh6m/Rl9oGIdf20fxgRaT1ZoO7VYsY371M6wF2WZJt/JA+rDX9VXM46Bfg+Ncz8ROqROA2tTt2PXl/YgIfmBY5wu9GJaz; 4:KQonIljxfZrmM4rSvUlNfj/rSoxVizcufOvMhyYcsNqD4RN8JeVzMaYI4PvrV+13PM99a+s/W/ICTwIO9Ugv2oy7OkyMFDmPve1l7KJ+biyFMYqyn2ZmD2IxUShyc7D8VfgcTzZ+2MwInU2DQh+J3X+1hm7Bl6ywyPHDrYF5cbcuHQNpCyXujFXpeq9x2/I4QrncNx7/qzbpX9NOYp+j1EJxApeJliJowogijgSrN12QRQFoNY6kHMtcNXzAvO1bChU49ELfl0jaL8x8kly0sw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201708071742011)(7699050); SRVR:BLUPR0701MB1652; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0701MB1652; X-Forefront-PRVS: 07880C4932 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39850400004)(396003)(376002)(366004)(346002)(136003)(189003)(199004)(81156014)(81166006)(44832011)(8936002)(2351001)(486006)(106356001)(2361001)(105586002)(446003)(2906002)(50226002)(476003)(11346002)(956004)(2616005)(305945005)(7736002)(16586007)(316002)(54906003)(14444005)(16526019)(66066001)(26005)(7696005)(52116002)(8676002)(51416003)(76176011)(186003)(386003)(68736007)(478600001)(6666003)(6916009)(6486002)(5660300001)(53936002)(47776003)(4326008)(25786009)(86362001)(48376002)(50466002)(72206003)(6116002)(97736004)(36756003)(3846002)(309714004); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0701MB1652; H:ubuntubox.rdc.aquantia.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: aquantia.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0701MB1652; 23:Yqox7Iv4NmFlBMTgZHbw+WtwM/iLBFDNV+R+u7r?= =?us-ascii?Q?12GBplQlzzLysl+Z61wPIaBV3IG9LWzgztL25wz1G8OnSHhSGGncgAAU44RO?= =?us-ascii?Q?eRUGITZlvjuBdFr0ArBGOelpjv7kvJ293LRpRD5JTrPvzuQSPwjPPWOMH1nv?= =?us-ascii?Q?ST8zO0qV0F8S80vG271IyD6ZOoRl1/19OmvxEj+XVH1sqqO8ZNzPH01huAvd?= =?us-ascii?Q?IveEWJQsb3QU3/m30l/eWe7R58+KNPKa2Hd1kWJZYRG0SJOJBLgvr2JVvIMR?= =?us-ascii?Q?dX8yPrdrz0W3yNzm5axifMitVzNxrNUnLroV7N4YwTihJm/WU5OnVV0TaOHP?= =?us-ascii?Q?zbBkiul7j/XC87G8f5xLLDIinUfclZMv6pr8bzDvFD0LWDSSBW6zGWMTI0Wu?= =?us-ascii?Q?YNisooQAHN9iKwmhPr1J7LgMP1+wbAXFiCx2NhU++yCJt7yJ43tbcBmPv+Rh?= =?us-ascii?Q?CJ6YJhJNlYY3jZ2qAOIyEu6MtLeivkb+bMffQ3UQi6nlr/tEV/w3MC3TR/PB?= =?us-ascii?Q?XjNbyQDnlnCbm+HC+kuPk+Ye6ix98EvomGlSkB1NhgZY/BLnp5sX7pDsM/Yj?= =?us-ascii?Q?OEA/zyFpzAF/JVbgy/VGH8EJvYJvowP1b/pjqjR0HMVbGYHKDOjFuNOeabrO?= =?us-ascii?Q?kTIPpCjzjxWQ4jWgxkRLFs+XD9wN/WN64WtfNlMKGXJVTZP9Ga+SQ7XAQRuL?= =?us-ascii?Q?Qsvur02+f+EmrcEG00l62qayCmVbPZ09ieZPZOteVbeo5DxksQGY+nUHgcAq?= =?us-ascii?Q?ROc8sYLEw6g9CF8vOKEzpYyvUrQr6OIrtY+64knYsdgW6xoHswuZT3HN4QVE?= =?us-ascii?Q?PjQM+NotgIadc8CMP3JGD0mHgRJigdXVl3FzXyCpQW1TulGAlIy6o7v4AW2B?= =?us-ascii?Q?Rpfj+Rct5cQzKN69jSdZ6LCstKcWxwrbrQCXDVGO66nVR1GeKbosTUblMsIR?= =?us-ascii?Q?B0ycSYSFNQmQgF+WEisfOISADOr5Sk7cO5/+QrpUkC0H4Acg+1p9REScCx/E?= =?us-ascii?Q?d47F2jNfH7Qw0unEDbxDyOGiTDwzVYNmZU9u/kxtpGCmhhuAUv0yvNWyBPDm?= =?us-ascii?Q?R0eSheuvmp1VnBuP7WpGMmRBSeIsGUKptqBWR7V/G2xaXo0TEWUx2XIUCl6B?= =?us-ascii?Q?vkscL92fToARAJGbbUHgDLAzNRfPSUl2kNON0V4DmFt4rP+EGtINIaKi795I?= =?us-ascii?Q?F7F2+AQ+zEnwtooiQHImGLUg+qohJ6fGT25winBYJMXeKCDQY/hJAT4K4Qmu?= =?us-ascii?Q?YsVcP2j8LHA+xdGv65rOqhjT8UmKYtkTpswUOQGGZiXG5x1YiWgVEcgM4kXK?= =?us-ascii?Q?dwA=3D=3D?= X-Microsoft-Antispam-Message-Info: TEBuiTNN+AC/IajzHam6BL4Lf+MDDcpH7HJPoqD9f2o13OB2g8oxY52P9AmZZo26pk4NFjfyRx8REXvoMjKOUWmys/wjeK/fyNrZLNj531u3iVJpu7b44PdB8+4J9EEPggKfwpiJa9sPtmZYtRhN/3ji8QdWZ5LLvenye+KZo61ksSzDrdki9mnCtc7tiQ1j/owowInyo6vlDFlefq2dqT0rsr36YzJf1AdAMestf8t5G7yN6QdAP7ORLTiBRxbCTv4IgXmsvw5jZMWOidjSyq5b+DS3Id2z/EDGWJUI3n/b06Nw4L2VO+/KP0CdpzOLyyNjmdzVa+KBNBu0UEjXA/m7MlR596ERpJcGN/a7+r0= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB1652; 6:EVBM03Binkwe0qmnYnraG1JR0CpgxFu4X7n3nZoxMU0ile7Kl7J0PpSboTppaJHpOPaUhDIi+IylbGHThjNaTP7RQ+ROyUzo1afNlzE1agVP05/h0g49+F2FChQ6WSUrluiN1xEzBYuRN/7iprrMbY8XDeXSPJvFVgpaV6F0JkpCTZGw9YZJ3/nRBbfBOHFyjWFYd1eBxco0OFBm8IBOETYcXXgHixDDoOBm3iTnCj5WRxT1YRXuBv2Jz+hge++ePBo/CXZwyZN/T0gUuATRaBNl5aHXNBmfbApDSH9dxrnfQH/xp2V2hj/SCoOlx2fXTkr7c+dokMm8bylgkOJmawTpC1FsySkd0NnLgC2Dh4Udoknjc9hcfNgD9sHrOQnE9wT2SZmh3X76t1qgvhUX2Bd1RPBUNXIGdE+zTRxh1Eem83s2U7xkyh0P+dAS7J1VIfDiS1Ja4yhMfo18yfkGgg==; 5:CukT6x/ofdxFafmaE4vtSzBLrJtLuYnCwAt2a0R5xDb9tGonNaVvHDQ0QglqJXbnd4uoxASsEem906RIVSGh6ua+HCi1nT/EfOR7BD7gK7L/kEgSeo7Zuoy4eEquSJMm9tW96+TPDzAvptvn3rlVa3if7F+qldKMCmWu4BUrGT8=; 7:VswktMqmwr/oYrt4+BLyKvWO4wi/gfZBB9Cq4ucaXiYsZEuhzid0v3eOjcf58VlW5+kDsoi4gsvPFAls5R0tXqYQZ2hMa/6Vi5YwcD9SJvRLSHTLC3R6VJ1VfsuJWM2HP3SxrLd1wDUwAyrI/pZVlzPJ1WctkTLfFMgfABQxQ6ZfhmkP608vxIYhzsOUuGKne5pW4fTAxvmiWwnfsdgNX2r/mh+ct1oFyZDFEBckDSMWSTG/faij8OQAus0ocCa0 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: aquantia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2018 15:23:41.0956 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5ed4e362-2b70-4c80-47cb-08d614d5e523 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 83e2e134-991c-4ede-8ced-34d47e38e6b1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1652 Subject: [dpdk-dev] [PATCH 21/21] net/atlantic: TX side structures and implementation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2018 15:23:45 -0000 From: Pavel Belous Signed-off-by: Igor Russkikh --- drivers/net/atlantic/atl_ethdev.c | 10 + drivers/net/atlantic/atl_rxtx.c | 566 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 576 insertions(+) diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c index bb4d96bb1..7abae2236 100644 --- a/drivers/net/atlantic/atl_ethdev.c +++ b/drivers/net/atlantic/atl_ethdev.c @@ -297,6 +297,11 @@ static const struct eth_dev_ops atl_eth_dev_ops = { .rx_queue_setup = atl_rx_queue_setup, .rx_queue_release = atl_rx_queue_release, + .tx_queue_start = atl_tx_queue_start, + .tx_queue_stop = atl_tx_queue_stop, + .tx_queue_setup = atl_tx_queue_setup, + .tx_queue_release = atl_tx_queue_release, + .rx_queue_intr_enable = atl_dev_rx_queue_intr_enable, .rx_queue_intr_disable = atl_dev_rx_queue_intr_disable, @@ -324,6 +329,8 @@ static const struct eth_dev_ops atl_eth_dev_ops = { .mac_addr_set = atl_set_default_mac_addr, .set_mc_addr_list = atl_dev_set_mc_addr_list, .rxq_info_get = atl_rxq_info_get, + .txq_info_get = atl_txq_info_get, + .reta_update = atl_reta_update, .reta_query = atl_reta_query, .rss_hash_update = atl_rss_hash_update, @@ -639,6 +646,9 @@ atl_dev_start(struct rte_eth_dev *dev) } } + /* initialize transmission unit */ + atl_tx_init(dev); + /* This can fail when allocating mbufs for descriptor rings */ err = atl_rx_init(dev); if (err) { diff --git a/drivers/net/atlantic/atl_rxtx.c b/drivers/net/atlantic/atl_rxtx.c index 6198f5dfe..bed2265b9 100644 --- a/drivers/net/atlantic/atl_rxtx.c +++ b/drivers/net/atlantic/atl_rxtx.c @@ -46,6 +46,20 @@ #include "hw_atl/hw_atl_b0.h" #include "hw_atl/hw_atl_b0_internal.h" +#define ATL_TX_CKSUM_OFFLOAD_MASK ( \ + PKT_TX_IP_CKSUM | \ + PKT_TX_L4_MASK | \ + PKT_TX_TCP_SEG) + +#define ATL_TX_OFFLOAD_MASK ( \ + PKT_TX_VLAN_PKT | \ + PKT_TX_IP_CKSUM | \ + PKT_TX_L4_MASK | \ + PKT_TX_TCP_SEG) + +#define ATL_TX_OFFLOAD_NOTSUP_MASK \ + (PKT_TX_OFFLOAD_MASK ^ ATL_TX_OFFLOAD_MASK) + /** * Structure associated with each descriptor of the RX ring of a RX queue. */ @@ -54,6 +68,15 @@ struct atl_rx_entry { }; /** + * Structure associated with each descriptor of the TX ring of a TX queue. + */ +struct atl_tx_entry { + struct rte_mbuf *mbuf; + uint16_t next_id; + uint16_t last_id; +}; + +/** * Structure associated with each RX queue. */ struct atl_rx_queue { @@ -72,6 +95,22 @@ struct atl_rx_queue { bool l4_csum_enabled; }; +/** + * Structure associated with each TX queue. + */ +struct atl_tx_queue { + struct hw_atl_txd_s *hw_ring; + uint64_t hw_ring_phys_addr; + struct atl_tx_entry *sw_ring; + uint16_t nb_tx_desc; + uint16_t tx_tail; + uint16_t tx_head; + uint16_t queue_id; + uint16_t port_id; + uint16_t tx_free_thresh; + uint16_t tx_free; +}; + inline static void atl_reset_rx_queue(struct atl_rx_queue *rxq) { @@ -169,6 +208,134 @@ atl_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id, return 0; } +static inline void +atl_reset_tx_queue(struct atl_tx_queue *txq) +{ + struct atl_tx_entry *tx_entry; + union hw_atl_txc_s *txc; + uint16_t i; + + PMD_INIT_FUNC_TRACE(); + + if (!txq) { + PMD_DRV_LOG(ERR, "Pointer to txq is NULL"); + return; + } + + tx_entry = txq->sw_ring; + + for (i = 0; i < txq->nb_tx_desc; i++) { + txc = (union hw_atl_txc_s *)&txq->hw_ring[i]; + txc->flags1 = 0; + txc->flags2 = 2; + } + + for (i = 0; i < txq->nb_tx_desc; i++) { + txq->hw_ring[i].dd = 1; + tx_entry[i].mbuf = NULL; + } + + txq->tx_tail = 0; + txq->tx_head = 0; + txq->tx_free = txq->nb_tx_desc - 1; +} + +int +atl_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id, + uint16_t nb_tx_desc, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf) +{ + struct atl_tx_queue *txq; + const struct rte_memzone *mz; + + PMD_INIT_FUNC_TRACE(); + + /* make sure a valid number of descriptors have been requested */ + if (nb_tx_desc < AQ_HW_MIN_TX_RING_SIZE || nb_tx_desc > AQ_HW_MAX_TX_RING_SIZE) { + PMD_INIT_LOG(ERR, "Number of Tx descriptors must be " + "less than or equal to %d, " + "greater than or equal to %d", AQ_HW_MAX_TX_RING_SIZE, AQ_HW_MIN_TX_RING_SIZE); + return -EINVAL; + } + + /* + * if this queue existed already, free the associated memory. The + * queue cannot be reused in case we need to allocate memory on + * different socket than was previously used. + */ + if (dev->data->tx_queues[tx_queue_id] != NULL) { + atl_tx_queue_release(dev->data->tx_queues[tx_queue_id]); + dev->data->tx_queues[tx_queue_id] = NULL; + } + + /* allocate memory for the queue structure */ + txq = rte_zmalloc_socket("atlantic Tx queue", sizeof(*txq), RTE_CACHE_LINE_SIZE, socket_id); + if (txq == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate queue structure"); + return -ENOMEM; + } + + /* setup queue */ + txq->nb_tx_desc = nb_tx_desc; + txq->port_id = dev->data->port_id; + txq->queue_id = tx_queue_id; + txq->tx_free_thresh = tx_conf->tx_free_thresh; + + + /* allocate memory for the software ring */ + txq->sw_ring = rte_zmalloc_socket("atlantic sw tx ring", + nb_tx_desc * sizeof(struct atl_tx_entry), + RTE_CACHE_LINE_SIZE, socket_id); + if (txq->sw_ring == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate software ring"); + rte_free(txq); + return -ENOMEM; + } + + /* + * allocate memory for the hardware descriptor ring. A memzone large + * enough to hold the maximum ring size is requested to allow for + * resizing in later calls to the queue setup function. + */ + mz = rte_eth_dma_zone_reserve(dev, "tx hw_ring", tx_queue_id, + HW_ATL_B0_MAX_TXD * sizeof(struct hw_atl_txd_s), + 128, socket_id); + if (mz == NULL) { + PMD_INIT_LOG(ERR, "Cannot allocate hardware ring"); + rte_free(txq->sw_ring); + rte_free(txq); + return -ENOMEM; + } + txq->hw_ring = mz->addr; + txq->hw_ring_phys_addr = mz->iova; + + atl_reset_tx_queue(txq); + + dev->data->tx_queues[tx_queue_id] = txq; + return 0; +} + +int +atl_tx_init(struct rte_eth_dev *eth_dev) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private); + struct atl_tx_queue *txq; + uint64_t base_addr = 0; + int i = 0; + int err = 0; + + PMD_INIT_FUNC_TRACE(); + + for (i =0 ; i < eth_dev->data->nb_tx_queues; i++) { + txq = eth_dev->data->tx_queues[i]; + base_addr = txq->hw_ring_phys_addr; + + err = hw_atl_b0_hw_ring_tx_init(hw, base_addr, txq->queue_id, txq->nb_tx_desc, 0, txq->port_id); + } + + return err; +} + int atl_rx_init(struct rte_eth_dev *eth_dev) { @@ -324,6 +491,77 @@ atl_rx_queue_release(void *rx_queue) } } +static void +atl_tx_queue_release_mbufs(struct atl_tx_queue *txq) +{ + int i; + + PMD_INIT_FUNC_TRACE(); + + if (txq->sw_ring != NULL) { + for (i = 0; i < txq->nb_tx_desc; i++) { + if (txq->sw_ring[i].mbuf != NULL) { + rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf); + txq->sw_ring[i].mbuf = NULL; + } + } + } +} + +int +atl_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + PMD_INIT_FUNC_TRACE(); + + if (tx_queue_id < dev->data->nb_tx_queues) { + hw_atl_b0_hw_ring_tx_start(hw, tx_queue_id); + + rte_wmb(); + hw_atl_b0_hw_tx_ring_tail_update(hw, 0, tx_queue_id); + dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED; + } else { + return -1; + } + + return 0; +} + +int +atl_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id) +{ + struct aq_hw_s *hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + struct atl_tx_queue *txq; + + PMD_INIT_FUNC_TRACE(); + + txq = dev->data->tx_queues[tx_queue_id]; + + hw_atl_b0_hw_ring_tx_stop(hw, tx_queue_id); + + atl_tx_queue_release_mbufs(txq); + atl_reset_tx_queue(txq); + dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +} + +void +atl_tx_queue_release(void *tx_queue) +{ + + PMD_INIT_FUNC_TRACE(); + + if (tx_queue != NULL) { + struct atl_tx_queue *txq = (struct atl_tx_queue *)tx_queue; + + atl_tx_queue_release_mbufs(txq); + rte_free(txq->sw_ring); + rte_free(txq); + } +} + void atl_free_queues(struct rte_eth_dev *dev) { @@ -337,6 +575,11 @@ atl_free_queues(struct rte_eth_dev *dev) } dev->data->nb_rx_queues = 0; + for (i = 0; i < dev->data->nb_tx_queues; i++) { + atl_tx_queue_release(dev->data->tx_queues[i]); + dev->data->tx_queues[i] = 0; + } + dev->data->nb_tx_queues = 0; } int @@ -346,6 +589,13 @@ atl_start_queues(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + if (atl_tx_queue_start(dev, i) != 0) { + PMD_DRV_LOG(ERR, "Start Tx queue %d failed", i); + return -1; + } + } + for (i = 0; i < dev->data->nb_rx_queues; i++) { if (atl_rx_queue_start(dev, i) != 0) { PMD_DRV_LOG(ERR, "Start Rx queue %d failed", i); @@ -363,6 +613,13 @@ atl_stop_queues(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); + for (i = 0; i < dev->data->nb_tx_queues; i++) { + if (atl_tx_queue_stop(dev, i) != 0) { + PMD_DRV_LOG(ERR, "Stop Tx queue %d failed", i); + return -1; + } + } + for (i = 0; i < dev->data->nb_rx_queues; i++) { if (atl_rx_queue_stop(dev, i) != 0) { PMD_DRV_LOG(ERR, "Stop Rx queue %d failed", i); @@ -387,6 +644,18 @@ atl_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_rxq_ qinfo->nb_desc = rxq->nb_rx_desc; } +void +atl_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id, struct rte_eth_txq_info *qinfo) +{ + struct atl_tx_queue *txq; + + PMD_INIT_FUNC_TRACE(); + + txq = dev->data->tx_queues[queue_id]; + + qinfo->nb_desc = txq->nb_tx_desc; +} + /* Return Rx queue avail count */ uint32_t @@ -459,6 +728,31 @@ atl_dev_rx_descriptor_status(void *rx_queue, uint16_t offset) return RTE_ETH_RX_DESC_AVAIL; } +int +atl_dev_tx_descriptor_status(void *tx_queue, uint16_t offset) +{ + struct atl_tx_queue *txq = tx_queue; + struct hw_atl_txd_s *txd; + uint32_t idx; + + PMD_INIT_FUNC_TRACE(); + + if (unlikely(offset >= txq->nb_tx_desc)) + return -EINVAL; + + idx = txq->tx_tail + offset; + + if (idx >= txq->nb_tx_desc) + idx -= txq->nb_tx_desc; + + txd = &txq->hw_ring[idx]; + + if (txd->dd) + return RTE_ETH_TX_DESC_DONE; + + return RTE_ETH_TX_DESC_FULL; +} + static int atl_rx_enable_intr(struct rte_eth_dev *dev, uint16_t queue_id, bool enable) { @@ -495,6 +789,46 @@ atl_dev_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t queue_id) return atl_rx_enable_intr(eth_dev, queue_id, false); } +uint16_t +atl_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + int i, ret; + uint64_t ol_flags; + struct rte_mbuf *m; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < nb_pkts; i++) { + m = tx_pkts[i]; + ol_flags = m->ol_flags; + + if (m->nb_segs > AQ_HW_MAX_SEGS_SIZE) { + rte_errno = -EINVAL; + return i; + } + + if (ol_flags & ATL_TX_OFFLOAD_NOTSUP_MASK) { + rte_errno = -ENOTSUP; + return i; + } + +#ifdef RTE_LIBRTE_ETHDEV_DEBUG + ret = rte_validate_tx_offload(m); + if (ret != 0) { + rte_errno = ret; + return i; + } +#endif + ret = rte_net_intel_cksum_prepare(m); + if (ret != 0) { + rte_errno = ret; + return i; + } + } + + return i; +} + static uint64_t atl_desc_to_offload_flags(struct atl_rx_queue *rxq, struct hw_atl_rxd_wb_s *rxd_wb) { @@ -756,3 +1090,235 @@ atl_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return nb_rx; } + +static void +atl_xmit_cleanup(struct atl_tx_queue *txq) +{ + struct atl_tx_entry *sw_ring; + struct hw_atl_txd_s *txd; + int to_clean = 0; + + PMD_INIT_FUNC_TRACE(); + + if (txq != NULL) { + sw_ring = txq->sw_ring; + int head = txq->tx_head; + int cnt; + int i; + + for (i = 0, cnt = head; ; i++) { + txd = &txq->hw_ring[cnt]; + + if (txd->dd) + to_clean++; + + cnt = (cnt + 1) % txq->nb_tx_desc; + if (cnt == txq->tx_tail) + break; + } + + if (to_clean == 0) + return; + + while (to_clean) { + txd = &txq->hw_ring[head]; + + struct atl_tx_entry *rx_entry = &sw_ring[head]; + + if (rx_entry->mbuf) { + rte_pktmbuf_free_seg(rx_entry->mbuf); + rx_entry->mbuf = NULL; + } + + if (txd->dd) { + to_clean--; + } + + txd->buf_addr = 0; + txd->flags = 0; + + head = (head + 1) % txq->nb_tx_desc; + txq->tx_free++; + } + + txq->tx_head = head; + } +} + +static int +atl_tso_setup(struct rte_mbuf *tx_pkt, union hw_atl_txc_s *txc) +{ + uint32_t tx_cmd = 0; + uint64_t ol_flags = tx_pkt->ol_flags; + + PMD_INIT_FUNC_TRACE(); + + if (ol_flags & PKT_TX_TCP_SEG) { + PMD_DRV_LOG(DEBUG, "xmit TSO pkt"); + + tx_cmd |= tx_desc_cmd_lso | tx_desc_cmd_l4cs; + + txc->cmd = 0x4; + + if (ol_flags & PKT_TX_IPV6) + txc->cmd |= 0x2; + + txc->l2_len = tx_pkt->l2_len; + txc->l3_len = tx_pkt->l3_len; + txc->l4_len = tx_pkt->l4_len; + + txc->mss_len = tx_pkt->tso_segsz; + } + + if (ol_flags & PKT_TX_VLAN_PKT) { + tx_cmd |= tx_desc_cmd_vlan; + txc->vlan_tag = tx_pkt->vlan_tci; + } + + if (tx_cmd) { + txc->type = tx_desc_type_ctx; + txc->idx = 0; + } + + return tx_cmd; +} + +static inline void +atl_setup_csum_offload(struct rte_mbuf *mbuf, struct hw_atl_txd_s *txd, uint32_t tx_cmd) +{ + txd->cmd |= tx_desc_cmd_fcs; + txd->cmd |= (mbuf->ol_flags & PKT_TX_IP_CKSUM) ? tx_desc_cmd_ipv4 : 0; + /* L4 csum requested */ + txd->cmd |= (mbuf->ol_flags & PKT_TX_L4_MASK) ? tx_desc_cmd_l4cs : 0; + txd->cmd |= tx_cmd; +} + +static inline void +atl_xmit_pkt(struct aq_hw_s *hw, struct atl_tx_queue *txq, struct rte_mbuf *tx_pkt) +{ + struct atl_adapter *adapter = ATL_DEV_TO_ADAPTER(&rte_eth_devices[txq->port_id]); + uint32_t pay_len = 0; + int tail = 0; + struct atl_tx_entry *tx_entry; + uint64_t buf_dma_addr; + struct rte_mbuf *m_seg; + union hw_atl_txc_s *txc = NULL; + struct hw_atl_txd_s *txd = NULL; + u32 tx_cmd = 0U; + int desc_count = 0; + + PMD_INIT_FUNC_TRACE(); + + tail = txq->tx_tail; + + txc = (union hw_atl_txc_s *)&txq->hw_ring[tail]; + + txc->flags1 = 0U; + txc->flags2 = 0U; + + tx_cmd = atl_tso_setup(tx_pkt, txc); + + if (tx_cmd) { + /* We've consumed the first desc, adjust counters */ + tail = (tail + 1) % txq->nb_tx_desc; + txq->tx_tail = tail; + txq->tx_free -= 1; + + txd = &txq->hw_ring[tail]; + txd->flags = 0U; + } else { + txd = (struct hw_atl_txd_s *)txc; + } + + txd->ct_en = !!tx_cmd; + + txd->type = tx_desc_type_desc; + + atl_setup_csum_offload(tx_pkt, txd, tx_cmd); + + if (tx_cmd) { + txd->ct_idx = 0; + } + + pay_len = tx_pkt->pkt_len; + + txd->pay_len = pay_len; + + for (m_seg = tx_pkt; m_seg; m_seg = m_seg->next) { + if (desc_count > 0) { + txd = &txq->hw_ring[tail]; + txd->flags = 0U; + } + + buf_dma_addr = rte_mbuf_data_iova(m_seg); + txd->buf_addr = rte_cpu_to_le_64(buf_dma_addr); + + txd->type = tx_desc_type_desc; + txd->len = m_seg->data_len; + txd->pay_len = pay_len; + + /* Store mbuf for freeing later */ + tx_entry = &txq->sw_ring[tail]; + + if (tx_entry->mbuf) + rte_pktmbuf_free_seg(tx_entry->mbuf); + tx_entry->mbuf = m_seg; + + tail = (tail + 1) % txq->nb_tx_desc; + + desc_count++; + } + + // Last descriptor requires EOP and WB + txd->eop = 1U; + txd->cmd |= tx_desc_cmd_wb; + + hw_atl_b0_hw_tx_ring_tail_update(hw, tail, txq->queue_id); + + txq->tx_tail = tail; + + txq->tx_free -= desc_count; + + adapter->sw_stats.q_opackets[txq->queue_id]++; + adapter->sw_stats.q_obytes[txq->queue_id] += pay_len; +} + +uint16_t +atl_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + struct rte_eth_dev *dev = NULL; + struct aq_hw_s *hw = NULL; + struct atl_tx_queue *txq = tx_queue; + struct rte_mbuf *tx_pkt; + uint16_t nb_tx; + + dev = &rte_eth_devices[txq->port_id]; + hw = ATL_DEV_PRIVATE_TO_HW(dev->data->dev_private); + + PMD_TX_LOG(DEBUG, "txq%d pkts: %d tx_free=%d tx_tail=%d tx_head=%d", + txq->queue_id, nb_pkts, txq->tx_free, txq->tx_tail, txq->tx_head); + + for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) { + tx_pkt = *tx_pkts++; + + /* Clean Tx queue if needed */ + if (txq->tx_free < txq->tx_free_thresh) + atl_xmit_cleanup(txq); + + /* Check if we have enough free descriptors */ + if (txq->tx_free < tx_pkt->nb_segs) + break; + + /* check mbuf is valid */ + if ((tx_pkt->nb_segs == 0) || ((tx_pkt->nb_segs > 1) && (tx_pkt->next == NULL))) + break; + + /* Send the packet */ + atl_xmit_pkt(hw, txq, tx_pkt); + } + + PMD_TX_LOG(DEBUG, "atl_xmit_pkts %d transmitted", nb_tx); + + return nb_tx; +} + -- 2.13.3.windows.1