From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0088.outbound.protection.outlook.com [207.46.100.88]) by dpdk.org (Postfix) with ESMTP id 3C512AD90 for ; Tue, 14 Jun 2016 21:08:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Bcp3HsGcSktajT9l1rZu09QDyJYum0u4m5EAb/62mBo=; b=omM8Detund+ZJNlfKuIEG7IUa+wv/Su4y2c0ATa7mnwvN7JyF6OczVaBtjFKkwv+/pD5rNjk8BfROlD8tIfD0HLuXV0bC/8E5g2KuFmpUWcAn2lUo2Ge9SUxrqklc1WszFLbg4PKHU1oOAngaRGpa6rrRgsH8ynjmE/YOHkykyU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.caveonetworks.com (111.93.218.67) by BY1PR0701MB1723.namprd07.prod.outlook.com (10.162.111.142) with Microsoft SMTP Server (TLS) id 15.1.511.8; Tue, 14 Jun 2016 19:08:17 +0000 From: Jerin Jacob To: CC: , , , Jerin Jacob , Maciej Czekaj , Kamil Rytarowski , Zyta Szpak , Slawomir Rosek , Radoslaw Biernacki Date: Wed, 15 Jun 2016 00:36:33 +0530 Message-ID: <1465931200-23912-19-git-send-email-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1465931200-23912-1-git-send-email-jerin.jacob@caviumnetworks.com> References: <1465826143-22159-1-git-send-email-jerin.jacob@caviumnetworks.com> <1465931200-23912-1-git-send-email-jerin.jacob@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MA1PR01CA0040.INDPRD01.PROD.OUTLOOK.COM (10.164.116.140) To BY1PR0701MB1723.namprd07.prod.outlook.com (10.162.111.142) X-MS-Office365-Filtering-Correlation-Id: b8f99f18-0c4b-4cfc-fac4-08d394873f82 X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1723; 2:yETpUsBm/WDJVBOmy6c36foPGBpEaAeW72k7MFZTLHYdgOC6EPIfDU3WSC8eiQJDi9mVEJRw7oTQj5GcFRWJDTuGLyLn9JXbhHz39hOgjjnI+dxAsclTlTvfGKg+CKA3vpA7+5rYMLqmHaWyY270/MOHfxUWiUAGfwAutqpD2xrNt54+rhI3l+CoFVa3TuVw; 3:W69Q0ighw+OFz8N5q+Z999jxY+TM2OKuWl6JtOP6XRyW+nbkr8RODw6hjL+G6Qg2olVx3vo06v8MCQr8bgied4Md9wuVifKVD8YnAjJ3yK3anhFzj60bWjotaipXkzVp; 25:X0Ncv27K6QChGIdNbP9gcIfL6usrYprA0EnOs3C9YB0OpMztaOJ+znwWbLL+0yKeBjLtEMQ01fby4iv7JwsjrTztpv+3aHsqGjSvZMWbtLA4/KEn5hB3mBsl8ziZM7gKwzFhrGZwr3o2DFdUzvhR1EnVwTFvQiWeOTGcW4h19GS/jMCtrt2mmPPhtJEiy+JwxsfC5nGYw4+wr1XMC1NfhgjflQkZROl96rzFEOf10ws5yQiluZajqKrpe2xeBT2dFyf13qsPL4+xXOdBBoumII/h16n0Wrq29z3A2smXy/DR7lS4Lq1ad5LwTblir+BPOVYgsG7kVeSCUiqv+smOQ4zd+O9KLzqUXJh4+ikiPyWzEJtXIVOdeDO12asAp1UND0kPDsM7ZmS0y9R+P5e2a+wmZf7df8/5B/Gk5Sd8mTY= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR0701MB1723; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1723; 20:f5pipnukP7z77zcxmVk1aqqfAL9yYdAyBQ4Q7Tqm1/BPuCHf3gEq8O3zRKmJAxNHKM+1M5DdM24xb6IJrhIB1ES18a0WtWszSiOMyUiTwRJiAKj70BbKtKUaIGeIMqhVqbwZVdL6EvmotTJI1AIimyjt/eIUQndzQjyAZX3c6dAl+4H5rkRKXtI/kiWtZuA/S6FUan43t19n0H9isJceRrD7Eu3UaopLEIeL3VLmrDnzN0Vqwsctsx6kLJqhWxR1W2nUMzv7uAd3H9jXrqwIeh7AkD+pPxdWSgCri9QNlVTN6waVdi9bJNPTVHTAaxu8xzF3hM/G/BuQYgkRM1zEWEGziAVcGFUQTRKbJCL5cxqoWtgjq1Z+4xdKiKJjXMsCddc2691yuCCC8dT/53/UCuazM0dV+Pndt7R6YHQYCbcYP4uOW78AouuvVHGMxOozhalZxfphoxJyQKoqs0PrZVDjFeGM5fmy3fgAZ1f9bSZD81dTSvZeNyYIIcRdX7txAtQcTgThNH6UHmKcr5+eZB3Udz1bk0I5Uj4d2b/9Da0cwFKjGzw/dG2F6Bd0LGmxTTUU0LIEctXFtppRzkhz1+A8v+8DEmqVW/2XiMCsvDg= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:BY1PR0701MB1723; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0701MB1723; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1723; 4:lSrAea4d7rPdWy+1uyJwJMd6PPDpfnzT2BccdZ81MQX4sXZ6AVH+VMKAPQkRGikpePgMsYNakOCmj1T3liZpNggvFmtoBhXknB31tuDPalGLB4NMQhPc+MiXLj38ExB70SwQYAloFtUM6yRkLpQQk6MMlDiI3JntQKuvwJdtRrPWlSFIZu/wHu02ARlFTVTw1n8y/0qdahWT/fZRBhVYdLbqFcW9ks/Crjzi++sesB2tbGkVGn36FzMpUhLVqexGa9VA8xD5pZB1u8eEybANohvpefMMTV7tXbqfPV4Q0bbAUZAsaovStYSxjGED0qxdTIJk6ovtp911bCrPhLS6PjPaOS1hpKX8eeW3ARzodpkHk0ccj60QGVG4EgjIGPaM X-Forefront-PRVS: 09730BD177 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(189002)(199003)(69596002)(189998001)(2950100001)(48376002)(77096005)(36756003)(19580405001)(5004730100002)(19580395003)(76176999)(5009440100003)(50986999)(97736004)(5003940100001)(68736007)(110136002)(92566002)(101416001)(5008740100001)(81166006)(8676002)(229853001)(81156014)(66066001)(4326007)(50466002)(2906002)(105586002)(106356001)(2351001)(47776003)(53416004)(76506005)(42186005)(33646002)(586003)(3846002)(6116002)(50226002)(7099028); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0701MB1723; H:localhost.caveonetworks.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0701MB1723; 23:cnoySOIcNFrEvMb79EXRcRvjgSOyJHQ6nK2ixt0?= =?us-ascii?Q?PO9ICn9U4PQ2tP6wTakSNfnnpA3j5RCFyjXdgZ6bzwiDlH4I+QXfyVoN67gY?= =?us-ascii?Q?UUYByDkL3v1/MOwYUQ8ytYPA5DXXBaKw1B64vbLqaAcBh/K7nJ8L+O2itJ84?= =?us-ascii?Q?19nB2CQyDUmtg5NbAuLNBon+qx7yCNXR6gh/UxrSRkHzdFXu6tHt2H4Fn8K9?= =?us-ascii?Q?NFK0w6Q9bhKsEZdzJjcx19B3GZseEqLL+cNKiMUJjCj5AV+6murz1dI7rfX1?= =?us-ascii?Q?OE0dKXeb9BK0hSHaMsnD8LAXs8DWyb+h24z3hHL6DbU37/H3rvRKlaiDD1N3?= =?us-ascii?Q?nK3W1W4BuUf1vpyzwMZ14G9h37wv/deU0X3QjakFhURgsHmVObxj4Ol68lfL?= =?us-ascii?Q?hwhFSoxe4zlqWUNh05XEg6pVraVLo6G8xQFMVLEMQ4FfTbMD0J7OfjHU/Bmf?= =?us-ascii?Q?EBK3YnIO+sun0R9NEQ4F7WrgMci93Jxt6pYYAto1yLKL+RY6mmZhp9zUPIu/?= =?us-ascii?Q?YVTF43HAu33DTajhh52TTuejvypsFQ7YD3PH1BLClzlCfDtQNM2xl4XmzImn?= =?us-ascii?Q?lw/lZoVEBp+pYGaXu8QWIsBvKBKEZ2XOa5yznoBpqorSy3iYvSbkR57nV24r?= =?us-ascii?Q?wm4TjoQxzxKDYph8zEiZoHxDuPRfiqdnKG7Vt3fxvHn6tGQY+vimfX8HaDae?= =?us-ascii?Q?/o1xIL+x/40yKOPq4bCikFIIVwZKczsV7L9PMClCwAUQsmqBqYIzuHwMcw1M?= =?us-ascii?Q?PCARbHfWLikGsCCskdUW1U83Sr2iO8zAaFUIlni1qqNWUgxUviqPekxrDxyR?= =?us-ascii?Q?Zg9GhjjWMF0Y+YU8lPeZ8eU8Q6OBsvhY7kFW3jacxyzVcsdfiQevp8qIU8Kc?= =?us-ascii?Q?ohazwi0FCfpR3czdMki2iDATb9J3qzvGCmYDcRdJLsUU7MxWV6VqRYkgydog?= =?us-ascii?Q?PTzEbcnaPGa+DTc8BOKPR3JEvX+u6t+SwYV+XEUr9CyaYN6tKEcGOcS2284f?= =?us-ascii?Q?PKS7Khg6U4Jqh5isVORqSGG232uVEKtFC6M6M8xiBW32p7HNa0W9zpohAW5P?= =?us-ascii?Q?PzfVFaKHun7KxvO5id14kqT5TrioKJqMtJXjs8GSqGqTauVO678W/S1H6Oi3?= =?us-ascii?Q?wgQYiw9wYdezD0w4vySNGCxOtY8Z88QvxJAbyJJ9+r+P8UEnT2AP1V7TvZEb?= =?us-ascii?Q?G1MHiejZixOcOoic=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1723; 5:lygiNimTImRFjm5HE2YNdZGRyRy80IRUxfc66+zTUT+urT8js1D/iS+ywl2s5+31wLUVm6uc654zsnrWwRj4e7vhs1klTNmVkdk9pZye3Wzjl0gzQYFz+ROVoR9lKJH1/S8XNg0ORHyuEumJgCCbpQ==; 24:uMXwpvoQZV4X9NlHdVQC/X+K08MOGAG5QQ9tbZtGjCZczAuqL6/aTdyQEUihaVcNjVusrWOgZbKQb7H15KP2lKtWJcokrT+mfgbzHiO7Mro=; 7:yMVAhKdvCB3Pt6jwolRDglpcDzdZphX3o1mbu0FrKRpGc+4/cQxi875KaCLIKco2CcKDnteIpxZscP9JWh4lyE7csicNrfllekeM4fqabs/B6KRPOkG4Kptqk4CBVrhJDZoGK5q1W0dfQOecABgRnV3L9MnpbTA6g6rd1YdZ67E6trtWrn5ug8fC6NkEiGasDbpGl35b5ozYzQwfV/kA9Q== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2016 19:08:17.0008 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0701MB1723 Subject: [dpdk-dev] [PATCH v5 18/25] net/thunderx: add single and multi segment Tx functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2016 19:08:22 -0000 Signed-off-by: Jerin Jacob Signed-off-by: Maciej Czekaj Signed-off-by: Kamil Rytarowski Signed-off-by: Zyta Szpak Signed-off-by: Slawomir Rosek Signed-off-by: Radoslaw Biernacki --- drivers/net/thunderx/Makefile | 2 + drivers/net/thunderx/nicvf_ethdev.c | 5 +- drivers/net/thunderx/nicvf_rxtx.c | 255 ++++++++++++++++++++++++++++++++++++ drivers/net/thunderx/nicvf_rxtx.h | 93 +++++++++++++ 4 files changed, 354 insertions(+), 1 deletion(-) create mode 100644 drivers/net/thunderx/nicvf_rxtx.c create mode 100644 drivers/net/thunderx/nicvf_rxtx.h diff --git a/drivers/net/thunderx/Makefile b/drivers/net/thunderx/Makefile index eb9f100..9079b5b 100644 --- a/drivers/net/thunderx/Makefile +++ b/drivers/net/thunderx/Makefile @@ -51,10 +51,12 @@ VPATH += $(SRCDIR)/base # # all source are stored in SRCS-y # +SRCS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += nicvf_rxtx.c SRCS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += nicvf_hw.c SRCS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += nicvf_mbox.c SRCS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += nicvf_ethdev.c +CFLAGS_nicvf_rxtx.o += -fno-prefetch-loop-arrays -Ofast # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += lib/librte_eal lib/librte_ether diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 19ad85a..15f5cfc 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -66,7 +66,7 @@ #include "base/nicvf_plat.h" #include "nicvf_ethdev.h" - +#include "nicvf_rxtx.h" #include "nicvf_logs.h" static inline int @@ -617,6 +617,9 @@ nicvf_dev_tx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, (tx_conf->tx_free_thresh == NICVF_DEFAULT_TX_FREE_THRESH ? NICVF_TX_FREE_MPOOL_THRESH : tx_conf->tx_free_thresh); + txq->pool_free = nicvf_multi_pool_free_xmited_buffers; + } else { + txq->pool_free = nicvf_single_pool_free_xmited_buffers; } /* Allocate software ring */ diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c new file mode 100644 index 0000000..88a5152 --- /dev/null +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -0,0 +1,255 @@ +/* + * BSD LICENSE + * + * Copyright (C) Cavium networks Ltd. 2016. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "base/nicvf_plat.h" + +#include "nicvf_ethdev.h" +#include "nicvf_rxtx.h" +#include "nicvf_logs.h" + +static inline void __hot +fill_sq_desc_header(union sq_entry_t *entry, struct rte_mbuf *pkt) +{ + /* Local variable sqe to avoid read from sq desc memory*/ + union sq_entry_t sqe; + uint64_t ol_flags; + + /* Fill SQ header descriptor */ + sqe.buff[0] = 0; + sqe.hdr.subdesc_type = SQ_DESC_TYPE_HEADER; + /* Number of sub-descriptors following this one */ + sqe.hdr.subdesc_cnt = pkt->nb_segs; + sqe.hdr.tot_len = pkt->pkt_len; + + ol_flags = pkt->ol_flags & NICVF_TX_OFFLOAD_MASK; + if (unlikely(ol_flags)) { + /* L4 cksum */ + if (ol_flags & PKT_TX_TCP_CKSUM) + sqe.hdr.csum_l4 = SEND_L4_CSUM_TCP; + else if (ol_flags & PKT_TX_UDP_CKSUM) + sqe.hdr.csum_l4 = SEND_L4_CSUM_UDP; + else + sqe.hdr.csum_l4 = SEND_L4_CSUM_DISABLE; + sqe.hdr.l4_offset = pkt->l3_len + pkt->l2_len; + + /* L3 cksum */ + if (ol_flags & PKT_TX_IP_CKSUM) { + sqe.hdr.csum_l3 = 1; + sqe.hdr.l3_offset = pkt->l2_len; + } + } + + entry->buff[0] = sqe.buff[0]; +} + +void __hot +nicvf_single_pool_free_xmited_buffers(struct nicvf_txq *sq) +{ + int j = 0; + uint32_t curr_head; + uint32_t head = sq->head; + struct rte_mbuf **txbuffs = sq->txbuffs; + void *obj_p[NICVF_MAX_TX_FREE_THRESH] __rte_cache_aligned; + + curr_head = nicvf_addr_read(sq->sq_head) >> 4; + while (head != curr_head) { + if (txbuffs[head]) + obj_p[j++] = txbuffs[head]; + + head = (head + 1) & sq->qlen_mask; + } + + rte_mempool_put_bulk(sq->pool, obj_p, j); + sq->head = curr_head; + sq->xmit_bufs -= j; + NICVF_TX_ASSERT(sq->xmit_bufs >= 0); +} + +void __hot +nicvf_multi_pool_free_xmited_buffers(struct nicvf_txq *sq) +{ + uint32_t n = 0; + uint32_t curr_head; + uint32_t head = sq->head; + struct rte_mbuf **txbuffs = sq->txbuffs; + + curr_head = nicvf_addr_read(sq->sq_head) >> 4; + while (head != curr_head) { + if (txbuffs[head]) { + rte_pktmbuf_free_seg(txbuffs[head]); + n++; + } + + head = (head + 1) & sq->qlen_mask; + } + + sq->head = curr_head; + sq->xmit_bufs -= n; + NICVF_TX_ASSERT(sq->xmit_bufs >= 0); +} + +static inline uint32_t __hot +nicvf_free_tx_desc(struct nicvf_txq *sq) +{ + return ((sq->head - sq->tail - 1) & sq->qlen_mask); +} + +/* Send Header + Packet */ +#define TX_DESC_PER_PKT 2 + +static inline uint32_t __hot +nicvf_free_xmitted_buffers(struct nicvf_txq *sq, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + uint32_t free_desc = nicvf_free_tx_desc(sq); + + if (free_desc < nb_pkts * TX_DESC_PER_PKT || + sq->xmit_bufs > sq->tx_free_thresh) { + if (unlikely(sq->pool == NULL)) + sq->pool = tx_pkts[0]->pool; + + sq->pool_free(sq); + /* Freed now, let see the number of free descs again */ + free_desc = nicvf_free_tx_desc(sq); + } + return free_desc; +} + +uint16_t __hot +nicvf_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) +{ + int i; + uint32_t free_desc; + uint32_t tail; + struct nicvf_txq *sq = tx_queue; + union sq_entry_t *desc_ptr = sq->desc; + struct rte_mbuf **txbuffs = sq->txbuffs; + struct rte_mbuf *pkt; + uint32_t qlen_mask = sq->qlen_mask; + + tail = sq->tail; + free_desc = nicvf_free_xmitted_buffers(sq, tx_pkts, nb_pkts); + + for (i = 0; i < nb_pkts && (int)free_desc >= TX_DESC_PER_PKT; i++) { + pkt = tx_pkts[i]; + + txbuffs[tail] = NULL; + fill_sq_desc_header(desc_ptr + tail, pkt); + tail = (tail + 1) & qlen_mask; + + txbuffs[tail] = pkt; + fill_sq_desc_gather(desc_ptr + tail, pkt); + tail = (tail + 1) & qlen_mask; + free_desc -= TX_DESC_PER_PKT; + } + + sq->tail = tail; + sq->xmit_bufs += i; + rte_wmb(); + + /* Inform HW to xmit the packets */ + nicvf_addr_write(sq->sq_door, i * TX_DESC_PER_PKT); + return i; +} + +uint16_t __hot +nicvf_xmit_pkts_multiseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + int i, k; + uint32_t used_desc, next_used_desc, used_bufs, free_desc, tail; + struct nicvf_txq *sq = tx_queue; + union sq_entry_t *desc_ptr = sq->desc; + struct rte_mbuf **txbuffs = sq->txbuffs; + struct rte_mbuf *pkt, *seg; + uint32_t qlen_mask = sq->qlen_mask; + uint16_t nb_segs; + + tail = sq->tail; + used_desc = 0; + used_bufs = 0; + + free_desc = nicvf_free_xmitted_buffers(sq, tx_pkts, nb_pkts); + + for (i = 0; i < nb_pkts; i++) { + pkt = tx_pkts[i]; + + nb_segs = pkt->nb_segs; + + next_used_desc = used_desc + nb_segs + 1; + if (next_used_desc > free_desc) + break; + used_desc = next_used_desc; + used_bufs += nb_segs; + + txbuffs[tail] = NULL; + fill_sq_desc_header(desc_ptr + tail, pkt); + tail = (tail + 1) & qlen_mask; + + txbuffs[tail] = pkt; + fill_sq_desc_gather(desc_ptr + tail, pkt); + tail = (tail + 1) & qlen_mask; + + seg = pkt->next; + for (k = 1; k < nb_segs; k++) { + txbuffs[tail] = seg; + fill_sq_desc_gather(desc_ptr + tail, seg); + tail = (tail + 1) & qlen_mask; + seg = seg->next; + } + } + + sq->tail = tail; + sq->xmit_bufs += used_bufs; + rte_wmb(); + + /* Inform HW to xmit the packets */ + nicvf_addr_write(sq->sq_door, used_desc); + return nb_pkts; +} diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h new file mode 100644 index 0000000..b1fdc69 --- /dev/null +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -0,0 +1,93 @@ +/* + * BSD LICENSE + * + * Copyright (C) Cavium networks Ltd. 2016. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __THUNDERX_NICVF_RXTX_H__ +#define __THUNDERX_NICVF_RXTX_H__ + +#include + +#define NICVF_TX_OFFLOAD_MASK (PKT_TX_IP_CKSUM | PKT_TX_L4_MASK) + +#ifndef __hot +#define __hot __attribute__((hot)) +#endif + +#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN +static inline uint16_t __attribute__((const)) +nicvf_frag_num(uint16_t i) +{ + return (i & ~3) + 3 - (i & 3); +} + +static inline void __hot +fill_sq_desc_gather(union sq_entry_t *entry, struct rte_mbuf *pkt) +{ + /* Local variable sqe to avoid read from sq desc memory*/ + union sq_entry_t sqe; + + /* Fill the SQ gather entry */ + sqe.buff[0] = 0; sqe.buff[1] = 0; + sqe.gather.subdesc_type = SQ_DESC_TYPE_GATHER; + sqe.gather.ld_type = NIC_SEND_LD_TYPE_E_LDT; + sqe.gather.size = pkt->data_len; + sqe.gather.addr = rte_mbuf_data_dma_addr(pkt); + + entry->buff[0] = sqe.buff[0]; + entry->buff[1] = sqe.buff[1]; +} + +#else + +static inline uint16_t __attribute__((const)) +nicvf_frag_num(uint16_t i) +{ + return i; +} + +static inline void __hot +fill_sq_desc_gather(union sq_entry_t *entry, struct rte_mbuf *pkt) +{ + entry->buff[0] = (uint64_t)SQ_DESC_TYPE_GATHER << 60 | + (uint64_t)NIC_SEND_LD_TYPE_E_LDT << 58 | + pkt->data_len; + entry->buff[1] = rte_mbuf_data_dma_addr(pkt); +} +#endif + +uint16_t nicvf_xmit_pkts(void *txq, struct rte_mbuf **tx_pkts, uint16_t pkts); +uint16_t nicvf_xmit_pkts_multiseg(void *txq, struct rte_mbuf **tx_pkts, + uint16_t pkts); + +void nicvf_single_pool_free_xmited_buffers(struct nicvf_txq *sq); +void nicvf_multi_pool_free_xmited_buffers(struct nicvf_txq *sq); + +#endif /* __THUNDERX_NICVF_RXTX_H__ */ -- 2.5.5