From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 514BCA0A03; Mon, 18 Jan 2021 10:38:26 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A4956140D9F; Mon, 18 Jan 2021 10:37:20 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D2A06140D24 for ; Mon, 18 Jan 2021 10:37:09 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 10I9QQ3O001359 for ; Mon, 18 Jan 2021 01:37:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=LLyeU5TVlbdouCnihdYtw7WbQbeoforvg/JpLIpvcrs=; b=ARs9Shfb1D/hCqBkHWG2ZtbCDrCvtqYdXUPrcp/lEMo8exMwgAEgUCj+xLGTqgZwEcEo k/1cFgrd01i5Siw/ojSlx0tGKBouOAmFygBTPQwUOPZSOQ0Y1ZTGQbqNwJhueh/tP2L7 tUBhvWytASSYYGWWf1Eg5SlT8/jsFRwa6ClpgzfI78AWHlA1/bAFa5VDzzC/KkgOVz4Q xcGaPgvYVCoSM1ikZqClHyL+FoGJZtpVrI1PMrvMKSECvpdU/iwMy2rBgdGkDj08fzqJ 6RlMuQtdYNeNeZh4WmmSpTFwi4QZXGkewPac5h3ilKYpJIi8sIluWboib2z+WK/Nq1Da 3g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3640hsu1xk-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 18 Jan 2021 01:37:09 -0800 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 18 Jan 2021 01:37:07 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 18 Jan 2021 01:37:07 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 18 Jan 2021 01:37:07 -0800 Received: from localhost.localdomain (unknown [10.111.145.157]) by maili.marvell.com (Postfix) with ESMTP id DB4443F7040; Mon, 18 Jan 2021 01:37:06 -0800 (PST) From: Nalla Pradeep To: Nalla Pradeep , Radha Mohan Chintakuntla , Veerasenareddy Burru CC: , , Date: Mon, 18 Jan 2021 09:36:02 +0000 Message-ID: <20210118093602.5449-11-pnalla@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210118093602.5449-1-pnalla@marvell.com> References: <20210118093602.5449-1-pnalla@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.343, 18.0.737 definitions=2021-01-18_07:2021-01-15, 2021-01-18 signatures=0 Subject: [dpdk-dev] [PATCH v2 11/11] net/octeontx_ep: Transmit data path function added X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 1. Packet transmit function for both otx and otx2 are added. 2. Flushing transmit(command) queue when pending commands are more than maximum allowed value (currently 16). 3. Scatter gather support if the packet spans multiple buffers. Signed-off-by: Nalla Pradeep --- drivers/net/octeontx_ep/otx2_ep_vf.h | 19 + drivers/net/octeontx_ep/otx_ep_common.h | 51 +++ drivers/net/octeontx_ep/otx_ep_ethdev.c | 5 + drivers/net/octeontx_ep/otx_ep_rxtx.c | 448 +++++++++++++++++++++++- drivers/net/octeontx_ep/otx_ep_rxtx.h | 26 ++ drivers/net/octeontx_ep/otx_ep_vf.h | 68 ++++ 6 files changed, 615 insertions(+), 2 deletions(-) diff --git a/drivers/net/octeontx_ep/otx2_ep_vf.h b/drivers/net/octeontx_ep/otx2_ep_vf.h index 0d7c6e8e17..9f37ab8728 100644 --- a/drivers/net/octeontx_ep/otx2_ep_vf.h +++ b/drivers/net/octeontx_ep/otx2_ep_vf.h @@ -7,5 +7,24 @@ int otx2_ep_vf_setup_device(struct otx_ep_device *sdpvf); +struct otx2_ep_instr_64B { + /* Pointer where the input data is available. */ + uint64_t dptr; + + /* OTX_EP Instruction Header. */ + union otx_ep_instr_ih ih; + + /** Pointer where the response for a RAW mode packet + * will be written by OCTEON TX. + */ + uint64_t rptr; + + /* Input Request Header. */ + union otx_ep_instr_irh irh; + + /* Additional headers available in a 64-byte instruction. */ + uint64_t exhdr[4]; +}; + #endif /*_OTX2_EP_VF_H_ */ diff --git a/drivers/net/octeontx_ep/otx_ep_common.h b/drivers/net/octeontx_ep/otx_ep_common.h index 22598f9969..1618f8225c 100644 --- a/drivers/net/octeontx_ep/otx_ep_common.h +++ b/drivers/net/octeontx_ep/otx_ep_common.h @@ -4,6 +4,10 @@ #ifndef _OTX_EP_COMMON_H_ #define _OTX_EP_COMMON_H_ + +#define OTX_EP_NW_PKT_OP 0x1220 +#define OTX_EP_NW_CMD_OP 0x1221 + #define OTX_EP_MAX_RINGS_PER_VF (8) #define OTX_EP_CFG_IO_QUEUES OTX_EP_MAX_RINGS_PER_VF #define OTX_EP_64BYTE_INSTR (64) @@ -16,9 +20,24 @@ #define OTX_EP_OQ_INFOPTR_MODE (0) #define OTX_EP_OQ_REFIL_THRESHOLD (16) + +/* IQ instruction req types */ +#define OTX_EP_REQTYPE_NONE (0) +#define OTX_EP_REQTYPE_NORESP_INSTR (1) +#define OTX_EP_REQTYPE_NORESP_NET_DIRECT (2) +#define OTX_EP_REQTYPE_NORESP_NET OTX_EP_REQTYPE_NORESP_NET_DIRECT +#define OTX_EP_REQTYPE_NORESP_GATHER (3) +#define OTX_EP_NORESP_OHSM_SEND (4) +#define OTX_EP_NORESP_LAST (4) #define OTX_EP_PCI_RING_ALIGN 65536 #define SDP_PKIND 40 #define SDP_OTX2_PKIND 57 + +#define ORDERED_TAG 0 +#define ATOMIC_TAG 1 +#define NULL_TAG 2 +#define NULL_NULL_TAG 3 + #define OTX_EP_BUSY_LOOP_COUNT (10000) #define OTX_EP_MAX_IOQS_PER_VF 8 @@ -447,7 +466,39 @@ int otx_ep_setup_oqs(struct otx_ep_device *otx_ep, int oq_no, int num_descs, unsigned int socket_id); int otx_ep_delete_oqs(struct otx_ep_device *otx_ep, uint32_t oq_no); +struct otx_ep_sg_entry { + /** The first 64 bit gives the size of data in each dptr. */ + union { + uint16_t size[4]; + uint64_t size64; + } u; + + /** The 4 dptr pointers for this entry. */ + uint64_t ptr[4]; +}; + +#define OTX_EP_SG_ENTRY_SIZE (sizeof(struct otx_ep_sg_entry)) + +/** Structure of a node in list of gather components maintained by + * driver for each network device. + */ +struct otx_ep_gather { + /** number of gather entries. */ + int num_sg; + + /** Gather component that can accommodate max sized fragment list + * received from the IP layer. + */ + struct otx_ep_sg_entry *sg; +}; + +struct otx_ep_buf_free_info { + struct rte_mbuf *mbuf; + struct otx_ep_gather g; +}; + #define OTX_EP_MAX_PKT_SZ 64000U #define OTX_EP_MAX_MAC_ADDRS 1 +#define OTX_EP_SG_ALIGN 8 #endif /* _OTX_EP_COMMON_H_ */ diff --git a/drivers/net/octeontx_ep/otx_ep_ethdev.c b/drivers/net/octeontx_ep/otx_ep_ethdev.c index 23899bc129..a0296db1f6 100644 --- a/drivers/net/octeontx_ep/otx_ep_ethdev.c +++ b/drivers/net/octeontx_ep/otx_ep_ethdev.c @@ -152,6 +152,11 @@ otx_epdev_init(struct otx_ep_device *otx_epvf) } otx_epvf->eth_dev->rx_pkt_burst = &otx_ep_recv_pkts; + if (otx_epvf->chip_id == PCI_DEVID_OCTEONTX_EP_VF) + otx_epvf->eth_dev->tx_pkt_burst = &otx_ep_xmit_pkts; + else if (otx_epvf->chip_id == PCI_DEVID_OCTEONTX2_EP_NET_VF || + otx_epvf->chip_id == PCI_DEVID_CN98XX_EP_NET_VF) + otx_epvf->eth_dev->tx_pkt_burst = &otx2_ep_xmit_pkts; ethdev_queues = (uint32_t)(otx_epvf->sriov_info.rings_per_vf); otx_epvf->max_rx_queues = ethdev_queues; otx_epvf->max_tx_queues = ethdev_queues; diff --git a/drivers/net/octeontx_ep/otx_ep_rxtx.c b/drivers/net/octeontx_ep/otx_ep_rxtx.c index 6672455507..d7bb09b89b 100644 --- a/drivers/net/octeontx_ep/otx_ep_rxtx.c +++ b/drivers/net/octeontx_ep/otx_ep_rxtx.c @@ -130,8 +130,6 @@ otx_ep_init_instr_queue(struct otx_ep_device *otx_ep, int iq_no, int num_descs, iq->flush_index = 0; iq->instr_pending = 0; - - otx_ep->io_qmask.iq |= (1ull << iq_no); /* Set 32B/64B mode for each input queue */ @@ -363,6 +361,452 @@ otx_ep_setup_oqs(struct otx_ep_device *otx_ep, int oq_no, int num_descs, return -ENOMEM; } +static inline void +otx_ep_iqreq_delete(struct otx_ep_instr_queue *iq, uint32_t idx) +{ + uint32_t reqtype; + void *buf; + struct otx_ep_buf_free_info *finfo; + + buf = iq->req_list[idx].buf; + reqtype = iq->req_list[idx].reqtype; + + switch (reqtype) { + case OTX_EP_REQTYPE_NORESP_NET: + rte_pktmbuf_free((struct rte_mbuf *)buf); + otx_ep_dbg("IQ buffer freed at idx[%d]\n", idx); + break; + + case OTX_EP_REQTYPE_NORESP_GATHER: + finfo = (struct otx_ep_buf_free_info *)buf; + /* This will take care of multiple segments also */ + rte_pktmbuf_free(finfo->mbuf); + rte_free(finfo->g.sg); + rte_free(finfo); + break; + + case OTX_EP_REQTYPE_NONE: + default: + otx_ep_info("This iqreq mode is not supported:%d\n", reqtype); + } + + /* Reset the request list at this index */ + iq->req_list[idx].buf = NULL; + iq->req_list[idx].reqtype = 0; +} + +static inline void +otx_ep_iqreq_add(struct otx_ep_instr_queue *iq, void *buf, + uint32_t reqtype, int index) +{ + iq->req_list[index].buf = buf; + iq->req_list[index].reqtype = reqtype; + + /*otx_ep_dbg("IQ buffer added at idx[%d]\n", iq->host_write_index);*/ +} + +static uint32_t +otx_vf_update_read_index(struct otx_ep_instr_queue *iq) +{ + uint32_t new_idx = rte_read32(iq->inst_cnt_reg); + if (unlikely(new_idx == 0xFFFFFFFFU)) { + /*otx2_sdp_dbg("%s Going to reset IQ index\n", __func__);*/ + rte_write32(new_idx, iq->inst_cnt_reg); + } + /* Modulo of the new index with the IQ size will give us + * the new index. + */ + new_idx &= (iq->nb_desc - 1); + + return new_idx; +} + +static void +otx_ep_flush_iq(struct otx_ep_instr_queue *iq) +{ + uint32_t instr_processed = 0; + + iq->otx_read_index = otx_vf_update_read_index(iq); + while (iq->flush_index != iq->otx_read_index) { + /* Free the IQ data buffer to the pool */ + otx_ep_iqreq_delete(iq, iq->flush_index); + iq->flush_index = + otx_ep_incr_index(iq->flush_index, 1, iq->nb_desc); + + instr_processed++; + } + + iq->stats.instr_processed = instr_processed; + iq->instr_pending -= instr_processed; +} + +static inline void +otx_ep_ring_doorbell(struct otx_ep_device *otx_ep __rte_unused, + struct otx_ep_instr_queue *iq) +{ + rte_wmb(); + rte_write64(iq->fill_cnt, iq->doorbell_reg); + iq->fill_cnt = 0; +} + +static inline int +post_iqcmd(struct otx_ep_instr_queue *iq, uint8_t *iqcmd) +{ + uint8_t *iqptr, cmdsize; + + /* This ensures that the read index does not wrap around to + * the same position if queue gets full before OCTEON TX2 could + * fetch any instr. + */ + if (iq->instr_pending > (iq->nb_desc - 1)) + return OTX_EP_IQ_SEND_FAILED; + + /* Copy cmd into iq */ + cmdsize = 64; + iqptr = iq->base_addr + (iq->host_write_index << 6); + + rte_memcpy(iqptr, iqcmd, cmdsize); + + /* Increment the host write index */ + iq->host_write_index = + otx_ep_incr_index(iq->host_write_index, 1, iq->nb_desc); + + iq->fill_cnt++; + + /* Flush the command into memory. We need to be sure the data + * is in memory before indicating that the instruction is + * pending. + */ + iq->instr_pending++; + /* OTX_EP_IQ_SEND_SUCCESS */ + return 0; +} + + +static int +otx_ep_send_data(struct otx_ep_device *otx_ep, struct otx_ep_instr_queue *iq, + void *cmd, int dbell) +{ + uint32_t ret; + + /* Submit IQ command */ + ret = post_iqcmd(iq, cmd); + + if (ret == OTX_EP_IQ_SEND_SUCCESS) { + if (dbell) + otx_ep_ring_doorbell(otx_ep, iq); + iq->stats.instr_posted++; + + } else { + iq->stats.instr_dropped++; + if (iq->fill_cnt) + otx_ep_ring_doorbell(otx_ep, iq); + } + return ret; +} + +static inline void +set_sg_size(struct otx_ep_sg_entry *sg_entry, uint16_t size, uint32_t pos) +{ +#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN + sg_entry->u.size[pos] = size; +#elif RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + sg_entry->u.size[3 - pos] = size; +#endif +} + +/* Enqueue requests/packets to OTX_EP IQ queue. + * returns number of requests enqueued successfully + */ +uint16_t +otx_ep_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts) +{ + struct otx_ep_instr_64B iqcmd; + struct otx_ep_instr_queue *iq; + struct otx_ep_device *otx_ep; + struct rte_mbuf *m; + + uint32_t iqreq_type, sgbuf_sz; + int dbell, index, count = 0; + unsigned int pkt_len, i; + int gather, gsz; + void *iqreq_buf; + uint64_t dptr; + + iq = (struct otx_ep_instr_queue *)tx_queue; + otx_ep = iq->otx_ep_dev; + + /* if (!otx_ep->started || !otx_ep->linkup) { + * goto xmit_fail; + * } + */ + + iqcmd.ih.u64 = 0; + iqcmd.pki_ih3.u64 = 0; + iqcmd.irh.u64 = 0; + + /* ih invars */ + iqcmd.ih.s.fsz = OTX_EP_FSZ; + iqcmd.ih.s.pkind = otx_ep->pkind; /* The SDK decided PKIND value */ + + /* pki ih3 invars */ + iqcmd.pki_ih3.s.w = 1; + iqcmd.pki_ih3.s.utt = 1; + iqcmd.pki_ih3.s.tagtype = ORDERED_TAG; + /* sl will be sizeof(pki_ih3) */ + iqcmd.pki_ih3.s.sl = OTX_EP_FSZ + OTX_CUST_DATA_LEN; + + /* irh invars */ + iqcmd.irh.s.opcode = OTX_EP_NW_PKT_OP; + + for (i = 0; i < nb_pkts; i++) { + m = pkts[i]; + if (m->nb_segs == 1) { + /* dptr */ + dptr = rte_mbuf_data_iova(m); + pkt_len = rte_pktmbuf_data_len(m); + iqreq_buf = m; + iqreq_type = OTX_EP_REQTYPE_NORESP_NET; + gather = 0; + gsz = 0; + } else { + struct otx_ep_buf_free_info *finfo; + int j, frags, num_sg; + + if (!(otx_ep->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS)) + goto xmit_fail; + + finfo = (struct otx_ep_buf_free_info *)rte_malloc(NULL, + sizeof(*finfo), 0); + if (finfo == NULL) { + otx_ep_err("free buffer alloc failed\n"); + goto xmit_fail; + } + num_sg = (m->nb_segs + 3) / 4; + sgbuf_sz = sizeof(struct otx_ep_sg_entry) * num_sg; + finfo->g.sg = + rte_zmalloc(NULL, sgbuf_sz, OTX_EP_SG_ALIGN); + if (finfo->g.sg == NULL) { + rte_free(finfo); + otx_ep_err("sg entry alloc failed\n"); + goto xmit_fail; + } + gather = 1; + gsz = m->nb_segs; + finfo->g.num_sg = num_sg; + finfo->g.sg[0].ptr[0] = rte_mbuf_data_iova(m); + set_sg_size(&finfo->g.sg[0], m->data_len, 0); + pkt_len = m->data_len; + finfo->mbuf = m; + + frags = m->nb_segs - 1; + j = 1; + m = m->next; + while (frags--) { + finfo->g.sg[(j >> 2)].ptr[(j & 3)] = + rte_mbuf_data_iova(m); + set_sg_size(&finfo->g.sg[(j >> 2)], + m->data_len, (j & 3)); + pkt_len += m->data_len; + j++; + m = m->next; + } + dptr = rte_mem_virt2iova(finfo->g.sg); + iqreq_buf = finfo; + iqreq_type = OTX_EP_REQTYPE_NORESP_GATHER; + if (pkt_len > OTX_EP_MAX_PKT_SZ) { + rte_free(finfo->g.sg); + rte_free(finfo); + otx_ep_err("failed\n"); + goto xmit_fail; + } + } + /* ih vars */ + iqcmd.ih.s.tlen = pkt_len + iqcmd.ih.s.fsz; + iqcmd.ih.s.gather = gather; + iqcmd.ih.s.gsz = gsz; + /* PKI_IH3 vars */ + /* irh vars */ + /* irh.rlenssz = ; */ + + iqcmd.dptr = dptr; + /* Swap FSZ(front data) here, to avoid swapping on + * OCTEON TX side rprt is not used so not swapping + */ + /* otx_ep_swap_8B_data(&iqcmd.rptr, 1); */ + otx_ep_swap_8B_data(&iqcmd.irh.u64, 1); + +#ifdef OTX_EP_IO_DEBUG + otx_ep_dbg("After swapping\n"); + otx_ep_dbg("Word0 [dptr]: 0x%016lx\n", + (unsigned long)iqcmd.dptr); + otx_ep_dbg("Word1 [ihtx]: 0x%016lx\n", (unsigned long)iqcmd.ih); + otx_ep_dbg("Word2 [pki_ih3]: 0x%016lx\n", + (unsigned long)iqcmd.pki_ih3); + otx_ep_dbg("Word3 [rptr]: 0x%016lx\n", + (unsigned long)iqcmd.rptr); + otx_ep_dbg("Word4 [irh]: 0x%016lx\n", (unsigned long)iqcmd.irh); + otx_ep_dbg("Word5 [exhdr[0]]: 0x%016lx\n", + (unsigned long)iqcmd.exhdr[0]); + rte_pktmbuf_dump(stdout, m, rte_pktmbuf_pkt_len(m)); +#endif + dbell = (i == (unsigned int)(nb_pkts - 1)) ? 1 : 0; + index = iq->host_write_index; + if (otx_ep_send_data(otx_ep, iq, &iqcmd, dbell)) + goto xmit_fail; + otx_ep_iqreq_add(iq, iqreq_buf, iqreq_type, index); + iq->stats.tx_pkts++; + iq->stats.tx_bytes += pkt_len; + count++; + } + +xmit_fail: + if (iq->instr_pending >= OTX_EP_MAX_INSTR) + otx_ep_flush_iq(iq); + + /* Return no# of instructions posted successfully. */ + return count; +} + +/* Enqueue requests/packets to OTX_EP IQ queue. + * returns number of requests enqueued successfully + */ +uint16_t +otx2_ep_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts) +{ + struct otx2_ep_instr_64B iqcmd2; + struct otx_ep_instr_queue *iq; + struct otx_ep_device *otx_ep; + uint64_t dptr; + int count = 0; + unsigned int i; + struct rte_mbuf *m; + unsigned int pkt_len; + void *iqreq_buf; + uint32_t iqreq_type, sgbuf_sz; + int gather, gsz; + int dbell; + int index; + + iq = (struct otx_ep_instr_queue *)tx_queue; + otx_ep = iq->otx_ep_dev; + + iqcmd2.ih.u64 = 0; + iqcmd2.irh.u64 = 0; + + /* ih invars */ + iqcmd2.ih.s.fsz = OTX2_EP_FSZ; + iqcmd2.ih.s.pkind = otx_ep->pkind; /* The SDK decided PKIND value */ + /* irh invars */ + iqcmd2.irh.s.opcode = OTX_EP_NW_PKT_OP; + + for (i = 0; i < nb_pkts; i++) { + m = pkts[i]; + if (m->nb_segs == 1) { + /* dptr */ + dptr = rte_mbuf_data_iova(m); + pkt_len = rte_pktmbuf_data_len(m); + iqreq_buf = m; + iqreq_type = OTX_EP_REQTYPE_NORESP_NET; + gather = 0; + gsz = 0; + } else { + struct otx_ep_buf_free_info *finfo; + int j, frags, num_sg; + + if (!(otx_ep->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS)) + goto xmit_fail; + + finfo = (struct otx_ep_buf_free_info *) + rte_malloc(NULL, sizeof(*finfo), 0); + if (finfo == NULL) { + otx_ep_err("free buffer alloc failed\n"); + goto xmit_fail; + } + num_sg = (m->nb_segs + 3) / 4; + sgbuf_sz = sizeof(struct otx_ep_sg_entry) * num_sg; + finfo->g.sg = + rte_zmalloc(NULL, sgbuf_sz, OTX_EP_SG_ALIGN); + if (finfo->g.sg == NULL) { + rte_free(finfo); + otx_ep_err("sg entry alloc failed\n"); + goto xmit_fail; + } + gather = 1; + gsz = m->nb_segs; + finfo->g.num_sg = num_sg; + finfo->g.sg[0].ptr[0] = rte_mbuf_data_iova(m); + set_sg_size(&finfo->g.sg[0], m->data_len, 0); + pkt_len = m->data_len; + finfo->mbuf = m; + + frags = m->nb_segs - 1; + j = 1; + m = m->next; + while (frags--) { + finfo->g.sg[(j >> 2)].ptr[(j & 3)] = + rte_mbuf_data_iova(m); + set_sg_size(&finfo->g.sg[(j >> 2)], + m->data_len, (j & 3)); + pkt_len += m->data_len; + j++; + m = m->next; + } + dptr = rte_mem_virt2iova(finfo->g.sg); + iqreq_buf = finfo; + iqreq_type = OTX_EP_REQTYPE_NORESP_GATHER; + if (pkt_len > OTX_EP_MAX_PKT_SZ) { + rte_free(finfo->g.sg); + rte_free(finfo); + otx_ep_err("failed\n"); + goto xmit_fail; + } + } + /* ih vars */ + iqcmd2.ih.s.tlen = pkt_len + iqcmd2.ih.s.fsz; + iqcmd2.ih.s.gather = gather; + iqcmd2.ih.s.gsz = gsz; + /* irh vars */ + /* irh.rlenssz = ; */ + iqcmd2.dptr = dptr; + /* Swap FSZ(front data) here, to avoid swapping on + * OCTEON TX side rptr is not used so not swapping. + */ + /* otx_ep_swap_8B_data(&iqcmd2.rptr, 1); */ + otx_ep_swap_8B_data(&iqcmd2.irh.u64, 1); + +#ifdef OTX_EP_IO_DEBUG + otx_ep_dbg("After swapping\n"); + otx_ep_dbg("Word0 [dptr]: 0x%016lx\n", + (unsigned long)iqcmd.dptr); + otx_ep_dbg("Word1 [ihtx]: 0x%016lx\n", (unsigned long)iqcmd.ih); + otx_ep_dbg("Word2 [pki_ih3]: 0x%016lx\n", + (unsigned long)iqcmd.pki_ih3); + otx_ep_dbg("Word3 [rptr]: 0x%016lx\n", + (unsigned long)iqcmd.rptr); + otx_ep_dbg("Word4 [irh]: 0x%016lx\n", (unsigned long)iqcmd.irh); + otx_ep_dbg("Word5 [exhdr[0]]: 0x%016lx\n", + (unsigned long)iqcmd.exhdr[0]); +#endif + /* rte_pktmbuf_dump(stdout, m, rte_pktmbuf_pkt_len(m)); */ + index = iq->host_write_index; + dbell = (i == (unsigned int)(nb_pkts - 1)) ? 1 : 0; + if (otx_ep_send_data(otx_ep, iq, &iqcmd2, dbell)) + goto xmit_fail; + otx_ep_iqreq_add(iq, iqreq_buf, iqreq_type, index); + iq->stats.tx_pkts++; + iq->stats.tx_bytes += pkt_len; + count++; + } + +xmit_fail: + if (iq->instr_pending >= OTX_EP_MAX_INSTR) + otx_ep_flush_iq(iq); + + /* Return no# of instructions posted successfully. */ + return count; +} + static uint32_t otx_ep_droq_refill(struct otx_ep_droq *droq) { diff --git a/drivers/net/octeontx_ep/otx_ep_rxtx.h b/drivers/net/octeontx_ep/otx_ep_rxtx.h index 669446fa4a..09172ecd68 100644 --- a/drivers/net/octeontx_ep/otx_ep_rxtx.h +++ b/drivers/net/octeontx_ep/otx_ep_rxtx.h @@ -5,15 +5,41 @@ #ifndef _OTX_EP_RXTX_H_ #define _OTX_EP_RXTX_H_ +#include + #define OTX_EP_RXD_ALIGN 1 #define OTX_EP_TXD_ALIGN 1 + +#define OTX_EP_IQ_SEND_FAILED (-1) +#define OTX_EP_IQ_SEND_SUCCESS (0) + #define OTX_EP_MAX_DELAYED_PKT_RETRIES 10000 + +#define OTX_EP_FSZ 28 +#define OTX2_EP_FSZ 24 +#define OTX_EP_MAX_INSTR 16 + +static inline void +otx_ep_swap_8B_data(uint64_t *data, uint32_t blocks) +{ + /* Swap 8B blocks */ + while (blocks) { + *data = rte_bswap64(*data); + blocks--; + data++; + } +} + static inline uint32_t otx_ep_incr_index(uint32_t index, uint32_t count, uint32_t max) { return ((index + count) & (max - 1)); } uint16_t +otx_ep_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts); +uint16_t +otx2_ep_xmit_pkts(void *tx_queue, struct rte_mbuf **pkts, uint16_t nb_pkts); +uint16_t otx_ep_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t budget); diff --git a/drivers/net/octeontx_ep/otx_ep_vf.h b/drivers/net/octeontx_ep/otx_ep_vf.h index fa08141b2b..28976bb7f4 100644 --- a/drivers/net/octeontx_ep/otx_ep_vf.h +++ b/drivers/net/octeontx_ep/otx_ep_vf.h @@ -100,6 +100,74 @@ */ #define SDP_GBL_WMARK 0x100 + +/* Optional PKI Instruction Header(PKI IH) */ +typedef union { + uint64_t u64; + struct { + /** Tag Value */ + uint64_t tag:32; + + /** QPG Value */ + uint64_t qpg:11; + + /** Reserved1 */ + uint64_t reserved1:2; + + /** Tag type */ + uint64_t tagtype:2; + + /** Use Tag Type */ + uint64_t utt:1; + + /** Skip Length */ + uint64_t sl:8; + + /** Parse Mode */ + uint64_t pm:3; + + /** Reserved2 */ + uint64_t reserved2:1; + + /** Use QPG */ + uint64_t uqpg:1; + + /** Use Tag */ + uint64_t utag:1; + + /** Raw mode indicator 1 = RAW */ + uint64_t raw:1; + + /** Wider bit */ + uint64_t w:1; + } s; +} otx_ep_instr_pki_ih3_t; + + +/* OTX_EP 64B instruction format */ +struct otx_ep_instr_64B { + /* Pointer where the input data is available. */ + uint64_t dptr; + + /* OTX_EP Instruction Header. */ + union otx_ep_instr_ih ih; + + /* PKI Optional Instruction Header. */ + otx_ep_instr_pki_ih3_t pki_ih3; + + /** Pointer where the response for a RAW mode packet + * will be written by OCTEON TX. + */ + uint64_t rptr; + + /* Input Request Header. */ + union otx_ep_instr_irh irh; + + /* Additional headers available in a 64-byte instruction. */ + uint64_t exhdr[3]; +}; +#define OTX_EP_64B_INSTR_SIZE (sizeof(otx_ep_instr_64B)) + int otx_ep_vf_setup_device(struct otx_ep_device *otx_ep); #endif /*_OTX_EP_VF_H_ */ -- 2.17.1