From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B500DA09E4; Fri, 29 Jan 2021 13:46:08 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 362EF240209; Fri, 29 Jan 2021 13:45:27 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 3F62D2401E8 for ; Fri, 29 Jan 2021 13:45:18 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 10TCZ3Pw023534 for ; Fri, 29 Jan 2021 04:45:17 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=mugCMA/Xr4eYhmYB2xlB8570FzdrxQWuq5j74WbIcWg=; b=HFSRUgkpj6G+sW866aUbX44GLDARgU95XM1Jan2Rcum3uF5QB4YK7Rs+LnxRbdDb4eQk +FJwSh6tOus8P8rEAPrFImFNBCsJ/Wh/ToD35Q2buDQ/qWjcLk1j+hP2hOJDAywdaID1 HeHUInwzMeH7pARqMoIGJBnn53tBEfMp/Ggrtzje8A999FYLjTYO3LpHBz3r3RJk3Xql uGQWzZjvj0CrWgMX5a0DxxcclkZI6PYNE7pKw4dIqNKqLW7f7MC5VPPdDs8ZQd7XMQg/ ISDTQw/hyrterJmfTyxNqHhDjDAopwi/2CPky7hBuApnOKwl/PiyBMOs338J+36b4CHL Dw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 36b1xpqw4v-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 29 Jan 2021 04:45:17 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 29 Jan 2021 04:45:15 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 29 Jan 2021 04:45:15 -0800 Received: from sburla-PowerEdge-T630.caveonetworks.com (unknown [10.106.27.217]) by maili.marvell.com (Postfix) with ESMTP id 6F0D43F704B; Fri, 29 Jan 2021 04:45:15 -0800 (PST) From: Nalla Pradeep To: Nalla Pradeep , Radha Mohan Chintakuntla , Veerasenareddy Burru CC: , , Date: Fri, 29 Jan 2021 04:45:06 -0800 Message-ID: <20210129124510.12158-9-pnalla@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210129001640.1251-1-pnalla@marvell.com> References: <20210129001640.1251-1-pnalla@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.737 definitions=2021-01-29_05:2021-01-28, 2021-01-29 signatures=0 Subject: [dpdk-dev] [PATCH v7 08/12] net/octeontx_ep: added tx queue setup and release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Transmit queue setup involves allocating memory for the command queue considering tx descriptor count and initializing data structure representing the queue. Transmit queue release function frees the command queue. Signed-off-by: Nalla Pradeep --- drivers/net/octeontx_ep/otx_ep_common.h | 88 +++++++++++++++- drivers/net/octeontx_ep/otx_ep_ethdev.c | 81 +++++++++++++++ drivers/net/octeontx_ep/otx_ep_rxtx.c | 133 ++++++++++++++++++++++++ 3 files changed, 301 insertions(+), 1 deletion(-) diff --git a/drivers/net/octeontx_ep/otx_ep_common.h b/drivers/net/octeontx_ep/otx_ep_common.h index b31a6c675..493dbf8e1 100644 --- a/drivers/net/octeontx_ep/otx_ep_common.h +++ b/drivers/net/octeontx_ep/otx_ep_common.h @@ -45,7 +45,21 @@ rte_write64(val, ((base_addr) + off)); \ } -struct otx_ep_device; +/* OTX_EP IQ request list */ +struct otx_ep_instr_list { + void *buf; + uint32_t reqtype; +}; +#define OTX_EP_IQREQ_LIST_SIZE (sizeof(struct otx_ep_instr_list)) + +/* Input Queue statistics. Each input queue has four stats fields. */ +struct otx_ep_iq_stats { + uint64_t instr_posted; /* Instructions posted to this queue. */ + uint64_t instr_processed; /* Instructions processed in this queue. */ + uint64_t instr_dropped; /* Instructions that could not be processed */ + uint64_t tx_pkts; + uint64_t tx_bytes; +}; /* Structure to define the configuration attributes for each Input queue. */ struct otx_ep_iq_config { @@ -59,6 +73,66 @@ struct otx_ep_iq_config { uint32_t pending_list_size; }; +/** The instruction (input) queue. + * The input queue is used to post raw (instruction) mode data or packet data + * to OCTEON TX2 device from the host. Each IQ of a OTX_EP EP VF device has one + * such structure to represent it. + */ +struct otx_ep_instr_queue { + struct otx_ep_device *otx_ep_dev; + + uint32_t q_no; + uint32_t pkt_in_done; + + /* Flag for 64 byte commands. */ + uint32_t iqcmd_64B:1; + uint32_t rsvd:17; + uint32_t status:8; + + /* Number of descriptors in this ring. */ + uint32_t nb_desc; + + /* Input ring index, where the driver should write the next packet */ + uint32_t host_write_index; + + /* Input ring index, where the OCTEON TX2 should read the next packet */ + uint32_t otx_read_index; + + uint32_t reset_instr_cnt; + + /** This index aids in finding the window in the queue where OCTEON TX2 + * has read the commands. + */ + uint32_t flush_index; + + /* This keeps track of the instructions pending in this queue. */ + uint64_t instr_pending; + + /* Pointer to the Virtual Base addr of the input ring. */ + uint8_t *base_addr; + + /* This IQ request list */ + struct otx_ep_instr_list *req_list; + + /* OTX_EP doorbell register for the ring. */ + void *doorbell_reg; + + /* OTX_EP instruction count register for this ring. */ + void *inst_cnt_reg; + + /* Number of instructions pending to be posted to OCTEON TX2. */ + uint32_t fill_cnt; + + /* Statistics for this input queue. */ + struct otx_ep_iq_stats stats; + + /* DMA mapped base address of the input descriptor ring. */ + uint64_t base_addr_dma; + + /* Memory zone */ + const struct rte_memzone *iq_mz; +}; + /** Descriptor format. * The descriptor ring is made of descriptors which have 2 64-bit values: * -# Physical (bus) address of the data buffer. @@ -229,6 +303,8 @@ struct otx_ep_sriov_info { /* Required functions for each VF device */ struct otx_ep_fn_list { + void (*setup_iq_regs)(struct otx_ep_device *otx_ep, uint32_t q_no); + void (*setup_oq_regs)(struct otx_ep_device *otx_ep, uint32_t q_no); void (*setup_device_regs)(struct otx_ep_device *otx_ep); @@ -258,6 +334,12 @@ struct otx_ep_device { uint32_t max_rx_queues; + /* Num IQs */ + uint32_t nb_tx_queues; + + /* The input instruction queues */ + struct otx_ep_instr_queue *instr_queue[OTX_EP_MAX_IOQS_PER_VF]; + /* Num OQs */ uint32_t nb_rx_queues; @@ -278,6 +360,10 @@ struct otx_ep_device { uint64_t tx_offloads; }; +int otx_ep_setup_iqs(struct otx_ep_device *otx_ep, uint32_t iq_no, + int num_descs, unsigned int socket_id); +int otx_ep_delete_iqs(struct otx_ep_device *otx_ep, uint32_t iq_no); + int otx_ep_setup_oqs(struct otx_ep_device *otx_ep, int oq_no, int num_descs, int desc_size, struct rte_mempool *mpool, unsigned int socket_id); diff --git a/drivers/net/octeontx_ep/otx_ep_ethdev.c b/drivers/net/octeontx_ep/otx_ep_ethdev.c index da36c5953..58bf6e6ae 100644 --- a/drivers/net/octeontx_ep/otx_ep_ethdev.c +++ b/drivers/net/octeontx_ep/otx_ep_ethdev.c @@ -219,11 +219,83 @@ otx_ep_rx_queue_release(void *rxq) otx_ep_err("Failed to delete OQ:%d\n", q_id); } +/** + * Allocate and initialize SW ring. Initialize associated HW registers. + * + * @param eth_dev + * Pointer to structure rte_eth_dev + * + * @param q_no + * Queue number + * + * @param num_tx_descs + * Number of ringbuffer descriptors + * + * @param socket_id + * NUMA socket id, used for memory allocations + * + * @param tx_conf + * Pointer to the structure rte_eth_txconf + * + * @return + * - On success, return 0 + * - On failure, return -errno value + */ +static int +otx_ep_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t q_no, + uint16_t num_tx_descs, unsigned int socket_id, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct otx_ep_device *otx_epvf = OTX_EP_DEV(eth_dev); + int retval; + + if (q_no >= otx_epvf->max_tx_queues) { + otx_ep_err("Invalid tx queue number %u\n", q_no); + return -EINVAL; + } + if (num_tx_descs & (num_tx_descs - 1)) { + otx_ep_err("Invalid tx desc number should be pow 2 %u\n", + num_tx_descs); + return -EINVAL; + } + + retval = otx_ep_setup_iqs(otx_epvf, q_no, num_tx_descs, socket_id); + + if (retval) { + otx_ep_err("IQ(TxQ) creation failed.\n"); + return retval; + } + + eth_dev->data->tx_queues[q_no] = otx_epvf->instr_queue[q_no]; + otx_ep_dbg("tx queue[%d] setup\n", q_no); + return 0; +} + +/** + * Release the transmit queue/ringbuffer. Called by + * the upper layers. + * + * @param txq + * Opaque pointer to the transmit queue to release + * + * @return + * - nothing + */ +static void +otx_ep_tx_queue_release(void *txq) +{ + struct otx_ep_instr_queue *tq = (struct otx_ep_instr_queue *)txq; + + otx_ep_delete_iqs(tq->otx_ep_dev, tq->q_no); +} + /* Define our ethernet definitions */ static const struct eth_dev_ops otx_ep_eth_dev_ops = { .dev_configure = otx_ep_dev_configure, .rx_queue_setup = otx_ep_rx_queue_setup, .rx_queue_release = otx_ep_rx_queue_release, + .tx_queue_setup = otx_ep_tx_queue_setup, + .tx_queue_release = otx_ep_tx_queue_release, .dev_infos_get = otx_ep_dev_info_get, }; @@ -246,6 +318,15 @@ otx_epdev_exit(struct rte_eth_dev *eth_dev) } otx_ep_info("Num OQs:%d freed\n", otx_epvf->nb_rx_queues); + num_queues = otx_epvf->nb_tx_queues; + for (q = 0; q < num_queues; q++) { + if (otx_ep_delete_iqs(otx_epvf, q)) { + otx_ep_err("Failed to delete IQ:%d\n", q); + return -EINVAL; + } + } + otx_ep_dbg("Num IQs:%d freed\n", otx_epvf->nb_tx_queues); + return 0; } diff --git a/drivers/net/octeontx_ep/otx_ep_rxtx.c b/drivers/net/octeontx_ep/otx_ep_rxtx.c index d411e693f..d3cf0e2a9 100644 --- a/drivers/net/octeontx_ep/otx_ep_rxtx.c +++ b/drivers/net/octeontx_ep/otx_ep_rxtx.c @@ -36,6 +36,139 @@ otx_ep_dmazone_free(const struct rte_memzone *mz) otx_ep_err("Memzone free failed : ret = %d\n", ret); } +/* Free IQ resources */ +int +otx_ep_delete_iqs(struct otx_ep_device *otx_ep, uint32_t iq_no) +{ + struct otx_ep_instr_queue *iq; + + iq = otx_ep->instr_queue[iq_no]; + if (iq == NULL) { + otx_ep_err("Invalid IQ[%d]\n", iq_no); + return -EINVAL; + } + + rte_free(iq->req_list); + iq->req_list = NULL; + + if (iq->iq_mz) { + otx_ep_dmazone_free(iq->iq_mz); + iq->iq_mz = NULL; + } + + rte_free(otx_ep->instr_queue[iq_no]); + otx_ep->instr_queue[iq_no] = NULL; + + otx_ep->nb_tx_queues--; + + otx_ep_info("IQ[%d] is deleted\n", iq_no); + + return 0; +} + +/* IQ initialization */ +static int +otx_ep_init_instr_queue(struct otx_ep_device *otx_ep, int iq_no, int num_descs, + unsigned int socket_id) +{ + const struct otx_ep_config *conf; + struct otx_ep_instr_queue *iq; + uint32_t q_size; + + conf = otx_ep->conf; + iq = otx_ep->instr_queue[iq_no]; + q_size = conf->iq.instr_type * num_descs; + + /* IQ memory creation for Instruction submission to OCTEON TX2 */ + iq->iq_mz = rte_eth_dma_zone_reserve(otx_ep->eth_dev, + "instr_queue", iq_no, q_size, + OTX_EP_PCI_RING_ALIGN, + socket_id); + if (iq->iq_mz == NULL) { + otx_ep_err("IQ[%d] memzone alloc failed\n", iq_no); + goto iq_init_fail; + } + + iq->base_addr_dma = iq->iq_mz->iova; + iq->base_addr = (uint8_t *)iq->iq_mz->addr; + + if (num_descs & (num_descs - 1)) { + otx_ep_err("IQ[%d] descs not in power of 2\n", iq_no); + goto iq_init_fail; + } + + iq->nb_desc = num_descs; + + /* Create a IQ request list to hold requests that have been + * posted to OCTEON TX2. This list will be used for freeing the IQ + * data buffer(s) later once the OCTEON TX2 fetched the requests. + */ + iq->req_list = rte_zmalloc_socket("request_list", + (iq->nb_desc * OTX_EP_IQREQ_LIST_SIZE), + RTE_CACHE_LINE_SIZE, + rte_socket_id()); + if (iq->req_list == NULL) { + otx_ep_err("IQ[%d] req_list alloc failed\n", iq_no); + goto iq_init_fail; + } + + otx_ep_info("IQ[%d]: base: %p basedma: %lx count: %d\n", + iq_no, iq->base_addr, (unsigned long)iq->base_addr_dma, + iq->nb_desc); + + iq->otx_ep_dev = otx_ep; + iq->q_no = iq_no; + iq->fill_cnt = 0; + iq->host_write_index = 0; + iq->otx_read_index = 0; + iq->flush_index = 0; + iq->instr_pending = 0; + + otx_ep->io_qmask.iq |= (1ull << iq_no); + + /* Set 32B/64B mode for each input queue */ + if (conf->iq.instr_type == 64) + otx_ep->io_qmask.iq64B |= (1ull << iq_no); + + iq->iqcmd_64B = (conf->iq.instr_type == 64); + + /* Set up IQ registers */ + otx_ep->fn_list.setup_iq_regs(otx_ep, iq_no); + + return 0; + +iq_init_fail: + return -ENOMEM; +} + +int +otx_ep_setup_iqs(struct otx_ep_device *otx_ep, uint32_t iq_no, int num_descs, + unsigned int socket_id) +{ + struct otx_ep_instr_queue *iq; + + iq = (struct otx_ep_instr_queue *)rte_zmalloc("otx_ep_IQ", sizeof(*iq), + RTE_CACHE_LINE_SIZE); + if (iq == NULL) + return -ENOMEM; + + otx_ep->instr_queue[iq_no] = iq; + + if (otx_ep_init_instr_queue(otx_ep, iq_no, num_descs, socket_id)) { + otx_ep_err("IQ init is failed\n"); + goto delete_IQ; + } + otx_ep->nb_tx_queues++; + + otx_ep_info("IQ[%d] is created.\n", iq_no); + + return 0; + +delete_IQ: + otx_ep_delete_iqs(otx_ep, iq_no); + return -ENOMEM; +} + static void otx_ep_droq_reset_indices(struct otx_ep_droq *droq) { -- 2.17.1