From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 17322A05D3 for ; Thu, 23 May 2019 10:15:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E06495689; Thu, 23 May 2019 10:15:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 033B84C99 for ; Thu, 23 May 2019 10:15:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x4N89ijs019064; Thu, 23 May 2019 01:15:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=TKsRMZ8c0UKxfYZi9c+YQXEizrmFgiozFL0X/yRCqIY=; b=nC+ou0K/AM8sZ2a5uin09v1I85AQTjjfoWLrpA4T0d0y/bmN0nhlbaLXh4DhYQ1e+vXY Vho9XLADT4fHjFe+E5sOL9hX1uKHmdwqZl3XkjeaJIEd1sCYDaIEDW8U7D0nxi0r3mDC OJ3GRYgAAadK5cet8QlttxJTL09KlA5QiYD3CIDncOD+1rzgU8VREFCAVaagta0TTrVe OnYMcvoWD89+DnoyzEZZFCbE0HCU1nli1YRRvlMC0++4FEbR8OKF4cEjUUqu464RqZ0X 6/zIiWwlQR61zQAAmGZJO6pH1IFsIBU8gyIwaZNEhWw5aM5YkrEGKZ3O20r80CpN4V3P dQ== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2smnwk0s8w-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 23 May 2019 01:15:34 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 23 May 2019 01:15:23 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Thu, 23 May 2019 01:15:23 -0700 Received: from jerin-lab.marvell.com (unknown [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id 60BD53F703F; Thu, 23 May 2019 01:15:22 -0700 (PDT) From: To: CC: , Jerin Jacob , Nithin Dabilpuram Date: Thu, 23 May 2019 13:43:19 +0530 Message-ID: <20190523081339.56348-8-jerinj@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190523081339.56348-1-jerinj@marvell.com> References: <20190523081339.56348-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-05-23_08:, , signatures=0 Subject: [dpdk-dev] [PATCH v1 07/27] common/octeontx2: introduce common device class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Introduce otx2_dev class to hold octeontx2 PCIe device specific information and operations. All PCIe drivers(ethdev, mempool, cryptodev and eventdev) in octeontx2, inherits this base object to avail the common functionalities such as mailbox creation, interrupt registration, etc of the PCIe device. Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram --- drivers/common/octeontx2/Makefile | 2 + drivers/common/octeontx2/meson.build | 4 +- drivers/common/octeontx2/otx2_common.h | 13 ++ drivers/common/octeontx2/otx2_dev.c | 197 ++++++++++++++++++ drivers/common/octeontx2/otx2_dev.h | 84 ++++++++ drivers/common/octeontx2/otx2_irq.h | 19 ++ .../rte_common_octeontx2_version.map | 3 + 7 files changed, 320 insertions(+), 2 deletions(-) create mode 100644 drivers/common/octeontx2/otx2_dev.c create mode 100644 drivers/common/octeontx2/otx2_dev.h create mode 100644 drivers/common/octeontx2/otx2_irq.h diff --git a/drivers/common/octeontx2/Makefile b/drivers/common/octeontx2/Makefile index 3fd67f0ab..a6f94553d 100644 --- a/drivers/common/octeontx2/Makefile +++ b/drivers/common/octeontx2/Makefile @@ -11,6 +11,7 @@ LIB = librte_common_octeontx2.a CFLAGS += $(WERROR_FLAGS) CFLAGS += -I$(RTE_SDK)/drivers/common/octeontx2 +CFLAGS += -I$(RTE_SDK)/drivers/bus/pci ifneq ($(CONFIG_RTE_ARCH_64),y) CFLAGS += -Wno-int-to-pointer-cast @@ -24,6 +25,7 @@ LIBABIVER := 1 # # all source are stored in SRCS-y # +SRCS-y += otx2_dev.c SRCS-y += otx2_mbox.c SRCS-y += otx2_common.c diff --git a/drivers/common/octeontx2/meson.build b/drivers/common/octeontx2/meson.build index 4771b1942..feaf75d92 100644 --- a/drivers/common/octeontx2/meson.build +++ b/drivers/common/octeontx2/meson.build @@ -2,7 +2,7 @@ # Copyright(C) 2019 Marvell International Ltd. # -sources= files( +sources= files('otx2_dev.c', 'otx2_mbox.c', 'otx2_common.c', ) @@ -19,6 +19,6 @@ foreach flag: extra_flags endif endforeach -deps = ['eal', 'ethdev'] +deps = ['eal', 'pci', 'ethdev'] includes += include_directories('../../common/octeontx2', '../../bus/pci') diff --git a/drivers/common/octeontx2/otx2_common.h b/drivers/common/octeontx2/otx2_common.h index c000db296..0750cdd14 100644 --- a/drivers/common/octeontx2/otx2_common.h +++ b/drivers/common/octeontx2/otx2_common.h @@ -67,6 +67,19 @@ extern int otx2_logtype_tim; #define otx2_tm_dbg(fmt, ...) otx2_dbg(tm, fmt, ##__VA_ARGS__) #define otx2_tim_dbg(fmt, ...) otx2_dbg(tim, fmt, ##__VA_ARGS__) +/* PCI IDs */ +#define PCI_VENDOR_ID_CAVIUM 0x177D +#define PCI_DEVID_OCTEONTX2_RVU_PF 0xA063 +#define PCI_DEVID_OCTEONTX2_RVU_VF 0xA064 +#define PCI_DEVID_OCTEONTX2_RVU_AF 0xA065 +#define PCI_DEVID_OCTEONTX2_RVU_SSO_TIM_PF 0xA0F9 +#define PCI_DEVID_OCTEONTX2_RVU_SSO_TIM_VF 0xA0FA +#define PCI_DEVID_OCTEONTX2_RVU_NPA_PF 0xA0FB +#define PCI_DEVID_OCTEONTX2_RVU_NPA_VF 0xA0FC +#define PCI_DEVID_OCTEONTX2_RVU_CPT_PF 0xA0FD +#define PCI_DEVID_OCTEONTX2_RVU_CPT_VF 0xA0FE +#define PCI_DEVID_OCTEONTX2_RVU_AF_VF 0xA0f8 + /* IO Access */ #define otx2_read64(addr) rte_read64_relaxed((void *)(addr)) #define otx2_write64(val, addr) rte_write64_relaxed((val), (void *)(addr)) diff --git a/drivers/common/octeontx2/otx2_dev.c b/drivers/common/octeontx2/otx2_dev.c new file mode 100644 index 000000000..ccdb2df78 --- /dev/null +++ b/drivers/common/octeontx2/otx2_dev.c @@ -0,0 +1,197 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#include +#include +#include +#include + +#include +#include +#include + +#include "otx2_dev.h" +#include "otx2_mbox.h" + +/* PF/VF message handling timer */ +#define VF_PF_MBOX_TIMER_MS (20 * 1000) + +static void * +mbox_mem_map(off_t off, size_t size) +{ + void *va = MAP_FAILED; + int mem_fd; + + if (size <= 0) + goto error; + + mem_fd = open("/dev/mem", O_RDWR); + if (mem_fd < 0) + goto error; + + va = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, mem_fd, off); + close(mem_fd); + + if (va == MAP_FAILED) + otx2_err("Failed to mmap sz=0x%zx, fd=%d, off=%ld", + size, mem_fd, off); +error: + return va; +} + +static void +mbox_mem_unmap(void *va, size_t size) +{ + if (va) + munmap(va, size); +} + +static void +otx2_update_pass_hwcap(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + RTE_SET_USED(pci_dev); + + /* Update this logic when we have A1 */ + dev->hwcap |= OTX2_HWCAP_F_A0; +} + +static void +otx2_update_vf_hwcap(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + dev->hwcap = 0; + + switch (pci_dev->id.device_id) { + case PCI_DEVID_OCTEONTX2_RVU_PF: + break; + case PCI_DEVID_OCTEONTX2_RVU_SSO_TIM_VF: + case PCI_DEVID_OCTEONTX2_RVU_NPA_VF: + case PCI_DEVID_OCTEONTX2_RVU_CPT_VF: + case PCI_DEVID_OCTEONTX2_RVU_AF_VF: + case PCI_DEVID_OCTEONTX2_RVU_VF: + dev->hwcap |= OTX2_HWCAP_F_VF; + break; + } +} + +/** + * @internal + * Initialize the otx2 device + */ +int +otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev) +{ + int up_direction = MBOX_DIR_PFAF_UP; + int rc, direction = MBOX_DIR_PFAF; + struct otx2_dev *dev = otx2_dev; + uintptr_t bar2, bar4; + uint64_t bar4_addr; + void *hwbase; + + bar2 = (uintptr_t)pci_dev->mem_resource[2].addr; + bar4 = (uintptr_t)pci_dev->mem_resource[4].addr; + + if (bar2 == 0 || bar4 == 0) { + otx2_err("Failed to get pci bars"); + rc = -ENODEV; + goto error; + } + + dev->node = pci_dev->device.numa_node; + dev->maxvf = pci_dev->max_vfs; + dev->bar2 = bar2; + dev->bar4 = bar4; + + otx2_update_vf_hwcap(pci_dev, dev); + otx2_update_pass_hwcap(pci_dev, dev); + + if (otx2_dev_is_vf(dev)) { + direction = MBOX_DIR_VFPF; + up_direction = MBOX_DIR_VFPF_UP; + } + + /* Initialize the local mbox */ + rc = otx2_mbox_init(&dev->mbox_local, bar4, bar2, direction, 1); + if (rc) + goto error; + dev->mbox = &dev->mbox_local; + + rc = otx2_mbox_init(&dev->mbox_up, bar4, bar2, up_direction, 1); + if (rc) + goto error; + + /* Check the readiness of PF/VF */ + rc = otx2_send_ready_msg(dev->mbox, &dev->pf_func); + if (rc) + goto mbox_fini; + + dev->pf = otx2_get_pf(dev->pf_func); + dev->vf = otx2_get_vf(dev->pf_func); + memset(&dev->active_vfs, 0, sizeof(dev->active_vfs)); + + /* Found VF devices in a PF device */ + if (pci_dev->max_vfs > 0) { + + /* Remap mbox area for all vf's */ + bar4_addr = otx2_read64(bar2 + RVU_PF_VF_BAR4_ADDR); + if (bar4_addr == 0) { + rc = -ENODEV; + goto mbox_fini; + } + + hwbase = mbox_mem_map(bar4_addr, MBOX_SIZE * pci_dev->max_vfs); + if (hwbase == MAP_FAILED) { + rc = -ENOMEM; + goto mbox_fini; + } + /* Init mbox object */ + rc = otx2_mbox_init(&dev->mbox_vfpf, (uintptr_t)hwbase, + bar2, MBOX_DIR_PFVF, pci_dev->max_vfs); + if (rc) + goto iounmap; + + /* PF -> VF UP messages */ + rc = otx2_mbox_init(&dev->mbox_vfpf_up, (uintptr_t)hwbase, + bar2, MBOX_DIR_PFVF_UP, pci_dev->max_vfs); + if (rc) + goto mbox_fini; + } + + dev->mbox_active = 1; + return rc; + +iounmap: + mbox_mem_unmap(hwbase, MBOX_SIZE * pci_dev->max_vfs); +mbox_fini: + otx2_mbox_fini(dev->mbox); + otx2_mbox_fini(&dev->mbox_up); +error: + return rc; +} + +/** + * @internal + * Finalize the otx2 device + */ +void +otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev) +{ + struct otx2_dev *dev = otx2_dev; + struct otx2_mbox *mbox; + + /* Release PF - VF */ + mbox = &dev->mbox_vfpf; + if (mbox->hwbase && mbox->dev) + mbox_mem_unmap((void *)mbox->hwbase, + MBOX_SIZE * pci_dev->max_vfs); + otx2_mbox_fini(mbox); + mbox = &dev->mbox_vfpf_up; + otx2_mbox_fini(mbox); + + /* Release PF - AF */ + mbox = dev->mbox; + otx2_mbox_fini(mbox); + mbox = &dev->mbox_up; + otx2_mbox_fini(mbox); + dev->mbox_active = 0; +} diff --git a/drivers/common/octeontx2/otx2_dev.h b/drivers/common/octeontx2/otx2_dev.h new file mode 100644 index 000000000..a89570b62 --- /dev/null +++ b/drivers/common/octeontx2/otx2_dev.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef _OTX2_DEV_H +#define _OTX2_DEV_H + +#include + +#include "otx2_common.h" +#include "otx2_irq.h" +#include "otx2_mbox.h" + +/* Common HWCAP flags. Use from LSB bits */ +#define OTX2_HWCAP_F_VF BIT_ULL(0) /* VF device */ +#define otx2_dev_is_vf(dev) (dev->hwcap & OTX2_HWCAP_F_VF) +#define otx2_dev_is_pf(dev) (!(dev->hwcap & OTX2_HWCAP_F_VF)) +#define otx2_dev_is_lbk(dev) ((dev->hwcap & OTX2_HWCAP_F_VF) && \ + (dev->tx_chan_base < 0x700)) + +#define OTX2_HWCAP_F_A0 BIT_ULL(1) /* A0 device */ +#define otx2_dev_is_A0(dev) (dev->hwcap & OTX2_HWCAP_F_A0) + +struct otx2_dev; + +#define OTX2_DEV \ + int node __rte_cache_aligned; \ + uint16_t pf; \ + int16_t vf; \ + uint16_t pf_func; \ + uint8_t mbox_active; \ + bool drv_inited; \ + uint64_t active_vfs[MAX_VFPF_DWORD_BITS]; \ + uintptr_t bar2; \ + uintptr_t bar4; \ + struct otx2_mbox mbox_local; \ + struct otx2_mbox mbox_up; \ + struct otx2_mbox mbox_vfpf; \ + struct otx2_mbox mbox_vfpf_up; \ + otx2_intr_t intr; \ + int timer_set; /* ~0 : no alarm handling */ \ + uint64_t hwcap; \ + struct otx2_mbox *mbox; \ + uint16_t maxvf; \ + const struct otx2_dev_ops *ops + +struct otx2_dev { + OTX2_DEV; +}; + +int otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev); +void otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev); +int otx2_dev_active_vfs(void *otx2_dev); + +#define RVU_PFVF_PF_SHIFT 10 +#define RVU_PFVF_PF_MASK 0x3F +#define RVU_PFVF_FUNC_SHIFT 0 +#define RVU_PFVF_FUNC_MASK 0x3FF + +static inline int +otx2_get_vf(uint16_t pf_func) +{ + return (((pf_func >> RVU_PFVF_FUNC_SHIFT) & RVU_PFVF_FUNC_MASK) - 1); +} + +static inline int +otx2_get_pf(uint16_t pf_func) +{ + return (pf_func >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK; +} + +static inline int +otx2_pfvf_func(int pf, int vf) +{ + return (pf << RVU_PFVF_PF_SHIFT) | ((vf << RVU_PFVF_FUNC_SHIFT) + 1); +} + +static inline int +otx2_is_afvf(uint16_t pf_func) +{ + return !(pf_func & ~RVU_PFVF_FUNC_MASK); +} + +#endif /* _OTX2_DEV_H */ diff --git a/drivers/common/octeontx2/otx2_irq.h b/drivers/common/octeontx2/otx2_irq.h new file mode 100644 index 000000000..df44ddfba --- /dev/null +++ b/drivers/common/octeontx2/otx2_irq.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef _OTX2_IRQ_H_ +#define _OTX2_IRQ_H_ + +#include +#include + +#include "otx2_common.h" + +typedef struct { +/* 128 devices translate to two 64 bits dwords */ +#define MAX_VFPF_DWORD_BITS 2 + uint64_t bits[MAX_VFPF_DWORD_BITS]; +} otx2_intr_t; + +#endif /* _OTX2_IRQ_H_ */ diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index 7d2cfc70e..34a6ffb93 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -1,6 +1,9 @@ DPDK_19.05 { global: + otx2_dev_fini; + otx2_dev_init; + otx2_logtype_base; otx2_logtype_mbox; otx2_logtype_npa; -- 2.21.0