From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2B61458CA; Mon, 2 Sep 2024 08:30:36 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 172D240664; Mon, 2 Sep 2024 08:30:10 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id CF7E040650 for ; Mon, 2 Sep 2024 08:30:07 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 481MxpfD032080 for ; Sun, 1 Sep 2024 23:30:07 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=/ FPll0jV4PxyNdXrm1LD7vl41L61qPHg5WSGzP/3BB4=; b=IfS7dUn466V9lU7+o /pN2I3d5skrLuTRAyJd6nTA480ISw2hwWfjVK6fdY4hFag8pdv/pPEPrb/MsJlcN 0Fg7NJZAwSdfqqU0MqD8DFtCB29wdQT08gzNcJFEa2kC8CGYJi0CNzjfychUZzBd oHxMfja4yW7l/ISeJ4Yo0aTq07gvMdAIzZHTNRLnQHo4CyVQ2kFgBqu7xQPC3fXd 5E2Nz2Q1djily8yaaFGDfgMbe8zo7vQNRDbUIdqxyUcPk3n4Ta7fdSZAJppyYYSc nSOby9FGk6fKuLIxAayv7YZo1ya3sMvvDeVOFS1I2Gwdjf3vEhMoUDRD43kH22/Q 2/ENQ== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 41c2pgvcn6-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 01 Sep 2024 23:30:06 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sun, 1 Sep 2024 23:30:05 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Sun, 1 Sep 2024 23:30:05 -0700 Received: from localhost.localdomain (unknown [10.29.52.211]) by maili.marvell.com (Postfix) with ESMTP id 1723B3F70D4; Sun, 1 Sep 2024 23:30:02 -0700 (PDT) From: Harman Kalra To: Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: Subject: [PATCH 5/5] common/cnxk: enable PF VF mbox Date: Mon, 2 Sep 2024 11:59:40 +0530 Message-ID: <20240902062940.182273-6-hkalra@marvell.com> X-Mailer: git-send-email 2.46.0.469.g4590f2e941 In-Reply-To: <20240902062940.182273-1-hkalra@marvell.com> References: <20240902062940.182273-1-hkalra@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: wPkXjZLpFzt1CMvHNdsMadQtf1rRXyjg X-Proofpoint-GUID: wPkXjZLpFzt1CMvHNdsMadQtf1rRXyjg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16 definitions=2024-09-01_06,2024-08-30_01,2024-05-17_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org * Setting up the mailbox communication between PF and VF for cn20k * Update PFVF mbox CSRs offset change introduced in cn20k * Memory to be allocated by PF driver for PF VF mbox should be aligned to next power of 2. Signed-off-by: Harman Kalra --- drivers/common/cnxk/hw/rvu.h | 18 ++- drivers/common/cnxk/roc_dev.c | 192 +++++++++++++++++++++++------ drivers/common/cnxk/roc_dev_priv.h | 16 ++- drivers/common/cnxk/roc_mbox.c | 16 +++ drivers/common/cnxk/roc_platform.h | 1 + 5 files changed, 200 insertions(+), 43 deletions(-) diff --git a/drivers/common/cnxk/hw/rvu.h b/drivers/common/cnxk/hw/rvu.h index 55e606f627..ee6cf30c5d 100644 --- a/drivers/common/cnxk/hw/rvu.h +++ b/drivers/common/cnxk/hw/rvu.h @@ -103,8 +103,12 @@ #define RVU_VF_MSIX_PBAX(a) (0xf0000ull | (uint64_t)(a) << 3) /* CN20k RVU mbox registers */ -#define RVU_MBOX_AF_AFPFX_TRIGX(a) (0x9000 | (a) << 3) -#define RVU_MBOX_PF_PFAF_TRIGX(a) RVU_PF_PFAF_MBOXX(a) +#define RVU_MBOX_AF_AFPFX_TRIGX(a) (0x9000 | (a) << 3) +#define RVU_MBOX_PF_PFAF_TRIGX(a) RVU_PF_PFAF_MBOXX(a) +#define RVU_MBOX_PF_VFX_PFVF_TRIGX(a) (0x2000 | (a) << 3) +#define RVU_MBOX_VF_VFPF_TRIGX(a) (0x3000 | (a) << 3) + +#define RVU_PF_VF_MBOX_ADDR (0xC40) /* cn20k Enum */ #define RVU_PFX_FUNC_PFAF_MBOX (0x80000) @@ -178,6 +182,16 @@ #define RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1 (0x7ull) #define RVU_MBOX_PF_INT_VEC_AFPF_MBOX (0x8ull) +#define RVU_MBOX_PF_VFPF_INTX(a) (0x1000 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_W1SX(a) (0x1020 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1SX(a) (0x1040 | (a) << 3) +#define RVU_MBOX_PF_VFPF_INT_ENA_W1CX(a) (0x1060 | (a) << 3) + +#define RVU_MBOX_PF_VFPF1_INTX(a) (0x1080 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_W1SX(a) (0x10a0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(a) (0x10c0 | (a) << 3) +#define RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(a) (0x10e0 | (a) << 3) + #define AF_BAR2_ALIASX_SIZE (0x100000ull) #define TIM_AF_BAR2_SEL (0x9000000ull) diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index 0b434e4a8a..f987410118 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -24,6 +24,9 @@ #define ROC_DEV_MBOX_PEND BIT_ULL(0) #define ROC_DEV_FLR_PEND BIT_ULL(1) +/* RVU PF interrupt status as received from AF*/ +#define RVU_PF_INTR_STATUS 0x3 + static void * mbox_mem_map(off_t off, size_t size) { @@ -84,7 +87,7 @@ pf_af_sync_msg(struct dev *dev, struct mbox_msghdr **rsp) break; } int_status = plt_read64(dev->mbox_reg_base + RVU_PF_INT); - } while ((int_status & 0x1) != 0x1); + } while (!(int_status & RVU_PF_INTR_STATUS)); /* Clear */ plt_write64(int_status, dev->mbox_reg_base + RVU_PF_INT); @@ -137,7 +140,7 @@ af_pf_wait_msg(struct dev *dev, uint16_t vf, int num_msg) break; } int_status = plt_read64(dev->mbox_reg_base + RVU_PF_INT); - } while ((int_status & 0x1) != 0x1); + } while (!(int_status & RVU_PF_INTR_STATUS)); /* Clear */ plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_INT); @@ -387,7 +390,7 @@ roc_vf_pf_mbox_irq(void *param) sz = sizeof(intrb.bits[0]) * MAX_VFPF_DWORD_BITS; memset(intrb.bits, 0, sz); for (vfpf = 0; vfpf < MAX_VFPF_DWORD_BITS; ++vfpf) { - intr = plt_read64(dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INTX(vfpf)); + intr = plt_read64(dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_intx[vfpf]); if (!intr) continue; @@ -396,7 +399,7 @@ roc_vf_pf_mbox_irq(void *param) /* Save and clear intr bits */ intrb.bits[vfpf] |= intr; - plt_write64(intr, dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INTX(vfpf)); + plt_write64(intr, dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_intx[vfpf]); signal_thread = true; } @@ -842,7 +845,7 @@ roc_pf_vf_mbox_irq(void *param) */ mbox_data = plt_read64(dev->mbox_reg_base + RVU_VF_VFPF_MBOX0); /* If interrupt occurred for down message */ - if (mbox_data & MBOX_DOWN_MSG) { + if (mbox_data & MBOX_DOWN_MSG || intr & BIT_ULL(1)) { mbox_data &= ~MBOX_DOWN_MSG; plt_write64(mbox_data, dev->mbox_reg_base + RVU_VF_VFPF_MBOX0); @@ -850,7 +853,7 @@ roc_pf_vf_mbox_irq(void *param) process_msgs(dev, dev->mbox); } /* If interrupt occurred for UP message */ - if (mbox_data & MBOX_UP_MSG) { + if (mbox_data & MBOX_UP_MSG || intr & BIT_ULL(0)) { mbox_data &= ~MBOX_UP_MSG; plt_write64(mbox_data, dev->mbox_reg_base + RVU_VF_VFPF_MBOX0); @@ -903,35 +906,61 @@ mbox_register_pf_irq(struct plt_pci_device *pci_dev, struct dev *dev) int i, rc; /* HW clear irq */ - for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) - plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INT_ENA_W1CX(i)); + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_int_ena_w1c[i]); + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf1_mbox_int_ena_w1c[i]); + } plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_INT_ENA_W1C); /* MBOX interrupt for VF(0...63) <-> PF */ - rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat.pfvf_vec); + rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat->pfvf_mbox0_vec); if (rc) { plt_err("Fail to register PF(VF0-63) mbox irq"); return rc; } /* MBOX interrupt for VF(64...128) <-> PF */ - rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat.pfvf1_vec); + rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat->pfvf_mbox1_vec); if (rc) { plt_err("Fail to register PF(VF64-128) mbox irq"); return rc; } + + /* Additional interrupt vector which can be used by VF -> PF using when + * RVU_VF_VFPF_TRIG(1) trigger register. + */ + if (roc_model_is_cn20k()) { + /* MBOX1 interrupt for VF(0...63) <-> PF */ + rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, + dev->mbox_plat->pfvf1_mbox0_vec); + + if (rc) { + plt_err("Fail to register PF1(VF0-63) mbox irq"); + return rc; + } + /* MBOX1 interrupt for VF(64...128) <-> PF */ + rc = dev_irq_register(intr_handle, roc_vf_pf_mbox_irq, dev, + dev->mbox_plat->pfvf1_mbox1_vec); + + if (rc) { + plt_err("Fail to register PF1(VF64-128) mbox irq"); + return rc; + } + } /* MBOX interrupt AF <-> PF */ - rc = dev_irq_register(intr_handle, roc_af_pf_mbox_irq, dev, dev->mbox_plat.pfaf_vec); + rc = dev_irq_register(intr_handle, roc_af_pf_mbox_irq, dev, dev->mbox_plat->pfaf_vec); if (rc) { plt_err("Fail to register AF<->PF mbox irq"); return rc; } /* HW enable intr */ - for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) - plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INT_ENA_W1SX(i)); + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_int_ena_w1s[i]); + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf1_mbox_int_ena_w1s[i]); + } plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_INT); plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_INT_ENA_W1S); @@ -978,20 +1007,32 @@ mbox_unregister_pf_irq(struct plt_pci_device *pci_dev, struct dev *dev) int i; /* HW clear irq */ - for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) - plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INT_ENA_W1CX(i)); + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_int_ena_w1c[i]); + plt_write64(~0ull, dev->mbox_reg_base + dev->mbox_plat->pfvf1_mbox_int_ena_w1c[i]); + } plt_write64(~0ull, dev->mbox_reg_base + RVU_PF_INT_ENA_W1C); /* Unregister the interrupt handler for each vectors */ /* MBOX interrupt for VF(0...63) <-> PF */ - dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat.pfvf_vec); + dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat->pfvf_mbox0_vec); /* MBOX interrupt for VF(64...128) <-> PF */ - dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat.pfvf1_vec); + dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, dev->mbox_plat->pfvf_mbox1_vec); + + if (roc_model_is_cn20k()) { + /* MBOX1 interrupt for VF(0...63) <-> PF */ + dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, + dev->mbox_plat->pfvf1_mbox0_vec); + + /* MBOX1 interrupt for VF(64...128) <-> PF */ + dev_irq_unregister(intr_handle, roc_vf_pf_mbox_irq, dev, + dev->mbox_plat->pfvf1_mbox1_vec); + } /* MBOX interrupt AF <-> PF */ - dev_irq_unregister(intr_handle, roc_af_pf_mbox_irq, dev, dev->mbox_plat.pfaf_vec); + dev_irq_unregister(intr_handle, roc_af_pf_mbox_irq, dev, dev->mbox_plat->pfaf_vec); } static void @@ -1222,9 +1263,15 @@ clear_rvum_interrupts(struct dev *dev) plt_write64(intr, dev->mbox_reg_base + RVU_PF_INT); for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { /* Clear MBOX interrupts */ - intr = plt_read64(dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INTX(i)); - if (intr) - plt_write64(intr, dev->mbox_reg_base + RVU_PF_VFPF_MBOX_INTX(i)); + intr = plt_read64(dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_intx[i]); + if (intr) { + plt_write64(intr, + dev->mbox_reg_base + dev->mbox_plat->pfvf_mbox_intx[i]); + if (roc_model_is_cn20k()) + plt_write64(intr, + dev->mbox_reg_base + + dev->mbox_plat->pfvf1_mbox_intx[i]); + } /* Clear VF FLR interrupts */ intr = plt_read64(dev->bar2 + RVU_PF_VFFLR_INTX(i)); if (intr) @@ -1265,6 +1312,32 @@ dev_vf_hwcap_update(struct plt_pci_device *pci_dev, struct dev *dev) } } +static uintptr_t +cn20k_pfvf_mbox_alloc(struct dev *dev, uint16_t max_vfs) +{ + char name[PLT_MEMZONE_NAMESIZE]; + const struct plt_memzone *mz; + uint32_t vf_mbox_region; + + vf_mbox_region = plt_align64pow2(MBOX_SIZE * max_vfs); + /* Allocating memory for LMT region */ + sprintf(name, "PFVF_MBOX_REGION%x", dev->pf_func); + + mz = plt_memzone_reserve_aligned(name, vf_mbox_region, 0, MBOX_SIZE); + if (!mz) { + plt_err("Memory alloc failed: %s", strerror(errno)); + goto fail; + } + + dev->vf_mbox_base = mz->iova; + dev->vf_mbox_mz = mz; + plt_write64(dev->vf_mbox_base, dev->mbox_reg_base + RVU_PF_VF_MBOX_ADDR); + + return dev->vf_mbox_base; +fail: + return (uintptr_t)NULL; +} + static uintptr_t dev_vf_mbase_get(struct plt_pci_device *pci_dev, struct dev *dev) { @@ -1274,8 +1347,11 @@ dev_vf_mbase_get(struct plt_pci_device *pci_dev, struct dev *dev) if (dev_is_vf(dev)) return 0; - /* For CN10K onwards, it is just after PF MBOX */ - if (!roc_model_is_cn9k()) + if (roc_model_is_cn20k()) + return cn20k_pfvf_mbox_alloc(dev, pci_dev->max_vfs); + + /* For CN10K, it is just after PF MBOX */ + if (roc_model_is_cn10k()) return dev->bar4 + MBOX_SIZE; pa = plt_read64(dev->bar2 + RVU_PF_VF_BAR4_ADDR); @@ -1430,8 +1506,10 @@ dev_cache_line_size_valid(void) } static void -mbox_platform_changes(struct mbox_platform *mbox_plat, uintptr_t bar2, uintptr_t bar4) +mbox_platform_changes(struct mbox_platform *mbox_plat, uintptr_t bar2, uintptr_t bar4, bool is_vf) { + int i; + if (roc_model_is_cn20k()) { /* For CN20K, AF allocates mbox memory in DRAM and writes PF * regions/offsets in RVU_MBOX_AF_PFX_ADDR, the RVU_PFX_FUNC_PFAF_MBOX @@ -1443,14 +1521,37 @@ mbox_platform_changes(struct mbox_platform *mbox_plat, uintptr_t bar2, uintptr_t ((uint64_t)RVU_BLOCK_ADDR_MBOX << RVU_FUNC_BLKADDR_SHIFT)); /* Interrupt vectors */ mbox_plat->pfaf_vec = RVU_MBOX_PF_INT_VEC_AFPF_MBOX; - mbox_plat->pfvf_vec = RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; - mbox_plat->pfvf1_vec = RVU_MBOX_PF_INT_VEC_VFPF_MBOX1; + mbox_plat->pfvf_mbox0_vec = RVU_MBOX_PF_INT_VEC_VFPF_MBOX0; + mbox_plat->pfvf_mbox1_vec = RVU_MBOX_PF_INT_VEC_VFPF_MBOX1; + mbox_plat->pfvf1_mbox0_vec = RVU_MBOX_PF_INT_VEC_VFPF1_MBOX0; + mbox_plat->pfvf1_mbox1_vec = RVU_MBOX_PF_INT_VEC_VFPF1_MBOX1; + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + mbox_plat->pfvf_mbox_int_ena_w1s[i] = RVU_MBOX_PF_VFPF_INT_ENA_W1SX(i); + mbox_plat->pfvf_mbox_int_ena_w1c[i] = RVU_MBOX_PF_VFPF_INT_ENA_W1CX(i); + mbox_plat->pfvf_mbox_intx[i] = RVU_MBOX_PF_VFPF_INTX(i); + mbox_plat->pfvf1_mbox_int_ena_w1s[i] = RVU_MBOX_PF_VFPF1_INT_ENA_W1SX(i); + mbox_plat->pfvf1_mbox_int_ena_w1c[i] = RVU_MBOX_PF_VFPF1_INT_ENA_W1CX(i); + mbox_plat->pfvf1_mbox_intx[i] = RVU_MBOX_PF_VFPF1_INTX(i); + } } else { mbox_plat->mbox_reg_base = bar2; mbox_plat->mbox_region_base = bar4; mbox_plat->pfaf_vec = RVU_PF_INT_VEC_AFPF_MBOX; - mbox_plat->pfvf_vec = RVU_PF_INT_VEC_VFPF_MBOX0; - mbox_plat->pfvf1_vec = RVU_PF_INT_VEC_VFPF_MBOX1; + mbox_plat->pfvf_mbox0_vec = RVU_PF_INT_VEC_VFPF_MBOX0; + mbox_plat->pfvf_mbox1_vec = RVU_PF_INT_VEC_VFPF_MBOX1; + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + mbox_plat->pfvf_mbox_int_ena_w1s[i] = RVU_PF_VFPF_MBOX_INT_ENA_W1SX(i); + mbox_plat->pfvf_mbox_int_ena_w1c[i] = RVU_PF_VFPF_MBOX_INT_ENA_W1CX(i); + mbox_plat->pfvf_mbox_intx[i] = RVU_PF_VFPF_MBOX_INTX(i); + } + } + if (is_vf) { + if (roc_model_is_cn20k()) + mbox_plat->mbox_region_base = + bar2 + (RVU_VF_MBOX_REGION + + ((uint64_t)RVU_BLOCK_ADDR_MBOX << RVU_FUNC_BLKADDR_SHIFT)); + if (roc_model_is_cn10k()) + mbox_plat->mbox_region_base = bar2 + RVU_VF_MBOX_REGION; } } @@ -1462,6 +1563,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) int direction, up_direction, rc; uintptr_t vf_mbase = 0; uint64_t intr_offset; + bool is_vf; if (!dev_cache_line_size_valid()) return -EFAULT; @@ -1471,12 +1573,21 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) return -EFAULT; } + dev_vf_hwcap_update(pci_dev, dev); + is_vf = dev_is_vf(dev); + bar2 = (uintptr_t)pci_dev->mem_resource[2].addr; bar4 = (uintptr_t)pci_dev->mem_resource[4].addr; - mbox_platform_changes(&dev->mbox_plat, bar2, bar4); + dev->mbox_plat = plt_zmalloc(sizeof(struct mbox_platform), 0); + if (!dev->mbox_plat) { + plt_err("Failed to allocate mem for mbox_plat"); + rc = -ENOMEM; + goto fail; + } + mbox_platform_changes(dev->mbox_plat, bar2, bar4, is_vf); - mbox_reg_base = dev->mbox_plat.mbox_reg_base; - mbox_region_base = dev->mbox_plat.mbox_region_base; + mbox_reg_base = dev->mbox_plat->mbox_reg_base; + mbox_region_base = dev->mbox_plat->mbox_region_base; if (mbox_reg_base == 0 || mbox_region_base == 0) { plt_err("Failed to get PCI bars"); rc = -ENODEV; @@ -1499,12 +1610,9 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) dev->maxvf = pci_dev->max_vfs; dev->bar2 = bar2; dev->bar4 = bar4; - dev->mbox_reg_base = dev->mbox_plat.mbox_reg_base; - dev_vf_hwcap_update(pci_dev, dev); + dev->mbox_reg_base = dev->mbox_plat->mbox_reg_base; - if (dev_is_vf(dev)) { - if (!roc_model_is_cn9k()) - mbox_region_base = bar2 + RVU_VF_MBOX_REGION; + if (is_vf) { direction = MBOX_DIR_VFPF; up_direction = MBOX_DIR_VFPF_UP; intr_offset = RVU_VF_INT; @@ -1585,7 +1693,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) } /* Register VF-FLR irq handlers */ - if (!dev_is_vf(dev)) { + if (!is_vf) { rc = dev_vf_flr_register_irqs(pci_dev, dev); if (rc) goto stop_msg_thrd; @@ -1622,6 +1730,8 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) mbox_fini(dev->mbox); mbox_fini(&dev->mbox_up); error: + plt_free(dev->mbox_plat); +fail: return rc; } @@ -1653,8 +1763,13 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) mbox_unregister_irq(pci_dev, dev); - if (!dev_is_vf(dev)) + if (!dev_is_vf(dev)) { vf_flr_unregister_irqs(pci_dev, dev); + /* Releasing memory allocated for mbox region */ + if (dev->vf_mbox_mz) + plt_memzone_free(dev->vf_mbox_mz); + } + /* Release PF - VF */ mbox = &dev->mbox_vfpf; if (mbox->hwbase && mbox->dev) @@ -1674,6 +1789,7 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) mbox_fini(mbox); dev->mbox_active = 0; + plt_free(dev->mbox_plat); /* Disable MSIX vectors */ dev_irqs_disable(intr_handle); return 0; diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h index 75395e5fd5..434e165b56 100644 --- a/drivers/common/cnxk/roc_dev_priv.h +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -102,8 +102,16 @@ struct mbox_sync { struct mbox_platform { uint8_t pfaf_vec; - uint8_t pfvf_vec; - uint8_t pfvf1_vec; + uint8_t pfvf_mbox0_vec; + uint8_t pfvf_mbox1_vec; + uint8_t pfvf1_mbox0_vec; + uint8_t pfvf1_mbox1_vec; + uint64_t pfvf_mbox_intx[MAX_VFPF_DWORD_BITS]; + uint64_t pfvf_mbox_int_ena_w1s[MAX_VFPF_DWORD_BITS]; + uint64_t pfvf_mbox_int_ena_w1c[MAX_VFPF_DWORD_BITS]; + uint64_t pfvf1_mbox_intx[MAX_VFPF_DWORD_BITS]; + uint64_t pfvf1_mbox_int_ena_w1s[MAX_VFPF_DWORD_BITS]; + uint64_t pfvf1_mbox_int_ena_w1c[MAX_VFPF_DWORD_BITS]; uintptr_t mbox_reg_base; uintptr_t mbox_region_base; }; @@ -137,7 +145,9 @@ struct dev { const struct plt_memzone *lmt_mz; struct mbox_sync sync; uintptr_t mbox_reg_base; - struct mbox_platform mbox_plat; + uintptr_t vf_mbox_base; + const struct plt_memzone *vf_mbox_mz; + struct mbox_platform *mbox_plat; } __plt_cache_aligned; struct npa { diff --git a/drivers/common/cnxk/roc_mbox.c b/drivers/common/cnxk/roc_mbox.c index 14368c5beb..db77babfdb 100644 --- a/drivers/common/cnxk/roc_mbox.c +++ b/drivers/common/cnxk/roc_mbox.c @@ -66,6 +66,22 @@ cn20k_mbox_setup(struct mbox *mbox, int direction) mbox->trigger = RVU_MBOX_PF_PFAF_TRIGX(1); mbox->tr_shift = 0; break; + case MBOX_DIR_PFVF: + mbox->trigger = RVU_MBOX_PF_VFX_PFVF_TRIGX(1); + mbox->tr_shift = 4; + break; + case MBOX_DIR_PFVF_UP: + mbox->trigger = RVU_MBOX_PF_VFX_PFVF_TRIGX(0); + mbox->tr_shift = 4; + break; + case MBOX_DIR_VFPF: + mbox->trigger = RVU_MBOX_VF_VFPF_TRIGX(0); + mbox->tr_shift = 0; + break; + case MBOX_DIR_VFPF_UP: + mbox->trigger = RVU_MBOX_VF_VFPF_TRIGX(1); + mbox->tr_shift = 0; + break; default: return -ENODEV; } diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 658bafb935..558c214f92 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -118,6 +118,7 @@ #define __plt_aligned __rte_aligned #define plt_align32pow2 rte_align32pow2 +#define plt_align64pow2 rte_align64pow2 #define plt_align32prevpow2 rte_align32prevpow2 #define plt_bitmap rte_bitmap -- 2.25.1