From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64EC6A052E; Tue, 21 Jan 2020 09:37:29 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5633F1252; Tue, 21 Jan 2020 09:37:28 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id EBB6511A4 for ; Tue, 21 Jan 2020 09:37:26 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00L8ZdaC015080 for ; Tue, 21 Jan 2020 00:37:26 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=; b=DrbaAbZRXYmYA972Ab5VvyxYbOgzDpdBghIgZkbwrnNIBvFbBROt5XeW3W1KD6JJfEgP 0lfjZrWyhTSJPw68x5Otztt31NsBj+QN/q4MiJtcu4XAmhhg8VucGk9UMpBj3OmQmNyW i0ZztGm7wY6Mmk73+r7jZa3+bMdyDz1ndNuqcts6rDNlki7KCKjpMbzwpFcbImvUu/VP Bp3+rt9K60MOAIsWJa/mERItNXYpu2LUMJZn0Oe8XbHl6XBSkQwGXL8Tw1tKZEVFFQ+A teZvUoLiOZSWCTbMztB8mi+77lSzw7c1H6WnQx+uvQIAu6podskjJH4sCD4Fh7HRbgJ5 zg== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2xm2dt14yj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 21 Jan 2020 00:37:25 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 21 Jan 2020 00:37:23 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.174) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 21 Jan 2020 00:37:22 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MP1NhptZ2/qBLuDvWErX/q71hv2GlRDgHMJ2hcXG83CrxoQfNJpRpWSAPk0Fq2JndBclbX0aoPpmqRfzwPrChgqB0s5i7ukOU5pD0tc7qKlUGUQO3RQVrGf3W+Mim0Vv7JPnMlocoIIVSfBBs6MR4ofXgMKISrkwrvXwkrCMQA3dvkSMTtt7c4lBzXLqPiDyUpARU6bPaM6ZM/MrD3FqZ5kIhbZs0ROwQ2PC3aHhj1ci6n6k6jSuO0jVMP3V+7ZH32ImD74lwqHjfhTeuCHHK0Mn6W0iHUuCM1itAB+h+Pgwi5LJ18ytGr8YA2ve2dLypkchs20+eP4MTKm7bT1puQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=; b=NQLceOdZ9ZcYdBrZpykrTDCd3+wK9jFiPfecYUo3KQNJ1EQFB5OS1yvHPrJx3HjRPGWfbu7mHpGgn1jR1yvnaw9g/ddAJbOgMnwSQDCsLolX5qIbGzaL60OZxFLb6CQHWkBbH4zGEuBBr2Txd0+u3C/2Q89KCpSI5e1rw+7o9QabWukc8O6BJ6eBqLWZqHxbRB1XVNvZBCLGxui52N/MZE45LNNL5e5zGhsED4q0uSwr+okn8XpFW1g/uvQDamVZXwk94A8zmZq2//DTVtOp/9wLaUUNko6kPYiaZ0Y+vq8CxnsnzJX4cei8JO/JeIz0JGgHKKuYTFXoLxQhJiDmWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=; b=WZDHgVZzXs7vIsUOSA3nY7OVLi1Bpgt6QQtDAG6a0ivAyILwSlSeRj1HCfTM/sjVWvS99XCTpH9SGLh3Ks6nhzG8SExeH8jH+EnbP9C3V+jEehVI2EknSsZnv7ip1asge9F2fBSPDpf6G8NPBwl/J4TG/O2leW+Us5XbqkQAvLM= Received: from CH2PR18MB3270.namprd18.prod.outlook.com (52.132.247.204) by CH2PR18MB3430.namprd18.prod.outlook.com (52.132.246.206) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2644.25; Tue, 21 Jan 2020 08:37:21 +0000 Received: from CH2PR18MB3270.namprd18.prod.outlook.com ([fe80::919e:f1c5:c6ec:ae09]) by CH2PR18MB3270.namprd18.prod.outlook.com ([fe80::919e:f1c5:c6ec:ae09%3]) with mapi id 15.20.2644.027; Tue, 21 Jan 2020 08:37:21 +0000 From: Sunil Kumar Kori To: Sunil Kumar Kori , Jerin Jacob Kollanukkaran , Nithin Kumar Dabilpuram , Vamsi Krishna Attunuru CC: "dev@dpdk.org" , Harman Kalra Thread-Topic: [PATCH v7 2/2] common/octeontx2: add polling based response mbox message Thread-Index: AQHVyrnBLdGrFQPf/k+O66gEkCMvtKf01Z6w Date: Tue, 21 Jan 2020 08:37:21 +0000 Message-ID: References: <20191220065645.22858-2-skori@marvell.com> <20200114090453.4548-1-skori@marvell.com> <20200114090453.4548-2-skori@marvell.com> In-Reply-To: <20200114090453.4548-2-skori@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.140.231.66] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: aec965d1-a00b-4586-a1f9-08d79e4d21c0 x-ms-traffictypediagnostic: CH2PR18MB3430: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0289B6431E x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(346002)(366004)(396003)(136003)(39860400002)(189003)(199004)(66946007)(76116006)(64756008)(66446008)(107886003)(2906002)(15650500001)(66476007)(5660300002)(54906003)(66556008)(110136005)(52536014)(4326008)(316002)(186003)(6636002)(81156014)(81166006)(9686003)(8676002)(8936002)(55016002)(71200400001)(33656002)(478600001)(6506007)(86362001)(55236004)(26005)(7696005)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:CH2PR18MB3430; H:CH2PR18MB3270.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JtkVAIUvYrMuz0kocc9VnAtR0SLFxiTOuIvW3lysuZ13uY0oJnD4HnfvfsuvzXNffxtNVRBu5wHEO/MAn4aGP7Tms7Cx2juXTHS//Hp+Q1l7sB7kqYaXh3NxJXX0LtTrQLxIrXaeaPKfl002jw6jYpynGzj/V+D/F8TLR73EL2ytnvZuYXW4g9jMhJtASnmxB/e91p6MuiNceT5Z67RIrvaI/2gKIjWKOth0sjDpw63RwF569zBKI1HwZ8A4/Rkh7kdkBijkVjIsVwWFLaRUQI9QbOFPyGShlMNc2pRSJpScwINQSAyXDDjN+8lPZF4sBiIXm58BZRM2J7jEk091K6qGxBIamW0duu8aKne8vARQCSm213BADEtOxlJyRzYZaodWzcyjXWDG23/ZCSDrPzOSHfkd2uN5kxEHdvLmQDhnxVtFULZFXRs4XYTbBTpLcZca/VN4tMyMO28QFnFiYWlm8vR2SKLsIpUwP0hf2Cus3bcADncQXgky+6fmvUVv Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: aec965d1-a00b-4586-a1f9-08d79e4d21c0 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jan 2020 08:37:21.0738 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: asupfFb1OrHbqNQaCk8+JyJ4cVCkQ2FyjcAGDhuBajsmoKY6pIeJMEhVIZiagQKovpIwevR3x0jQ3vlSKFNMxA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3430 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-21_02:2020-01-20, 2020-01-21 signatures=0 Subject: Re: [dpdk-dev] [PATCH v7 2/2] common/octeontx2: add polling based response mbox message X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hello Thomas, I have uploaded next version after handling Jerin's comments. Can you please look into the series so that It can be applied if there is n= o more comments ?=20 Thanks. Regards Sunil Kumar Kori >-----Original Message----- >From: Sunil Kumar Kori >Sent: Tuesday, January 14, 2020 2:35 PM >To: Jerin Jacob Kollanukkaran ; Nithin Kumar >Dabilpuram ; Vamsi Krishna Attunuru > >Cc: dev@dpdk.org; Sunil Kumar Kori ; Harman Kalra > >Subject: [PATCH v7 2/2] common/octeontx2: add polling based response >mbox message > >Currently otx2_mbox_get_rsp_xxx get response once AF driver interrupts >after completion. But this function will get into deadlock if called in an= other >interrupt context. > >To avoid it, implemented another version of this function which polls on >dedicated memory for a given timeout. > >Also after clearing interrupt, there could UP messages available for proce= ssing. >So irq handler must check mbox messages. > >Signed-off-by: Sunil Kumar Kori >Signed-off-by: Harman Kalra >Acked-by: Jerin Jacob >--- >v7: > - Corrected spelling in commit message >v6: > - Removed unnecessary code. >v5: > - Fix shared library compilation error >v4: > - used rte_io_rmb instead of rte_rmb in mbox_poll. >v3: > - Remove experimental tag as API is defined static. > - Merge all changes to single patch. >v2: > - Included Makefile and meson build changes. > - Rebased patch on 19.11-rc4 > > drivers/common/octeontx2/otx2_dev.c | 41 +++++++++++----------- >drivers/common/octeontx2/otx2_mbox.c | 51 ++++++++++++++++++++++++--- >- drivers/common/octeontx2/otx2_mbox.h | 5 +-- > 3 files changed, 70 insertions(+), 27 deletions(-) > >diff --git a/drivers/common/octeontx2/otx2_dev.c >b/drivers/common/octeontx2/otx2_dev.c >index 0fc799e4a..d61c712fa 100644 >--- a/drivers/common/octeontx2/otx2_dev.c >+++ b/drivers/common/octeontx2/otx2_dev.c >@@ -577,17 +577,16 @@ otx2_pf_vf_mbox_irq(void *param) > > intr =3D otx2_read64(dev->bar2 + RVU_VF_INT); > if (intr =3D=3D 0) >- return; >+ otx2_base_dbg("Proceeding to check mbox UP messages if >any"); > > otx2_write64(intr, dev->bar2 + RVU_VF_INT); > otx2_base_dbg("Irq 0x%" PRIx64 "(pf:%d,vf:%d)", intr, dev->pf, dev- >>vf); >- if (intr) { >- /* First process all configuration messages */ >- otx2_process_msgs(dev, dev->mbox); > >- /* Process Uplink messages */ >- otx2_process_msgs_up(dev, &dev->mbox_up); >- } >+ /* First process all configuration messages */ >+ otx2_process_msgs(dev, dev->mbox); >+ >+ /* Process Uplink messages */ >+ otx2_process_msgs_up(dev, &dev->mbox_up); > } > > static void >@@ -598,18 +597,16 @@ otx2_af_pf_mbox_irq(void *param) > > intr =3D otx2_read64(dev->bar2 + RVU_PF_INT); > if (intr =3D=3D 0) >- return; >+ otx2_base_dbg("Proceeding to check mbox UP messages if >any"); > > otx2_write64(intr, dev->bar2 + RVU_PF_INT); >- > otx2_base_dbg("Irq 0x%" PRIx64 "(pf:%d,vf:%d)", intr, dev->pf, dev- >>vf); >- if (intr) { >- /* First process all configuration messages */ >- otx2_process_msgs(dev, dev->mbox); > >- /* Process Uplink messages */ >- otx2_process_msgs_up(dev, &dev->mbox_up); >- } >+ /* First process all configuration messages */ >+ otx2_process_msgs(dev, dev->mbox); >+ >+ /* Process Uplink messages */ >+ otx2_process_msgs_up(dev, &dev->mbox_up); > } > > static int >@@ -900,6 +897,7 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev, >void *otx2_dev) { > int up_direction =3D MBOX_DIR_PFAF_UP; > int rc, direction =3D MBOX_DIR_PFAF; >+ uint64_t intr_offset =3D RVU_PF_INT; > struct otx2_dev *dev =3D otx2_dev; > uintptr_t bar2, bar4; > uint64_t bar4_addr; >@@ -924,15 +922,18 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev, >void *otx2_dev) > if (otx2_dev_is_vf(dev)) { > direction =3D MBOX_DIR_VFPF; > up_direction =3D MBOX_DIR_VFPF_UP; >+ intr_offset =3D RVU_VF_INT; > } > > /* Initialize the local mbox */ >- rc =3D otx2_mbox_init(&dev->mbox_local, bar4, bar2, direction, 1); >+ rc =3D otx2_mbox_init(&dev->mbox_local, bar4, bar2, direction, 1, >+ intr_offset); > if (rc) > goto error; > dev->mbox =3D &dev->mbox_local; > >- rc =3D otx2_mbox_init(&dev->mbox_up, bar4, bar2, up_direction, 1); >+ rc =3D otx2_mbox_init(&dev->mbox_up, bar4, bar2, up_direction, 1, >+ intr_offset); > if (rc) > goto error; > >@@ -967,13 +968,15 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev, >void *otx2_dev) > } > /* Init mbox object */ > rc =3D otx2_mbox_init(&dev->mbox_vfpf, (uintptr_t)hwbase, >- bar2, MBOX_DIR_PFVF, pci_dev->max_vfs); >+ bar2, MBOX_DIR_PFVF, pci_dev->max_vfs, >+ intr_offset); > if (rc) > goto iounmap; > > /* PF -> VF UP messages */ > rc =3D otx2_mbox_init(&dev->mbox_vfpf_up, (uintptr_t)hwbase, >- bar2, MBOX_DIR_PFVF_UP, pci_dev- >>max_vfs); >+ bar2, MBOX_DIR_PFVF_UP, pci_dev- >>max_vfs, >+ intr_offset); > if (rc) > goto mbox_fini; > } >diff --git a/drivers/common/octeontx2/otx2_mbox.c >b/drivers/common/octeontx2/otx2_mbox.c >index c359bf42f..1ec0d6f69 100644 >--- a/drivers/common/octeontx2/otx2_mbox.c >+++ b/drivers/common/octeontx2/otx2_mbox.c >@@ -11,6 +11,7 @@ > #include > > #include "otx2_mbox.h" >+#include "otx2_dev.h" > > #define RVU_AF_AFPF_MBOX0 (0x02000) > #define RVU_AF_AFPF_MBOX1 (0x02008) >@@ -59,12 +60,13 @@ otx2_mbox_reset(struct otx2_mbox *mbox, int devid) >} > > int >-otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase, >- uintptr_t reg_base, int direction, int ndevs) >+otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase, uintptr_t >reg_base, >+ int direction, int ndevs, uint64_t intr_offset) > { > struct otx2_mbox_dev *mdev; > int devid; > >+ mbox->intr_offset =3D intr_offset; > mbox->reg_base =3D reg_base; > mbox->hwbase =3D hwbase; > >@@ -244,6 +246,39 @@ otx2_mbox_get_rsp(struct otx2_mbox *mbox, int >devid, void **msg) > return msghdr->rc; > } > >+/** >+ * Polling for given wait time to get mailbox response */ static int >+mbox_poll(struct otx2_mbox *mbox, uint32_t wait) { >+ uint32_t timeout =3D 0, sleep =3D 1; >+ uint32_t wait_us =3D wait * 1000; >+ uint64_t rsp_reg =3D 0; >+ uintptr_t reg_addr; >+ >+ reg_addr =3D mbox->reg_base + mbox->intr_offset; >+ do { >+ rsp_reg =3D otx2_read64(reg_addr); >+ >+ if (timeout >=3D wait_us) >+ return -ETIMEDOUT; >+ >+ rte_delay_us(sleep); >+ timeout +=3D sleep; >+ } while (!rsp_reg); >+ >+ rte_smp_rmb(); >+ >+ /* Clear interrupt */ >+ otx2_write64(rsp_reg, reg_addr); >+ >+ /* Reset mbox */ >+ otx2_mbox_reset(mbox, 0); >+ >+ return 0; >+} >+ > /** > * @internal > * Wait and get mailbox response with timeout @@ -321,11 +356,15 @@ >otx2_mbox_wait_for_rsp_tmo(struct otx2_mbox *mbox, int devid, uint32_t >tmo) > } > > /* Wait message */ >- rc =3D mbox_wait(mbox, devid, tmo); >- if (rc) >- return rc; >+ if (rte_thread_is_intr()) >+ rc =3D mbox_poll(mbox, tmo); >+ else >+ rc =3D mbox_wait(mbox, devid, tmo); > >- return mdev->msgs_acked; >+ if (!rc) >+ rc =3D mdev->num_msgs; >+ >+ return rc; > } > > /** >diff --git a/drivers/common/octeontx2/otx2_mbox.h >b/drivers/common/octeontx2/otx2_mbox.h >index e0e4e2f63..0535cec36 100644 >--- a/drivers/common/octeontx2/otx2_mbox.h >+++ b/drivers/common/octeontx2/otx2_mbox.h >@@ -73,6 +73,7 @@ struct otx2_mbox { > uint16_t tx_size; /* Size of Tx region */ > uint16_t ndevs; /* The number of peers */ > struct otx2_mbox_dev *dev; >+ uint64_t intr_offset; /* Offset to interrupt register */ > }; > > /* Header which precedes all mbox messages */ @@ -1562,8 +1563,8 @@ >struct tim_enable_rsp { const char *otx2_mbox_id2name(uint16_t id); int >otx2_mbox_id2size(uint16_t id); void otx2_mbox_reset(struct otx2_mbox >*mbox, int devid); -int otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t >hwbase, >- uintptr_t reg_base, int direction, int ndevs); >+int otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase, uintptr_t >reg_base, >+ int direction, int ndevsi, uint64_t intr_offset); > void otx2_mbox_fini(struct otx2_mbox *mbox); void >otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid); int >otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid); >-- >2.17.1