From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 64EC6A052E;
Tue, 21 Jan 2020 09:37:29 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 5633F1252;
Tue, 21 Jan 2020 09:37:28 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com
[67.231.156.173]) by dpdk.org (Postfix) with ESMTP id EBB6511A4
for <dev@dpdk.org>; Tue, 21 Jan 2020 09:37:26 +0100 (CET)
Received: from pps.filterd (m0045851.ppops.net [127.0.0.1])
by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id
00L8ZdaC015080 for <dev@dpdk.org>; Tue, 21 Jan 2020 00:37:26 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
h=from : to : cc :
subject : date : message-id : references : in-reply-to : content-type :
content-transfer-encoding : mime-version; s=pfpt0818;
bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=;
b=DrbaAbZRXYmYA972Ab5VvyxYbOgzDpdBghIgZkbwrnNIBvFbBROt5XeW3W1KD6JJfEgP
0lfjZrWyhTSJPw68x5Otztt31NsBj+QN/q4MiJtcu4XAmhhg8VucGk9UMpBj3OmQmNyW
i0ZztGm7wY6Mmk73+r7jZa3+bMdyDz1ndNuqcts6rDNlki7KCKjpMbzwpFcbImvUu/VP
Bp3+rt9K60MOAIsWJa/mERItNXYpu2LUMJZn0Oe8XbHl6XBSkQwGXL8Tw1tKZEVFFQ+A
teZvUoLiOZSWCTbMztB8mi+77lSzw7c1H6WnQx+uvQIAu6podskjJH4sCD4Fh7HRbgJ5 zg==
Received: from sc-exch01.marvell.com ([199.233.58.181])
by mx0b-0016f401.pphosted.com with ESMTP id 2xm2dt14yj-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT)
for <dev@dpdk.org>; Tue, 21 Jan 2020 00:37:25 -0800
Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com
(10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 21 Jan
2020 00:37:23 -0800
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.174)
by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id
15.0.1497.2 via Frontend Transport; Tue, 21 Jan 2020 00:37:22 -0800
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=MP1NhptZ2/qBLuDvWErX/q71hv2GlRDgHMJ2hcXG83CrxoQfNJpRpWSAPk0Fq2JndBclbX0aoPpmqRfzwPrChgqB0s5i7ukOU5pD0tc7qKlUGUQO3RQVrGf3W+Mim0Vv7JPnMlocoIIVSfBBs6MR4ofXgMKISrkwrvXwkrCMQA3dvkSMTtt7c4lBzXLqPiDyUpARU6bPaM6ZM/MrD3FqZ5kIhbZs0ROwQ2PC3aHhj1ci6n6k6jSuO0jVMP3V+7ZH32ImD74lwqHjfhTeuCHHK0Mn6W0iHUuCM1itAB+h+Pgwi5LJ18ytGr8YA2ve2dLypkchs20+eP4MTKm7bT1puQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=;
b=NQLceOdZ9ZcYdBrZpykrTDCd3+wK9jFiPfecYUo3KQNJ1EQFB5OS1yvHPrJx3HjRPGWfbu7mHpGgn1jR1yvnaw9g/ddAJbOgMnwSQDCsLolX5qIbGzaL60OZxFLb6CQHWkBbH4zGEuBBr2Txd0+u3C/2Q89KCpSI5e1rw+7o9QabWukc8O6BJ6eBqLWZqHxbRB1XVNvZBCLGxui52N/MZE45LNNL5e5zGhsED4q0uSwr+okn8XpFW1g/uvQDamVZXwk94A8zmZq2//DTVtOp/9wLaUUNko6kPYiaZ0Y+vq8CxnsnzJX4cei8JO/JeIz0JGgHKKuYTFXoLxQhJiDmWw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com;
dkim=pass header.d=marvell.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=XH6fH5FrPQkegW3WgqeQ1sMAymLDDJL2Xrkw4GCDA+M=;
b=WZDHgVZzXs7vIsUOSA3nY7OVLi1Bpgt6QQtDAG6a0ivAyILwSlSeRj1HCfTM/sjVWvS99XCTpH9SGLh3Ks6nhzG8SExeH8jH+EnbP9C3V+jEehVI2EknSsZnv7ip1asge9F2fBSPDpf6G8NPBwl/J4TG/O2leW+Us5XbqkQAvLM=
Received: from CH2PR18MB3270.namprd18.prod.outlook.com (52.132.247.204) by
CH2PR18MB3430.namprd18.prod.outlook.com (52.132.246.206) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.2644.25; Tue, 21 Jan 2020 08:37:21 +0000
Received: from CH2PR18MB3270.namprd18.prod.outlook.com
([fe80::919e:f1c5:c6ec:ae09]) by CH2PR18MB3270.namprd18.prod.outlook.com
([fe80::919e:f1c5:c6ec:ae09%3]) with mapi id 15.20.2644.027; Tue, 21 Jan 2020
08:37:21 +0000
From: Sunil Kumar Kori <skori@marvell.com>
To: Sunil Kumar Kori <skori@marvell.com>, Jerin Jacob Kollanukkaran
<jerinj@marvell.com>, Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>,
Vamsi Krishna Attunuru <vattunuru@marvell.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, Harman Kalra <hkalra@marvell.com>
Thread-Topic: [PATCH v7 2/2] common/octeontx2: add polling based response mbox
message
Thread-Index: AQHVyrnBLdGrFQPf/k+O66gEkCMvtKf01Z6w
Date: Tue, 21 Jan 2020 08:37:21 +0000
Message-ID: <CH2PR18MB3270BF34EBDE94062F800F71B40D0@CH2PR18MB3270.namprd18.prod.outlook.com>
References: <20191220065645.22858-2-skori@marvell.com>
<20200114090453.4548-1-skori@marvell.com>
<20200114090453.4548-2-skori@marvell.com>
In-Reply-To: <20200114090453.4548-2-skori@marvell.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [14.140.231.66]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: aec965d1-a00b-4586-a1f9-08d79e4d21c0
x-ms-traffictypediagnostic: CH2PR18MB3430:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <CH2PR18MB343064663BF09347273EC675B40D0@CH2PR18MB3430.namprd18.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-forefront-prvs: 0289B6431E
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(4636009)(376002)(346002)(366004)(396003)(136003)(39860400002)(189003)(199004)(66946007)(76116006)(64756008)(66446008)(107886003)(2906002)(15650500001)(66476007)(5660300002)(54906003)(66556008)(110136005)(52536014)(4326008)(316002)(186003)(6636002)(81156014)(81166006)(9686003)(8676002)(8936002)(55016002)(71200400001)(33656002)(478600001)(6506007)(86362001)(55236004)(26005)(7696005)(21314003);
DIR:OUT; SFP:1101; SCL:1; SRVR:CH2PR18MB3430;
H:CH2PR18MB3270.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: marvell.com does not designate
permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: JtkVAIUvYrMuz0kocc9VnAtR0SLFxiTOuIvW3lysuZ13uY0oJnD4HnfvfsuvzXNffxtNVRBu5wHEO/MAn4aGP7Tms7Cx2juXTHS//Hp+Q1l7sB7kqYaXh3NxJXX0LtTrQLxIrXaeaPKfl002jw6jYpynGzj/V+D/F8TLR73EL2ytnvZuYXW4g9jMhJtASnmxB/e91p6MuiNceT5Z67RIrvaI/2gKIjWKOth0sjDpw63RwF569zBKI1HwZ8A4/Rkh7kdkBijkVjIsVwWFLaRUQI9QbOFPyGShlMNc2pRSJpScwINQSAyXDDjN+8lPZF4sBiIXm58BZRM2J7jEk091K6qGxBIamW0duu8aKne8vARQCSm213BADEtOxlJyRzYZaodWzcyjXWDG23/ZCSDrPzOSHfkd2uN5kxEHdvLmQDhnxVtFULZFXRs4XYTbBTpLcZca/VN4tMyMO28QFnFiYWlm8vR2SKLsIpUwP0hf2Cus3bcADncQXgky+6fmvUVv
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: aec965d1-a00b-4586-a1f9-08d79e4d21c0
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jan 2020 08:37:21.0738 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: asupfFb1OrHbqNQaCk8+JyJ4cVCkQ2FyjcAGDhuBajsmoKY6pIeJMEhVIZiagQKovpIwevR3x0jQ3vlSKFNMxA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3430
X-OriginatorOrg: marvell.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572
definitions=2020-01-21_02:2020-01-20,
2020-01-21 signatures=0
Subject: Re: [dpdk-dev] [PATCH v7 2/2] common/octeontx2: add polling based
response mbox message
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Hello Thomas,
I have uploaded next version after handling Jerin's comments.
Can you please look into the series so that It can be applied if there is n=
o more comments ?=20
Thanks.
Regards
Sunil Kumar Kori
>-----Original Message-----
>From: Sunil Kumar Kori <skori@marvell.com>
>Sent: Tuesday, January 14, 2020 2:35 PM
>To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Nithin Kumar
>Dabilpuram <ndabilpuram@marvell.com>; Vamsi Krishna Attunuru
><vattunuru@marvell.com>
>Cc: dev@dpdk.org; Sunil Kumar Kori <skori@marvell.com>; Harman Kalra
><hkalra@marvell.com>
>Subject: [PATCH v7 2/2] common/octeontx2: add polling based response
>mbox message
>
>Currently otx2_mbox_get_rsp_xxx get response once AF driver interrupts
>after completion. But this function will get into deadlock if called in an=
other
>interrupt context.
>
>To avoid it, implemented another version of this function which polls on
>dedicated memory for a given timeout.
>
>Also after clearing interrupt, there could UP messages available for proce=
ssing.
>So irq handler must check mbox messages.
>
>Signed-off-by: Sunil Kumar Kori <skori@marvell.com>
>Signed-off-by: Harman Kalra <hkalra@marvell.com>
>Acked-by: Jerin Jacob <jerinj@marvell.com>
>---
>v7:
> - Corrected spelling in commit message
>v6:
> - Removed unnecessary code.
>v5:
> - Fix shared library compilation error
>v4:
> - used rte_io_rmb instead of rte_rmb in mbox_poll.
>v3:
> - Remove experimental tag as API is defined static.
> - Merge all changes to single patch.
>v2:
> - Included Makefile and meson build changes.
> - Rebased patch on 19.11-rc4
>
> drivers/common/octeontx2/otx2_dev.c | 41 +++++++++++-----------
>drivers/common/octeontx2/otx2_mbox.c | 51 ++++++++++++++++++++++++---
>- drivers/common/octeontx2/otx2_mbox.h | 5 +--
> 3 files changed, 70 insertions(+), 27 deletions(-)
>
>diff --git a/drivers/common/octeontx2/otx2_dev.c
>b/drivers/common/octeontx2/otx2_dev.c
>index 0fc799e4a..d61c712fa 100644
>--- a/drivers/common/octeontx2/otx2_dev.c
>+++ b/drivers/common/octeontx2/otx2_dev.c
>@@ -577,17 +577,16 @@ otx2_pf_vf_mbox_irq(void *param)
>
> intr =3D otx2_read64(dev->bar2 + RVU_VF_INT);
> if (intr =3D=3D 0)
>- return;
>+ otx2_base_dbg("Proceeding to check mbox UP messages if
>any");
>
> otx2_write64(intr, dev->bar2 + RVU_VF_INT);
> otx2_base_dbg("Irq 0x%" PRIx64 "(pf:%d,vf:%d)", intr, dev->pf, dev-
>>vf);
>- if (intr) {
>- /* First process all configuration messages */
>- otx2_process_msgs(dev, dev->mbox);
>
>- /* Process Uplink messages */
>- otx2_process_msgs_up(dev, &dev->mbox_up);
>- }
>+ /* First process all configuration messages */
>+ otx2_process_msgs(dev, dev->mbox);
>+
>+ /* Process Uplink messages */
>+ otx2_process_msgs_up(dev, &dev->mbox_up);
> }
>
> static void
>@@ -598,18 +597,16 @@ otx2_af_pf_mbox_irq(void *param)
>
> intr =3D otx2_read64(dev->bar2 + RVU_PF_INT);
> if (intr =3D=3D 0)
>- return;
>+ otx2_base_dbg("Proceeding to check mbox UP messages if
>any");
>
> otx2_write64(intr, dev->bar2 + RVU_PF_INT);
>-
> otx2_base_dbg("Irq 0x%" PRIx64 "(pf:%d,vf:%d)", intr, dev->pf, dev-
>>vf);
>- if (intr) {
>- /* First process all configuration messages */
>- otx2_process_msgs(dev, dev->mbox);
>
>- /* Process Uplink messages */
>- otx2_process_msgs_up(dev, &dev->mbox_up);
>- }
>+ /* First process all configuration messages */
>+ otx2_process_msgs(dev, dev->mbox);
>+
>+ /* Process Uplink messages */
>+ otx2_process_msgs_up(dev, &dev->mbox_up);
> }
>
> static int
>@@ -900,6 +897,7 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev,
>void *otx2_dev) {
> int up_direction =3D MBOX_DIR_PFAF_UP;
> int rc, direction =3D MBOX_DIR_PFAF;
>+ uint64_t intr_offset =3D RVU_PF_INT;
> struct otx2_dev *dev =3D otx2_dev;
> uintptr_t bar2, bar4;
> uint64_t bar4_addr;
>@@ -924,15 +922,18 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev,
>void *otx2_dev)
> if (otx2_dev_is_vf(dev)) {
> direction =3D MBOX_DIR_VFPF;
> up_direction =3D MBOX_DIR_VFPF_UP;
>+ intr_offset =3D RVU_VF_INT;
> }
>
> /* Initialize the local mbox */
>- rc =3D otx2_mbox_init(&dev->mbox_local, bar4, bar2, direction, 1);
>+ rc =3D otx2_mbox_init(&dev->mbox_local, bar4, bar2, direction, 1,
>+ intr_offset);
> if (rc)
> goto error;
> dev->mbox =3D &dev->mbox_local;
>
>- rc =3D otx2_mbox_init(&dev->mbox_up, bar4, bar2, up_direction, 1);
>+ rc =3D otx2_mbox_init(&dev->mbox_up, bar4, bar2, up_direction, 1,
>+ intr_offset);
> if (rc)
> goto error;
>
>@@ -967,13 +968,15 @@ otx2_dev_priv_init(struct rte_pci_device *pci_dev,
>void *otx2_dev)
> }
> /* Init mbox object */
> rc =3D otx2_mbox_init(&dev->mbox_vfpf, (uintptr_t)hwbase,
>- bar2, MBOX_DIR_PFVF, pci_dev->max_vfs);
>+ bar2, MBOX_DIR_PFVF, pci_dev->max_vfs,
>+ intr_offset);
> if (rc)
> goto iounmap;
>
> /* PF -> VF UP messages */
> rc =3D otx2_mbox_init(&dev->mbox_vfpf_up, (uintptr_t)hwbase,
>- bar2, MBOX_DIR_PFVF_UP, pci_dev-
>>max_vfs);
>+ bar2, MBOX_DIR_PFVF_UP, pci_dev-
>>max_vfs,
>+ intr_offset);
> if (rc)
> goto mbox_fini;
> }
>diff --git a/drivers/common/octeontx2/otx2_mbox.c
>b/drivers/common/octeontx2/otx2_mbox.c
>index c359bf42f..1ec0d6f69 100644
>--- a/drivers/common/octeontx2/otx2_mbox.c
>+++ b/drivers/common/octeontx2/otx2_mbox.c
>@@ -11,6 +11,7 @@
> #include <rte_cycles.h>
>
> #include "otx2_mbox.h"
>+#include "otx2_dev.h"
>
> #define RVU_AF_AFPF_MBOX0 (0x02000)
> #define RVU_AF_AFPF_MBOX1 (0x02008)
>@@ -59,12 +60,13 @@ otx2_mbox_reset(struct otx2_mbox *mbox, int devid)
>}
>
> int
>-otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase,
>- uintptr_t reg_base, int direction, int ndevs)
>+otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase, uintptr_t
>reg_base,
>+ int direction, int ndevs, uint64_t intr_offset)
> {
> struct otx2_mbox_dev *mdev;
> int devid;
>
>+ mbox->intr_offset =3D intr_offset;
> mbox->reg_base =3D reg_base;
> mbox->hwbase =3D hwbase;
>
>@@ -244,6 +246,39 @@ otx2_mbox_get_rsp(struct otx2_mbox *mbox, int
>devid, void **msg)
> return msghdr->rc;
> }
>
>+/**
>+ * Polling for given wait time to get mailbox response */ static int
>+mbox_poll(struct otx2_mbox *mbox, uint32_t wait) {
>+ uint32_t timeout =3D 0, sleep =3D 1;
>+ uint32_t wait_us =3D wait * 1000;
>+ uint64_t rsp_reg =3D 0;
>+ uintptr_t reg_addr;
>+
>+ reg_addr =3D mbox->reg_base + mbox->intr_offset;
>+ do {
>+ rsp_reg =3D otx2_read64(reg_addr);
>+
>+ if (timeout >=3D wait_us)
>+ return -ETIMEDOUT;
>+
>+ rte_delay_us(sleep);
>+ timeout +=3D sleep;
>+ } while (!rsp_reg);
>+
>+ rte_smp_rmb();
>+
>+ /* Clear interrupt */
>+ otx2_write64(rsp_reg, reg_addr);
>+
>+ /* Reset mbox */
>+ otx2_mbox_reset(mbox, 0);
>+
>+ return 0;
>+}
>+
> /**
> * @internal
> * Wait and get mailbox response with timeout @@ -321,11 +356,15 @@
>otx2_mbox_wait_for_rsp_tmo(struct otx2_mbox *mbox, int devid, uint32_t
>tmo)
> }
>
> /* Wait message */
>- rc =3D mbox_wait(mbox, devid, tmo);
>- if (rc)
>- return rc;
>+ if (rte_thread_is_intr())
>+ rc =3D mbox_poll(mbox, tmo);
>+ else
>+ rc =3D mbox_wait(mbox, devid, tmo);
>
>- return mdev->msgs_acked;
>+ if (!rc)
>+ rc =3D mdev->num_msgs;
>+
>+ return rc;
> }
>
> /**
>diff --git a/drivers/common/octeontx2/otx2_mbox.h
>b/drivers/common/octeontx2/otx2_mbox.h
>index e0e4e2f63..0535cec36 100644
>--- a/drivers/common/octeontx2/otx2_mbox.h
>+++ b/drivers/common/octeontx2/otx2_mbox.h
>@@ -73,6 +73,7 @@ struct otx2_mbox {
> uint16_t tx_size; /* Size of Tx region */
> uint16_t ndevs; /* The number of peers */
> struct otx2_mbox_dev *dev;
>+ uint64_t intr_offset; /* Offset to interrupt register */
> };
>
> /* Header which precedes all mbox messages */ @@ -1562,8 +1563,8 @@
>struct tim_enable_rsp { const char *otx2_mbox_id2name(uint16_t id); int
>otx2_mbox_id2size(uint16_t id); void otx2_mbox_reset(struct otx2_mbox
>*mbox, int devid); -int otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t
>hwbase,
>- uintptr_t reg_base, int direction, int ndevs);
>+int otx2_mbox_init(struct otx2_mbox *mbox, uintptr_t hwbase, uintptr_t
>reg_base,
>+ int direction, int ndevsi, uint64_t intr_offset);
> void otx2_mbox_fini(struct otx2_mbox *mbox); void
>otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid); int
>otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
>--
>2.17.1