From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 733F8A0547; Thu, 9 Sep 2021 13:15:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB8F24114A; Thu, 9 Sep 2021 13:15:36 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2085.outbound.protection.outlook.com [40.107.21.85]) by mails.dpdk.org (Postfix) with ESMTP id 46E024115B for ; Thu, 9 Sep 2021 13:15:35 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DbLFG6jqSkk4HQk19Y/Wwnba8SmEC5yg1h8Pv1MN1ayQCiKRwi3a6mLbTEAJ/iZUL5NhHtP+GAsPc8n/7TVP4UyBnzbq9xQh4Stk2d5PqCeVwlL5Vh4lGsHXRIYA4zDOnvvzjAbTYMgHglZGQozPuvZkYUKivySlBtU4GLKS0V1appGCm9316tF6cgXqDsGyrt3wauzpnEEUSoUyneuW479BCKQmectoyaFOgcyDnK+PtNMTTSUrp8m7Qp7o/LCJi0XxTOErVXAhNq5/DqUlIT0f7Ijc6L+OBxpB2lG09ITL/vEecU8dAh80TxI7eJbWQI2dzs6cbNa/TeZ43HgTzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=r7gFii+SKVVFUHhRYaF1TAIK0BMrberUQ3+nIos+FQo=; b=bYGzYR/+tAFVM2q1GrRDJE2JS9SdhKo1Mpi7VeXoiYyDZdfNs86xgY3YO6O+omWvGSA0gJEqCDkt7VTygq/oC6jGYtre35S+1Xu8t0cDp1CUSm3QtjpLNcy6el9Lhp6zFYMVRQs6x/Wq7Qfc+RqE0SsZdriGwF0lIhpDvaPENi4jfrmNboCKsuWwEgIl895IvHBaY7oJFF/m3ZR4LYuuIHvYPa2qzP94O+EmWF3a5P34bc43Q6k7xx0KpUp2kmxn8wEeArbshMIxGYIrMYBCi7aOav35kNV++42wEJ9KFQvPuKh5uAEVZbjWt+Y0scGBwCpwB2PtCsYO1taSnV/glA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r7gFii+SKVVFUHhRYaF1TAIK0BMrberUQ3+nIos+FQo=; b=dXah6pLNTW5Syb5PILCCpDaJQ/OWxZjaWKI/119PuXrW1/Pfbe7jyvXIp8D40O5AWefEmlsvuQj1EJZquzCn3CEWOCv7ZEH91dCzdLNheHA7rmIPBCNFc9Lql/wP2xSmCvE8/oamRj+BR/F42ie5IJ/tqjP27ciuMzfvxhpYjOY= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=nxp.com; Received: from VI1PR04MB6960.eurprd04.prod.outlook.com (2603:10a6:803:12d::10) by VI1PR04MB5935.eurprd04.prod.outlook.com (2603:10a6:803:e9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.16; Thu, 9 Sep 2021 11:15:34 +0000 Received: from VI1PR04MB6960.eurprd04.prod.outlook.com ([fe80::d496:fcce:f667:7aa7]) by VI1PR04MB6960.eurprd04.prod.outlook.com ([fe80::d496:fcce:f667:7aa7%8]) with mapi id 15.20.4500.015; Thu, 9 Sep 2021 11:15:34 +0000 From: Gagandeep Singh To: dev@dpdk.org Cc: nipun.gupta@nxp.com, thomas@monjalon.net, Gagandeep Singh Date: Thu, 9 Sep 2021 16:44:59 +0530 Message-Id: <20210909111500.3901706-6-g.singh@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210909111500.3901706-1-g.singh@nxp.com> References: <20210909111500.3901706-1-g.singh@nxp.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SG2PR06CA0149.apcprd06.prod.outlook.com (2603:1096:1:1f::27) To VI1PR04MB6960.eurprd04.prod.outlook.com (2603:10a6:803:12d::10) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from lsv03457.swis.in-blr01.nxp.com (14.142.151.118) by SG2PR06CA0149.apcprd06.prod.outlook.com (2603:1096:1:1f::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 11:15:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a8f6cb75-049f-4ad3-25da-08d973832429 X-MS-TrafficTypeDiagnostic: VI1PR04MB5935: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RB1Kn/nSJmU4W1u/zmqO2/GjgDs4LuLjy7WAVnRRTD91z3dlXm/jfDSZNvYA8xvtXzyTOuFVrFSyIJZ20LdJWzwgn62HLW85LE/yKAEtJfpDbbhYEryP1AzoiV7uSBlVlfC3Bq8UD8/oebzKlG0T0mmquU6hJgqKPq6IJIA3Eh2aKRbEemyepvhwDPHtNY8lgrg/Sm5K/p/+3JDy3Ucjl2UZIOdaBqExSAO3jb9tbnQG/sPDCnT0PWJcGpHZqmrtNmROS7r+ycfiwmb+SzMsC+p6yJlIhLFHvOC85HbRxGq5kJCgt8+5tpvKVTyR5xGBF4UIgNA7aqbQEXnriGg9mlz3E2W+EBHXcNjJQz1HOOqWa/LwL049xRt1v+te5uICB/RCqQ5sEYDbhTd27PCqymT4O5pwn4PPWSY3hI4S7E6jPNPn07uC4QfvLpDVpRD+qN6zKKz+UutEqMaBFFC4pFsFo9gUdvi+2lk6oaXO6DW+ISWKQtTjEKwEqnbNpEx0lAXPZIcb/Tv/OOmR4Mi0jk8t5p3dpVjKiE6O5EVUE1Vw/xLYDs5EVIcTaLPrd9tNmApS+uUjX2nkJIDZ+JxWRyNssHh+aADGNeU8TG3RrEDmkkJQVTdv6QbwATVY3HuDxTQIxwyDehwyR5OgYm3e7XWajn0BZeVbLt0e2/lpB2Wh0s2nZw94a5Vy4Wf0lo6x66id8LHJ/D5sUutdZOay6tKIlZoeClRRMGx15k1nWAE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR04MB6960.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(346002)(366004)(376002)(396003)(6916009)(8936002)(38350700002)(1006002)(6666004)(38100700002)(86362001)(5660300002)(26005)(83380400001)(36756003)(55236004)(1076003)(2906002)(66946007)(186003)(7696005)(4326008)(52116002)(66556008)(316002)(6486002)(8676002)(478600001)(30864003)(66476007)(956004)(2616005)(110426009); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?j5V+hsxAzlst2FkTZmuyKsF5grGPYZB9+O1WPJdrvRMgj/gyyCA7Eoh6R/Z6?= =?us-ascii?Q?q3HiOjOA6pKvJHpfyQnnAQUcvJ7Y6zkV3VzXZBCOzXcGUkyN89wcg3TQEagp?= =?us-ascii?Q?XQ3ToQ81aKuS5cwgs/l5etcvU2wzKpAGRTuUGS53MBZTjiWAjvcCJfV5Rifz?= =?us-ascii?Q?ioEtc2XrZoQ4wuaau+jJlQBmhNfL39XUjGKuNqogXyKpCi193f1Mq9+LA4R+?= =?us-ascii?Q?ftIpwneyNn+3B9eqYSCgH5OGXIin5U4T/khkfkStHMn5mvza5WaACmAsgqLC?= =?us-ascii?Q?Rcmrbs1pnNrgLToI0SJEyAgAII945alc+ev9DyXbPYP/HPa5j10sSOgEWaZM?= =?us-ascii?Q?tpNWjGIGctXOVmAV0ZxU27pF+MgQZgFYNBCIWQIu83VF+ADp7kknX32/XX4X?= =?us-ascii?Q?suBCt+wDoq9iOBqHTyZ7C9aBCqNnxfeLw94rtfE3wSkDEw9LQUZliIl58+Ox?= =?us-ascii?Q?pUDfFnnFjcI3mjzhKkTxRLV40Wx5lNl6svHO+ZyUdby+ubqk2Nep74qanCTg?= =?us-ascii?Q?Inh5+1wPfLUpx1gT9QtkEFoXt6Rf43Uu4aD2yLZFhX0YCO8Xmqn3wE85WKys?= =?us-ascii?Q?2P44yre9xBtT6PDkvt3AMRUaSKNfbfobqEJ/vCY4zN/infrNAGAWG5NC9ZGC?= =?us-ascii?Q?PI7MvFFVvlhA446p9MMMvPylfphwkd7wkKtiActw0I5DbN+jXhnBl7SW7ZFO?= =?us-ascii?Q?4qTwUw3bVKBTBNYgqKv6v1r8boQtLRAxn55PbZLoNQiBF9e6YtneaQGxazya?= =?us-ascii?Q?KXQqnERuNwkojFVbBHzprhn9cEVB5MwS9ZhXNy4LtaiHrxiooMvFQrZ8c1/B?= =?us-ascii?Q?vpuFJgrDDpbpFlqxKeuVNiebjLwiJTP/yqSN6wfaXgG2cgGL1thBemx46Ils?= =?us-ascii?Q?w8f19ALlKEJg8Dvt3e6FNtYDOM252i5BCsmdWhGNGOoKx0048bO8qmrQscdq?= =?us-ascii?Q?eE/qtqiVtoVjHrZq0MxzvwslCRBOUG/lBdjkJBmz2/8g6eS34cuo4wfRWGR0?= =?us-ascii?Q?WJR5/pwv+dFzMg9GwmNOd6ZFXu2LvzVpIfAv5WO4n3YYDtMmGl0Kg92aILFt?= =?us-ascii?Q?aDRa5nXeMij+MnOzADbEc+DX/i5IfmUblA/l48R1eNm3WTpKyk9cZSAbNnJn?= =?us-ascii?Q?PPiWESkGlNUpjM4zMAv/NcNbj9Gn7gdkcBQlLksiTsYZimdib8aXmW3+sKmE?= =?us-ascii?Q?eLJ04hxXT15vMgqtQooqWKV1iCiz1agXR5126yTihZZIyEma49l0wBINqMcQ?= =?us-ascii?Q?ptJQxjedE3sF6WmdjP69va2c9brhtpOhw4f1Cw7azZ8qekLFGwcxWtW2TgyN?= =?us-ascii?Q?Jdts2gET7jMlE6YHCnk/b4fw?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8f6cb75-049f-4ad3-25da-08d973832429 X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB6960.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 11:15:34.1052 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8O3DmsVdTVrbnb76n4bC7E/VNBpqAsAa5vve4OBIN2d+xRj5ObONMejZ74Q5eUEM X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB5935 Subject: [dpdk-dev] [PATCH 5/6] dma/dpaa: support DMA operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch support copy, submit, completed and completed status functionality of DMA driver. Signed-off-by: Gagandeep Singh --- drivers/dma/dpaa/dpaa_qdma.c | 346 +++++++++++++++++++++++++++++++++++ drivers/dma/dpaa/dpaa_qdma.h | 4 + 2 files changed, 350 insertions(+) diff --git a/drivers/dma/dpaa/dpaa_qdma.c b/drivers/dma/dpaa/dpaa_qdma.c index 0297166550..1ce1999165 100644 --- a/drivers/dma/dpaa/dpaa_qdma.c +++ b/drivers/dma/dpaa/dpaa_qdma.c @@ -15,11 +15,48 @@ qdma_desc_addr_set64(struct fsl_qdma_format *ccdf, u64 addr) ccdf->addr_lo = rte_cpu_to_le_32(lower_32_bits(addr)); } +static inline u64 +qdma_ccdf_get_queue(const struct fsl_qdma_format *ccdf) +{ + return ccdf->cfg8b_w1 & 0xff; +} + +static inline int +qdma_ccdf_get_offset(const struct fsl_qdma_format *ccdf) +{ + return (rte_le_to_cpu_32(ccdf->cfg) & QDMA_CCDF_MASK) + >> QDMA_CCDF_OFFSET; +} + +static inline void +qdma_ccdf_set_format(struct fsl_qdma_format *ccdf, int offset) +{ + ccdf->cfg = rte_cpu_to_le_32(QDMA_CCDF_FOTMAT | offset); +} + +static inline int +qdma_ccdf_get_status(const struct fsl_qdma_format *ccdf) +{ + return (rte_le_to_cpu_32(ccdf->status) & QDMA_CCDF_MASK) + >> QDMA_CCDF_STATUS; +} + +static inline void +qdma_ccdf_set_ser(struct fsl_qdma_format *ccdf, int status) +{ + ccdf->status = rte_cpu_to_le_32(QDMA_CCDF_SER | status); +} + static inline void qdma_csgf_set_len(struct fsl_qdma_format *csgf, int len) { csgf->cfg = rte_cpu_to_le_32(len & QDMA_SG_LEN_MASK); } +static inline void qdma_csgf_set_f(struct fsl_qdma_format *csgf, int len) +{ + csgf->cfg = rte_cpu_to_le_32(QDMA_SG_FIN | (len & QDMA_SG_LEN_MASK)); +} + static inline int ilog2(int x) { int log = 0; @@ -43,6 +80,16 @@ static void qdma_writel(u32 val, void *addr) QDMA_OUT(addr, val); } +static u32 qdma_readl_be(void *addr) +{ + return QDMA_IN_BE(addr); +} + +static void qdma_writel_be(u32 val, void *addr) +{ + QDMA_OUT_BE(addr, val); +} + static void *dma_pool_alloc(int size, int aligned, dma_addr_t *phy_addr) { void *virt_addr; @@ -97,6 +144,31 @@ static void fsl_qdma_free_chan_resources(struct fsl_qdma_chan *fsl_chan) fsl_qdma->desc_allocated--; } +static void fsl_qdma_comp_fill_memcpy(struct fsl_qdma_comp *fsl_comp, + dma_addr_t dst, dma_addr_t src, u32 len) +{ + struct fsl_qdma_format *csgf_src, *csgf_dest; + + /* Note: command table (fsl_comp->virt_addr) is getting filled + * directly in cmd descriptors of queues while enqueuing the descriptor + * please refer fsl_qdma_enqueue_desc + * frame list table (virt_addr) + 1) and source, + * destination descriptor table + * (fsl_comp->desc_virt_addr and fsl_comp->desc_virt_addr+1) move to + * the control path to fsl_qdma_pre_request_enqueue_comp_sd_desc + */ + csgf_src = (struct fsl_qdma_format *)fsl_comp->virt_addr + 2; + csgf_dest = (struct fsl_qdma_format *)fsl_comp->virt_addr + 3; + + /* Status notification is enqueued to status queue. */ + qdma_desc_addr_set64(csgf_src, src); + qdma_csgf_set_len(csgf_src, len); + qdma_desc_addr_set64(csgf_dest, dst); + qdma_csgf_set_len(csgf_dest, len); + /* This entry is the last entry. */ + qdma_csgf_set_f(csgf_dest, len); +} + /* * Pre-request command descriptor and compound S/G for enqueue. */ @@ -153,6 +225,25 @@ static int fsl_qdma_pre_request_enqueue_comp_sd_desc( return 0; } +/* + * Request a command descriptor for enqueue. + */ +static struct fsl_qdma_comp * +fsl_qdma_request_enqueue_desc(struct fsl_qdma_chan *fsl_chan) +{ + struct fsl_qdma_queue *queue = fsl_chan->queue; + struct fsl_qdma_comp *comp_temp; + + if (!list_empty(&queue->comp_free)) { + comp_temp = list_first_entry(&queue->comp_free, + struct fsl_qdma_comp, + list); + list_del(&comp_temp->list); + return comp_temp; + } + + return NULL; +} static struct fsl_qdma_queue *fsl_qdma_alloc_queue_resources(struct fsl_qdma_engine *fsl_qdma) @@ -287,6 +378,54 @@ static int fsl_qdma_halt(struct fsl_qdma_engine *fsl_qdma) return 0; } +static int +fsl_qdma_queue_transfer_complete(struct fsl_qdma_engine *fsl_qdma, + void *block, int id, const uint16_t nb_cpls, + uint16_t *last_idx, + enum rte_dma_status_code *status) +{ + struct fsl_qdma_queue *fsl_queue = fsl_qdma->queue; + struct fsl_qdma_queue *fsl_status = fsl_qdma->status[id]; + struct fsl_qdma_queue *temp_queue; + struct fsl_qdma_format *status_addr; + struct fsl_qdma_comp *fsl_comp = NULL; + u32 reg, i; + int count = 0; + + while (count < nb_cpls) { + reg = qdma_readl_be(block + FSL_QDMA_BSQSR); + if (reg & FSL_QDMA_BSQSR_QE_BE) + return count; + + status_addr = fsl_status->virt_head; + + i = qdma_ccdf_get_queue(status_addr) + + id * fsl_qdma->n_queues; + temp_queue = fsl_queue + i; + fsl_comp = list_first_entry(&temp_queue->comp_used, + struct fsl_qdma_comp, + list); + list_del(&fsl_comp->list); + + reg = qdma_readl_be(block + FSL_QDMA_BSQMR); + reg |= FSL_QDMA_BSQMR_DI_BE; + + qdma_desc_addr_set64(status_addr, 0x0); + fsl_status->virt_head++; + if (fsl_status->virt_head == fsl_status->cq + fsl_status->n_cq) + fsl_status->virt_head = fsl_status->cq; + qdma_writel_be(reg, block + FSL_QDMA_BSQMR); + *last_idx = fsl_comp->index; + if (status != NULL) + status[count] = RTE_DMA_STATUS_SUCCESSFUL; + + list_add_tail(&fsl_comp->list, &temp_queue->comp_free); + count++; + + } + return count; +} + static int fsl_qdma_reg_init(struct fsl_qdma_engine *fsl_qdma) { struct fsl_qdma_queue *fsl_queue = fsl_qdma->queue; @@ -381,6 +520,65 @@ static int fsl_qdma_reg_init(struct fsl_qdma_engine *fsl_qdma) return 0; } +static void * +fsl_qdma_prep_memcpy(void *fsl_chan, dma_addr_t dst, + dma_addr_t src, size_t len, + void *call_back, + void *param) +{ + struct fsl_qdma_comp *fsl_comp; + + fsl_comp = + fsl_qdma_request_enqueue_desc((struct fsl_qdma_chan *)fsl_chan); + if (!fsl_comp) + return NULL; + + fsl_comp->qchan = fsl_chan; + fsl_comp->call_back_func = call_back; + fsl_comp->params = param; + + fsl_qdma_comp_fill_memcpy(fsl_comp, dst, src, len); + return (void *)fsl_comp; +} + +static int fsl_qdma_enqueue_desc(struct fsl_qdma_chan *fsl_chan, + struct fsl_qdma_comp *fsl_comp, + uint64_t flags) +{ + struct fsl_qdma_queue *fsl_queue = fsl_chan->queue; + void *block = fsl_queue->block_base; + struct fsl_qdma_format *ccdf; + u32 reg; + + /* retrieve and store the register value in big endian + * to avoid bits swap + */ + reg = qdma_readl_be(block + + FSL_QDMA_BCQSR(fsl_queue->id)); + if (reg & (FSL_QDMA_BCQSR_QF_XOFF_BE)) + return -1; + + /* filling descriptor command table */ + ccdf = (struct fsl_qdma_format *)fsl_queue->virt_head; + qdma_desc_addr_set64(ccdf, fsl_comp->bus_addr + 16); + qdma_ccdf_set_format(ccdf, qdma_ccdf_get_offset(fsl_comp->virt_addr)); + qdma_ccdf_set_ser(ccdf, qdma_ccdf_get_status(fsl_comp->virt_addr)); + fsl_comp->index = fsl_queue->virt_head - fsl_queue->cq; + fsl_queue->virt_head++; + + if (fsl_queue->virt_head == fsl_queue->cq + fsl_queue->n_cq) + fsl_queue->virt_head = fsl_queue->cq; + + list_add_tail(&fsl_comp->list, &fsl_queue->comp_used); + + if (flags == RTE_DMA_OP_FLAG_SUBMIT) { + reg = qdma_readl_be(block + FSL_QDMA_BCQMR(fsl_queue->id)); + reg |= FSL_QDMA_BCQMR_EI_BE; + qdma_writel_be(reg, block + FSL_QDMA_BCQMR(fsl_queue->id)); + } + return 0; +} + static int fsl_qdma_alloc_chan_resources(struct fsl_qdma_chan *fsl_chan) { struct fsl_qdma_queue *fsl_queue = fsl_chan->queue; @@ -490,6 +688,150 @@ dpaa_qdma_queue_setup(struct rte_dmadev *dmadev, return dpaa_get_channel(fsl_qdma, vchan); } +static int +dpaa_qdma_submit(struct rte_dmadev *dmadev, uint16_t vchan) +{ + struct fsl_qdma_engine *fsl_qdma = dmadev->dev_private; + struct fsl_qdma_chan *fsl_chan = + &fsl_qdma->chans[fsl_qdma->vchan_map[vchan]]; + struct fsl_qdma_queue *fsl_queue = fsl_chan->queue; + void *block = fsl_queue->block_base; + u32 reg; + + reg = qdma_readl_be(block + FSL_QDMA_BCQMR(fsl_queue->id)); + reg |= FSL_QDMA_BCQMR_EI_BE; + qdma_writel_be(reg, block + FSL_QDMA_BCQMR(fsl_queue->id)); + + return 0; +} + +static int +dpaa_qdma_enqueue(struct rte_dmadev *dmadev, uint16_t vchan, + rte_iova_t src, rte_iova_t dst, + uint32_t length, uint64_t flags) +{ + struct fsl_qdma_engine *fsl_qdma = dmadev->dev_private; + struct fsl_qdma_chan *fsl_chan = + &fsl_qdma->chans[fsl_qdma->vchan_map[vchan]]; + int ret; + + void *fsl_comp = NULL; + + fsl_comp = fsl_qdma_prep_memcpy(fsl_chan, + (dma_addr_t)dst, (dma_addr_t)src, + length, NULL, NULL); + if (!fsl_comp) { + DPAA_QDMA_DP_DEBUG("fsl_comp is NULL\n"); + return -1; + } + ret = fsl_qdma_enqueue_desc(fsl_chan, fsl_comp, flags); + if (ret) + return -1; + + return 0; +} + +static uint16_t +dpaa_qdma_dequeue_status(struct rte_dmadev *dmadev, uint16_t vchan, + const uint16_t nb_cpls, uint16_t *last_idx, + enum rte_dma_status_code *st) +{ + struct fsl_qdma_engine *fsl_qdma = dmadev->dev_private; + int id = (int)((fsl_qdma->vchan_map[vchan]) / QDMA_QUEUES); + void *block; + unsigned int reg; + int intr; + void *status = fsl_qdma->status_base; + + intr = qdma_readl_be(status + FSL_QDMA_DEDR); + if (intr) { + DPAA_QDMA_ERR("DMA transaction error! %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW0R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW0R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW1R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW1R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW2R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW2R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW3R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW3R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFQIDR); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFQIDR %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECBR); + DPAA_QDMA_INFO("reg FSL_QDMA_DECBR %x\n", intr); + qdma_writel(0xffffffff, + status + FSL_QDMA_DEDR); + intr = qdma_readl(status + FSL_QDMA_DEDR); + } + + block = fsl_qdma->block_base + + FSL_QDMA_BLOCK_BASE_OFFSET(fsl_qdma, id); + + intr = fsl_qdma_queue_transfer_complete(fsl_qdma, block, id, nb_cpls, + last_idx, st); + if (intr < 0) { + void *ctrl = fsl_qdma->ctrl_base; + + reg = qdma_readl(ctrl + FSL_QDMA_DMR); + reg |= FSL_QDMA_DMR_DQD; + qdma_writel(reg, ctrl + FSL_QDMA_DMR); + qdma_writel(0, block + FSL_QDMA_BCQIER(0)); + DPAA_QDMA_ERR("QDMA: status err!\n"); + } + + return intr; +} + + +static uint16_t +dpaa_qdma_dequeue(struct rte_dmadev *dmadev, + uint16_t vchan, const uint16_t nb_cpls, + uint16_t *last_idx, __rte_unused bool *has_error) +{ + struct fsl_qdma_engine *fsl_qdma = dmadev->dev_private; + int id = (int)((fsl_qdma->vchan_map[vchan]) / QDMA_QUEUES); + void *block; + unsigned int reg; + int intr; + void *status = fsl_qdma->status_base; + + intr = qdma_readl_be(status + FSL_QDMA_DEDR); + if (intr) { + DPAA_QDMA_ERR("DMA transaction error! %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW0R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW0R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW1R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW1R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW2R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW2R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFDW3R); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFDW3R %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECFQIDR); + DPAA_QDMA_INFO("reg FSL_QDMA_DECFQIDR %x\n", intr); + intr = qdma_readl(status + FSL_QDMA_DECBR); + DPAA_QDMA_INFO("reg FSL_QDMA_DECBR %x\n", intr); + qdma_writel(0xffffffff, + status + FSL_QDMA_DEDR); + intr = qdma_readl(status + FSL_QDMA_DEDR); + } + + block = fsl_qdma->block_base + + FSL_QDMA_BLOCK_BASE_OFFSET(fsl_qdma, id); + + intr = fsl_qdma_queue_transfer_complete(fsl_qdma, block, id, nb_cpls, + last_idx, NULL); + if (intr < 0) { + void *ctrl = fsl_qdma->ctrl_base; + + reg = qdma_readl(ctrl + FSL_QDMA_DMR); + reg |= FSL_QDMA_DMR_DQD; + qdma_writel(reg, ctrl + FSL_QDMA_DMR); + qdma_writel(0, block + FSL_QDMA_BCQIER(0)); + DPAA_QDMA_ERR("QDMA: status err!\n"); + } + + return intr; +} + static struct rte_dmadev_ops dpaa_qdma_ops = { .dev_info_get = dpaa_info_get, .dev_configure = dpaa_qdma_configure, @@ -612,6 +954,10 @@ dpaa_qdma_probe(__rte_unused struct rte_dpaa_driver *dpaa_drv, dpaa_dev->dmadev = dmadev; dmadev->dev_ops = &dpaa_qdma_ops; dmadev->device = &dpaa_dev->device; + dmadev->copy = dpaa_qdma_enqueue; + dmadev->submit = dpaa_qdma_submit; + dmadev->completed = dpaa_qdma_dequeue; + dmadev->completed_status = dpaa_qdma_dequeue_status; /* Invoke PMD device initialization function */ ret = dpaa_qdma_init(dmadev); diff --git a/drivers/dma/dpaa/dpaa_qdma.h b/drivers/dma/dpaa/dpaa_qdma.h index f482b16334..ef3c37e3a8 100644 --- a/drivers/dma/dpaa/dpaa_qdma.h +++ b/drivers/dma/dpaa/dpaa_qdma.h @@ -5,6 +5,10 @@ #ifndef _DPAA_QDMA_H_ #define _DPAA_QDMA_H_ +#ifndef BIT +#define BIT(nr) (1UL << (nr)) +#endif + #define CORE_NUMBER 4 #define RETRIES 5 -- 2.25.1