From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0076.outbound.protection.outlook.com [104.47.32.76]) by dpdk.org (Postfix) with ESMTP id 2EC4E1B024 for ; Tue, 23 Jan 2018 14:03:22 +0100 (CET) Received: from CY4PR03CA0013.namprd03.prod.outlook.com (10.168.162.23) by BL2PR03MB468.namprd03.prod.outlook.com (10.141.92.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.428.17; Tue, 23 Jan 2018 13:03:20 +0000 Received: from BN1AFFO11FD006.protection.gbl (2a01:111:f400:7c10::184) by CY4PR03CA0013.outlook.office365.com (2603:10b6:903:33::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.428.17 via Frontend Transport; Tue, 23 Jan 2018 13:03:19 +0000 Authentication-Results: spf=fail (sender IP is 192.88.158.2) smtp.mailfrom=nxp.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.158.2 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.158.2; helo=az84smr01.freescale.net; Received: from az84smr01.freescale.net (192.88.158.2) by BN1AFFO11FD006.mail.protection.outlook.com (10.58.52.66) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.20.428.12 via Frontend Transport; Tue, 23 Jan 2018 13:03:19 +0000 Received: from b27504-OptiPlex-790.ap.freescale.net (b27504-OptiPlex-790.ap.freescale.net [10.232.132.60]) by az84smr01.freescale.net (8.14.3/8.14.0) with ESMTP id w0NCR9X6020973; Tue, 23 Jan 2018 05:27:39 -0700 From: Nipun Gupta To: CC: , , , Nipun Gupta Date: Tue, 23 Jan 2018 17:57:07 +0530 Message-ID: <1516710427-22843-8-git-send-email-nipun.gupta@nxp.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1516710427-22843-1-git-send-email-nipun.gupta@nxp.com> References: <1516710671-13897-1-git-send-email-nipun.gupta@nxp.com> <1516710427-22843-1-git-send-email-nipun.gupta@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131611861994784340; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.158.2; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(346002)(376002)(39380400002)(396003)(2980300002)(1110001)(1109001)(339900001)(189003)(199004)(336011)(16586007)(2351001)(69596002)(104016004)(6666003)(54906003)(5660300001)(2950100002)(316002)(85426001)(97736004)(36756003)(53936002)(106466001)(50466002)(51416003)(50226002)(81156014)(8676002)(356003)(498600001)(8936002)(86362001)(81166006)(59450400001)(305945005)(4326008)(6862004)(48376002)(77096007)(76176011)(2906002)(68736007)(47776003)(26005)(105606002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR03MB468; H:az84smr01.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD006; 1:oe7147qCtevbR6o2oMyjzp3w+PfIm7Xf+mvmFXEncqfXqH41RtLLRw15gQpSxSau2DOyUCIDgZbFkA05CJUmEVTKzrH5LbGxecKQRHn+YqN2bln95qra8sUT5icN5dvN MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 726f8046-a5be-4ba1-e4f3-08d56261ace1 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(5600026)(4604075)(2017052603307); SRVR:BL2PR03MB468; X-Microsoft-Exchange-Diagnostics: 1; BL2PR03MB468; 3:Ztrn9wVV9sgy6jSYi0phENT/tQF0bBnvr41eYds7jFftjL7PeHeSXKtJq/XErvyQYzgT9mKhajxBvb34JJeig2U15S4sAaU9Jn9Fnp+MYbj2ucdgvFf/6n5Q9Vy7+lVV4WQ3qggv5pEtnzXI05ebPdM0hEf3U9tRODWZCDaaxWHzCdNlkIux4BNNeTMKA+KxCMmO65o9OISnTq7UhLbTkMws0OvFQyF8r9X/TL8/5omx+Git4Z6Lkush1IWlK8T9nPm/Nlp6RHALHfwZPkSQ8tirsWk7VbQG6iFlTEFC0sScPA6CGtzbcgvP80Ll51oyadFDmkbgs2WJgZjx0yQH/ho4SapZ7AtN+PblDmWQbaY=; 25:y7KtWOwOjb+wn3664Zfu25a0V+Lf3BtNsqES9IHxjG4RNtCJmRbuVBXG+9f5kHKjY1WOono7TCrxc9dKNmg/D+GsILqd2N2psG3HpBStzFZHlJe+gXC6AMNn+CSg4R6b7a1DC2lQR7hmS9sGIVbfW63RXLHp15hAFwsdTeJrwZhxa1S0t6ki7YP1IvklZVQgqcz/NLbPbzyx2F/xoM0TZCUCB0yPgLF5nqDQLPIsL7vb9QcRVrqKGPEcNQ4VdkLTyTZ3dirsMkpdRa0XWoI56n7vfUK3ZpIfcnLe6y3Hy/yCjAbjZkcvjh042g27/dhg1LA+/olAjmMVo5nbtliCDQ== X-MS-TrafficTypeDiagnostic: BL2PR03MB468: X-Microsoft-Exchange-Diagnostics: 1; BL2PR03MB468; 31:kDBmr8+LlSEaKqNu0iM/ctM8TLUy7/1keKTrx2DO31aJ7ZfRveUz4dKWYqcY0eF2K5BmBoU+itiHLBdkunb6t3CumQfEjMECBi7ivrEoq315vt/lkSda5SwPSFWwAaMygfxF6oLuUefW0qOmj+dHVMwLWXrC2YkIixdmEC71Z3K/82PPfVW05gYvvEA/z6hJtmHlr2aMe3RwN7BlaYhtX5ux2CZl9Zp8NY0l+fsjjeg=; 4:q1PwnWQpNvPT/oNZwmjcPrTiajNANJMqhC7zcEJaIzSrQLj2l+r3hdQ7mZR7LCBvPlNaVAe8DDhPHKDp9hCyVWQyMMl+T3qjZ4P9hPvrOsXvYocZ5q/7yHP7nKQSPTNPn3XRE5GDohYU3Zvv1tE0kfB9WjdbhTItqCqbgGn0+ss7321Di3KMKqM57tCScFDAWSD3DggwuT2Oyjmlm1IEomZ3Sv3U8hFkWWTxB1113PRi7v5gDd+a28R90pds/437nDAHA7nAA1nCBsJ+7tbOK7mCX3Udsk4NBZYBvwaiCKk8iR5Gv2h/GMW8/bHVILuO X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231023)(2400081)(944501161)(6055026)(6096035)(20161123556025)(20161123559100)(20161123563025)(20161123565025)(201703131430075)(201703131433075)(201703131448075)(201703151042153)(20161123561025)(201708071742011); SRVR:BL2PR03MB468; BCL:0; PCL:0; RULEID:(100000803101)(100110400095)(400006); SRVR:BL2PR03MB468; X-Forefront-PRVS: 05610E64EE X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL2PR03MB468; 23:z+0r0/Edq2xYfCUq/u0KKJ+3irFNaLachetW0XeOjW?= =?us-ascii?Q?ph4xFMos7AJ4zkGXd3SfP5hAV0oyh9yS+hALvUnkmDGBpVEzQtz0bfLEkMRJ?= =?us-ascii?Q?6b2uCrDEfZlQGdheugylrnwqBNO+QzGjgpgTXcIVNRAH8LB08AfyBOFDb2jL?= =?us-ascii?Q?+HvfsUc2Awjdq/Zfy+t0Lm3gC5BFJnNOJRk9NnAblcoQKgCkAmRLwqt8dKiA?= =?us-ascii?Q?qH4ozzlXGLKDXYKFRHglw/9kZcDjdO7mHVqmD+/DXGgvpzQ9pn9L17ZCT4hu?= =?us-ascii?Q?6xtXHT+v02QCPkn1hspbQX/txpNJdVswJZLKxp5O6/1uDO+D3XLcqRAPaS5q?= =?us-ascii?Q?VANxNu6JGam4wcOrgCfUvOmMzc9kHi7y9Pj94ZdQAbjFPsU+rSjWbecJzumG?= =?us-ascii?Q?woLHMpTTzSkc5ETVecRLOGEYZusjjygT6ZD52PJaSSagOgA5bKUR4sVxS9e3?= =?us-ascii?Q?/8HBC2BkVnK0EGFAq/oARreIXzK2yZVWP/O1ST5Ef0m7urhK3JDOjXLk04fM?= =?us-ascii?Q?dEQI/o/zRZv39SlYMLZqpurlRoy4KJaJsRjZwLXr6w/a7o7pjJYyUPQKJThR?= =?us-ascii?Q?G7hFNcVuw2YQGOW80GTUAzbck7vpt3NpmtW54+0m0bNV45xg0QMN9VWKZL+d?= =?us-ascii?Q?VTnJ9lUxPurHtOa1OTPG7K9a6oeDs/GILX3AQtIAJVRKgOp9lS/GCvxLR9XV?= =?us-ascii?Q?ASrSDANPMfKbm28xhQ5NAxVKgQChp32bukputjZ1hP0BQjomj1O4unbW/SOf?= =?us-ascii?Q?eQIJbk/SmSNFOJV87CCOPVk4BVSWyeTota2X6EGcDOCDPl9Y/vRXv43eI2oB?= =?us-ascii?Q?/AYNm5xJ84pgPdJsoAn4T6KnW/w3JyNIIKiCMCJo1DaOvvYpXQXghNhxIpkb?= =?us-ascii?Q?OYlAmWkEuwskwXRt5RCLpId0RpyTR6b6yGvBeoSMgAFW7PH8hznLjlpwNYwR?= =?us-ascii?Q?1rBHIF5+2oH4mZ1hUn+T1bf6Yw6XKToqoCwoK2+6K9WAQDatsAddY8AJ1zpn?= =?us-ascii?Q?pmx8hYHE/ot082LR6CurN91TgG87hYemlRPxMWECzMr1zA/qvX8HOUCQvVE1?= =?us-ascii?Q?qpNH5Fz6hK39Iwg4RJUFYi1DMz57gtSNL9H2UnJ9qGxxu3Rz4NrsFDXUnkTm?= =?us-ascii?Q?PXRekUrug=3D?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR03MB468; 6:tPK3BrJsfGgQC7LkG6UIaZzlmoukk3E7cyIXT4uZcI36Zudoed5bai5elAe9AVXm+Dz7ZmEBBOmZryV1hhrrbxlDOzLu7vEofVy2NqC2YcHUWy/JiNLw9vQUWzVZaMH3YZUyy8WZtSBbb5TEfxasGprNBH/cYZC4V0ZX1Ek7sf2tHlR62EbR66XMBAFbHVVrH6zwttoNtjTtv320HAg4m7oDY/aGe000Mpg/pkeDRDrvb2mc4kgAx2Ns1nHXqLVzt4Vl4IjVvQyk/27K2x/WmjbwbUyXjSCKqOuznIVZ+25PjvyZfm3QmEhHaBPD1yYUbZTE3rNnmka1btEB8kyJqvQBzKaDigtLOPFwb4YSlyM=; 5:ty4bnzyByesj9K4NSxlbxL+3KJJ17M4Bkt26aEeFhObxtfeayhvBQm4tN+Rao0jWFSyqCFR0ZRbCdZzg37h5gi7jlNqEm+pm8J8TbGVe9kMPgABWFL2zamSjlQDvjxRJfm+/kFQvjBgG7OayckTO2RMbSJ6OSF38WvQGNX0i1j8=; 24:RjK63VfAJKgWPrRXDWN4i/az7BDtXQStIh4tZrjCCR64CqGGlTrhlb6DK9HwCNgKY0J+YqNLTEtYXl0jb8AbWI7vgnlKt2hZ1aW0c6RFq8w=; 7:T4tTX0niNXSwQgSf8g1SBH2loxouvvG8qK0QtKOdvk4HtXfOblcQNs38VRn+86Oa/FZhlDuCrke6gwcsCPC3P0dD4F2XiZtZQDSbjUXVsOw7hqIvZcxpA7tGL9g5IuE6EI3rbMKHHtkwTrIlfcSzfbOzJ/OenXfA0XCL/ccjTv6gX2NAdiqhrgPwS5O/Z/EtVVc07ywb87nz51/UM+J6YRo4/ud2pRZ1hfyePs4v+pE3U8g4rkuIXpoCM6MwGPkY SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2018 13:03:19.2444 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 726f8046-a5be-4ba1-e4f3-08d56261ace1 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.158.2]; Helo=[az84smr01.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR03MB468 Subject: [dpdk-dev] [PATCH 7/7 v2] net/dpaa: further push mode optimizations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jan 2018 13:03:22 -0000 This patch supports batch processing of multiple packets in the Rx side Signed-off-by: Nipun Gupta Signed-off-by: Hemant Agrawal --- drivers/bus/dpaa/base/qbman/qman.c | 89 ++++++++++++++++++------------------- drivers/bus/dpaa/include/fsl_qman.h | 10 +++++ drivers/net/dpaa/dpaa_ethdev.c | 9 +++- drivers/net/dpaa/dpaa_rxtx.c | 81 +++++++++++++++++++++++++++++---- drivers/net/dpaa/dpaa_rxtx.h | 9 ++-- 5 files changed, 137 insertions(+), 61 deletions(-) diff --git a/drivers/bus/dpaa/base/qbman/qman.c b/drivers/bus/dpaa/base/qbman/qman.c index 4d8bdae..2b97671 100644 --- a/drivers/bus/dpaa/base/qbman/qman.c +++ b/drivers/bus/dpaa/base/qbman/qman.c @@ -1055,64 +1055,63 @@ unsigned int qman_portal_poll_rx(unsigned int poll_limit, void **bufs, struct qman_portal *p) { - const struct qm_dqrr_entry *dq; - struct qman_fq *fq; - enum qman_cb_dqrr_result res; - unsigned int limit = 0; -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - struct qm_dqrr_entry *shadow; -#endif - unsigned int rx_number = 0; + struct qm_portal *portal = &p->p; + register struct qm_dqrr *dqrr = &portal->dqrr; + struct qm_dqrr_entry *dq[QM_DQRR_SIZE], *shadow[QM_DQRR_SIZE]; + struct qman_fq *fq[QM_DQRR_SIZE]; + unsigned int limit = 0, rx_number = 0; + uint32_t consume = 0; do { qm_dqrr_pvb_update(&p->p); - dq = qm_dqrr_current(&p->p); - if (unlikely(!dq)) + if (!dqrr->fill) break; + + dq[rx_number] = dqrr->cursor; + dqrr->cursor = DQRR_CARRYCLEAR(dqrr->cursor + 1); + /* Prefetch the next DQRR entry */ + rte_prefetch0(dqrr->cursor); + #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - /* If running on an LE system the fields of the - * dequeue entry must be swapper. Because the - * QMan HW will ignore writes the DQRR entry is - * copied and the index stored within the copy - */ - shadow = &p->shadow_dqrr[DQRR_PTR2IDX(dq)]; - *shadow = *dq; - dq = shadow; - shadow->fqid = be32_to_cpu(shadow->fqid); - shadow->contextB = be32_to_cpu(shadow->contextB); - shadow->seqnum = be16_to_cpu(shadow->seqnum); - hw_fd_to_cpu(&shadow->fd); + /* If running on an LE system the fields of the + * dequeue entry must be swapper. Because the + * QMan HW will ignore writes the DQRR entry is + * copied and the index stored within the copy + */ + shadow[rx_number] = + &p->shadow_dqrr[DQRR_PTR2IDX(dq[rx_number])]; + shadow[rx_number]->fd.opaque_addr = + dq[rx_number]->fd.opaque_addr; + shadow[rx_number]->fd.addr = + be40_to_cpu(dq[rx_number]->fd.addr); + shadow[rx_number]->fd.opaque = + be32_to_cpu(dq[rx_number]->fd.opaque); +#else + shadow = dq; #endif /* SDQCR: context_b points to the FQ */ #ifdef CONFIG_FSL_QMAN_FQ_LOOKUP - fq = get_fq_table_entry(dq->contextB); + fq[rx_number] = qman_fq_lookup_table[be32_to_cpu( + dq[rx_number]->contextB)]; #else - fq = (void *)(uintptr_t)dq->contextB; + fq[rx_number] = (void *)(uintptr_t)be32_to_cpu(dq->contextB); #endif - /* Now let the callback do its stuff */ - res = fq->cb.dqrr_dpdk_cb(NULL, p, fq, dq, &bufs[rx_number]); + fq[rx_number]->cb.dqrr_prepare(shadow[rx_number], + &bufs[rx_number]); + + consume |= (1 << (31 - DQRR_PTR2IDX(shadow[rx_number]))); rx_number++; - /* Interpret 'dq' from a driver perspective. */ - /* - * Parking isn't possible unless HELDACTIVE was set. NB, - * FORCEELIGIBLE implies HELDACTIVE, so we only need to - * check for HELDACTIVE to cover both. - */ - DPAA_ASSERT((dq->stat & QM_DQRR_STAT_FQ_HELDACTIVE) || - (res != qman_cb_dqrr_park)); - qm_dqrr_cdc_consume_1ptr(&p->p, dq, res == qman_cb_dqrr_park); - /* Move forward */ - qm_dqrr_next(&p->p); - /* - * Entry processed and consumed, increment our counter. The - * callback can request that we exit after consuming the - * entry, and we also exit if we reach our processing limit, - * so loop back only if neither of these conditions is met. - */ - } while (likely(++limit < poll_limit)); + --dqrr->fill; + } while (++limit < poll_limit); - return limit; + if (rx_number) + fq[0]->cb.dqrr_dpdk_pull_cb(fq, shadow, bufs, rx_number); + + /* Consume all the DQRR enries together */ + qm_out(DQRR_DCAP, (1 << 8) | consume); + + return rx_number; } u32 qman_portal_dequeue(struct rte_event ev[], unsigned int poll_limit, diff --git a/drivers/bus/dpaa/include/fsl_qman.h b/drivers/bus/dpaa/include/fsl_qman.h index 99e46e1..e9793f3 100644 --- a/drivers/bus/dpaa/include/fsl_qman.h +++ b/drivers/bus/dpaa/include/fsl_qman.h @@ -1131,6 +1131,14 @@ typedef enum qman_cb_dqrr_result (*qman_dpdk_cb_dqrr)(void *event, const struct qm_dqrr_entry *dqrr, void **bd); +/* This callback type is used when handling buffers in dpdk pull mode */ +typedef void (*qman_dpdk_pull_cb_dqrr)(struct qman_fq **fq, + struct qm_dqrr_entry **dqrr, + void **bufs, + int num_bufs); + +typedef void (*qman_dpdk_cb_prepare)(struct qm_dqrr_entry *dq, void **bufs); + /* * This callback type is used when handling ERNs, FQRNs and FQRLs via MR. They * are always consumed after the callback returns. @@ -1191,8 +1199,10 @@ enum qman_fq_state { struct qman_fq_cb { union { /* for dequeued frames */ qman_dpdk_cb_dqrr dqrr_dpdk_cb; + qman_dpdk_pull_cb_dqrr dqrr_dpdk_pull_cb; qman_cb_dqrr dqrr; }; + qman_dpdk_cb_prepare dqrr_prepare; qman_cb_mr ern; /* for s/w ERNs */ qman_cb_mr fqs; /* frame-queue state changes*/ }; diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c index b60ed3b..97679eb 100644 --- a/drivers/net/dpaa/dpaa_ethdev.c +++ b/drivers/net/dpaa/dpaa_ethdev.c @@ -503,7 +503,11 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, QM_FQCTRL_CTXASTASHING | QM_FQCTRL_PREFERINCACHE; opts.fqd.context_a.stashing.exclusive = 0; - opts.fqd.context_a.stashing.annotation_cl = + /* In muticore scenario stashing becomes a bottleneck on LS1046. + * So do not enable stashing in this case + */ + if (dpaa_svr_family != SVR_LS1046A_FAMILY) + opts.fqd.context_a.stashing.annotation_cl = DPAA_IF_RX_ANNOTATION_STASH; opts.fqd.context_a.stashing.data_cl = DPAA_IF_RX_DATA_STASH; opts.fqd.context_a.stashing.context_cl = @@ -526,7 +530,8 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (ret) DPAA_PMD_ERR("Channel/Queue association failed. fqid %d" " ret: %d", rxq->fqid, ret); - rxq->cb.dqrr_dpdk_cb = dpaa_rx_cb; + rxq->cb.dqrr_dpdk_pull_cb = dpaa_rx_cb; + rxq->cb.dqrr_prepare = dpaa_rx_cb_prepare; rxq->is_static = true; } dev->data->rx_queues[queue_idx] = rxq; diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c index f969ccf..fc8144f 100644 --- a/drivers/net/dpaa/dpaa_rxtx.c +++ b/drivers/net/dpaa/dpaa_rxtx.c @@ -399,17 +399,80 @@ struct rte_mbuf * return mbuf; } -enum qman_cb_dqrr_result dpaa_rx_cb(void *event __always_unused, - struct qman_portal *qm __always_unused, - struct qman_fq *fq, - const struct qm_dqrr_entry *dqrr, - void **bufs) +void +dpaa_rx_cb(struct qman_fq **fq, struct qm_dqrr_entry **dqrr, + void **bufs, int num_bufs) { - const struct qm_fd *fd = &dqrr->fd; + struct rte_mbuf *mbuf; + struct dpaa_bp_info *bp_info; + const struct qm_fd *fd; + void *ptr; + struct dpaa_if *dpaa_intf; + uint16_t offset, i; + uint32_t length; + uint8_t format; + + if (dpaa_svr_family != SVR_LS1046A_FAMILY) { + bp_info = DPAA_BPID_TO_POOL_INFO(dqrr[0]->fd.bpid); + ptr = rte_dpaa_mem_ptov(qm_fd_addr(&dqrr[0]->fd)); + rte_prefetch0((void *)((uint8_t *)ptr + DEFAULT_RX_ICEOF)); + bufs[0] = (struct rte_mbuf *)((char *)ptr - + bp_info->meta_data_size); + } - *bufs = dpaa_eth_fd_to_mbuf(fd, - ((struct dpaa_if *)fq->dpaa_intf)->ifid); - return qman_cb_dqrr_consume; + for (i = 0; i < num_bufs; i++) { + if (dpaa_svr_family != SVR_LS1046A_FAMILY && + i < num_bufs - 1) { + bp_info = DPAA_BPID_TO_POOL_INFO(dqrr[i + 1]->fd.bpid); + ptr = rte_dpaa_mem_ptov(qm_fd_addr(&dqrr[i + 1]->fd)); + rte_prefetch0((void *)((uint8_t *)ptr + + DEFAULT_RX_ICEOF)); + bufs[i + 1] = (struct rte_mbuf *)((char *)ptr - + bp_info->meta_data_size); + } + + fd = &dqrr[i]->fd; + dpaa_intf = fq[i]->dpaa_intf; + + format = (fd->opaque & DPAA_FD_FORMAT_MASK) >> + DPAA_FD_FORMAT_SHIFT; + if (unlikely(format == qm_fd_sg)) { + bufs[i] = dpaa_eth_sg_to_mbuf(fd, dpaa_intf->ifid); + continue; + } + + offset = (fd->opaque & DPAA_FD_OFFSET_MASK) >> + DPAA_FD_OFFSET_SHIFT; + length = fd->opaque & DPAA_FD_LENGTH_MASK; + + mbuf = bufs[i]; + mbuf->data_off = offset; + mbuf->data_len = length; + mbuf->pkt_len = length; + mbuf->port = dpaa_intf->ifid; + + mbuf->nb_segs = 1; + mbuf->ol_flags = 0; + mbuf->next = NULL; + rte_mbuf_refcnt_set(mbuf, 1); + dpaa_eth_packet_info(mbuf, (uint64_t)mbuf->buf_addr); + } +} + +void dpaa_rx_cb_prepare(struct qm_dqrr_entry *dq, void **bufs) +{ + struct dpaa_bp_info *bp_info = DPAA_BPID_TO_POOL_INFO(dq->fd.bpid); + void *ptr = rte_dpaa_mem_ptov(qm_fd_addr(&dq->fd)); + + /* In case of LS1046, annotation stashing is disabled due to L2 cache + * being bottleneck in case of multicore scanario for this platform. + * So we prefetch the annoation beforehand, so that it is available + * in cache when accessed. + */ + if (dpaa_svr_family == SVR_LS1046A_FAMILY) + rte_prefetch0((void *)((uint8_t *)ptr + DEFAULT_RX_ICEOF)); + + *bufs = (struct rte_mbuf *)((char *)ptr - bp_info->meta_data_size); } static uint16_t diff --git a/drivers/net/dpaa/dpaa_rxtx.h b/drivers/net/dpaa/dpaa_rxtx.h index 29d8f95..d3e6351 100644 --- a/drivers/net/dpaa/dpaa_rxtx.h +++ b/drivers/net/dpaa/dpaa_rxtx.h @@ -268,9 +268,8 @@ int dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf, struct qm_fd *fd, uint32_t bpid); -enum qman_cb_dqrr_result dpaa_rx_cb(void *event, - struct qman_portal *qm, - struct qman_fq *fq, - const struct qm_dqrr_entry *dqrr, - void **bd); +void dpaa_rx_cb(struct qman_fq **fq, + struct qm_dqrr_entry **dqrr, void **bufs, int num_bufs); + +void dpaa_rx_cb_prepare(struct qm_dqrr_entry *dq, void **bufs); #endif -- 1.9.1