From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0043.outbound.protection.outlook.com [104.47.37.43]) by dpdk.org (Postfix) with ESMTP id 0318E397D for ; Tue, 23 Jan 2018 08:24:55 +0100 (CET) Received: from BLUPR0301CA0030.namprd03.prod.outlook.com (10.162.113.168) by BY2PR03MB476.namprd03.prod.outlook.com (10.141.141.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.428.17; Tue, 23 Jan 2018 07:24:54 +0000 Received: from BY2FFO11FD039.protection.gbl (2a01:111:f400:7c0c::151) by BLUPR0301CA0030.outlook.office365.com (2a01:111:e400:5259::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.428.17 via Frontend Transport; Tue, 23 Jan 2018 07:24:53 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD039.mail.protection.outlook.com (10.1.14.224) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.20.428.12 via Frontend Transport; Tue, 23 Jan 2018 07:24:50 +0000 Received: from b27504-OptiPlex-790.ap.freescale.net (b27504-OptiPlex-790.ap.freescale.net [10.232.132.60]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id w0N6G0e1023569; Mon, 22 Jan 2018 23:16:26 -0700 From: Nipun Gupta To: CC: , , , Nipun Gupta Date: Tue, 23 Jan 2018 18:01:11 +0530 Message-ID: <1516710671-13897-8-git-send-email-nipun.gupta@nxp.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1516710671-13897-1-git-send-email-nipun.gupta@nxp.com> References: <1516710671-13897-1-git-send-email-nipun.gupta@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131611658923759093; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(376002)(39380400002)(39860400002)(396003)(2980300002)(1110001)(1109001)(339900001)(199004)(189003)(68736007)(86362001)(47776003)(105606002)(2906002)(54906003)(106466001)(53936002)(2351001)(81166006)(16586007)(97736004)(8676002)(2950100002)(6666003)(5660300001)(356003)(81156014)(498600001)(8936002)(26005)(50226002)(36756003)(316002)(50466002)(48376002)(6862004)(77096007)(51416003)(59450400001)(336011)(104016004)(305945005)(76176011)(4326008)(85426001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR03MB476; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD039; 1:7LjqW0Ry2vETuYg2+dbpZePEmKCJ6OmZWZuLVw6NI89rX13rlTUTOx2rU4dAp3Uarr5HLfKv6YW6t5nrxLs1W1N5E8nr+ooXMazi16i0REHKhVBes2BK/BE1dN0O1v0W MIME-Version: 1.0 Content-Type: text/plain X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1b4d41d1-8f02-45c3-4aed-08d5623264d7 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(5600026)(4604075)(2017052603307); SRVR:BY2PR03MB476; X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB476; 3:m5yJHWdToJ2+iQiIsN5wGzKcFj3f2x1O4fvzOU8jD+580An5N1oBvtZSgN4HjxPbGcEFBsyzlY27cbnFonfiQg+fkqiEgJOc18f/+dm6C1ZVVBKfNHLOEdMoTkOC4JgNHLgKt1lUnhLvl3UNktSoeF+tkz30mGY84k38oZaD1w4iurqCqhD6jWSo17D/mN14dxkSCcsbGv1/OaZJ6AcMBZLuOxY3nYQviJL6PpVJNXAlQ+ysvrVKhVuRwTVJ6WQEW4PEZtrwew4dkqhUlyx1is+KqMqSj+jPm6BHpPiE3xmpspVqrvymsUnuHvqv4gyqyNbARHSl8PAdUO2U14RL73Bu++yQlgJpg2gpyQqsF/Y=; 25:IYJ1UJUJGpcWCBGNzQhxL29gmdO+B+sI/QheK35He5YvwZuTNEfAmP1STBPNY8RbSoMdCXI3PVZ533gpp6+ebulmzgX/q3D6bpfVwaM1SOwwj0qL8VvsoJqsNdi9jVltrazTyd2KRJtqqaq59iPdbmyYIJyo5FAG8AWwJ3Nr9vbhgaXfs2Y/wVuDQSJ4edNAn0+MlTHuX/Z8m77y1kDXLZ+0TiohhSFcIUI0M9bMSzz7BO7ct4Jm1CtulU7fD2RBDJL9W4xurHikrJmZsp8aN4aWZArOBAMIOFTI888REkTfX53t2Wm10MfKh0zs7Y4SZdTJzgcKnUQsSH+tFHAd2g== X-MS-TrafficTypeDiagnostic: BY2PR03MB476: X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB476; 31:2ys/Dqih2nV2PGft/LexUbIXUxRlxpWddm3f8GAbNJ0tYcvH4wnXk5nmB3vCVti6ySInu8WJutTxRLMBmMe9eKdRthupJqSdA0k3irPavo+Pg2mtCnE8gMt0lTkwcXQfIesOPA4rUIk9oQ/tkAOoULEECNDztbNNTstMcReOpzPMD68C9dpMITknPPiF5RzEDgpmpcwzFO7egj6nX/4Xf8eoX5B1NywbXlgm+T76aYw=; 4:qlYuEINNo3uooKoxqUUIv/+3cHkdgd8w9hfJjyMmIL34eBZ/N0oginQZlNKnd6ND8VY8zabxrkXcTOgxnxIDKMurp2nBCPcf+iJPPXfYXKEN3QQLBLGE87xr16XHVsrcHjdvKct7PT82WJ2T/sh/X1YxehqtfWEy+5tlh3JcpbQMUSj8wauxedOH1Joy5MBtcQZXzq5Vo8uXKgQDjBn+6iDjY27xu8Xg6dQaS2deE53ETEUjLFNyt52Twfy1lQlF0llae6PvocX8Ht9EkU7VfA4X4G25RB6c1ipfHT55QYJsVI5sgxZ5Qk0wd3M2+MTx X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095135)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(93001095)(3231023)(2400081)(944501161)(6055026)(6096035)(20161123556025)(20161123559100)(201703131430075)(201703131433075)(201703131448075)(201703151042153)(20161123561025)(20161123563025)(20161123565025)(201708071742011); SRVR:BY2PR03MB476; BCL:0; PCL:0; RULEID:(100000803101)(100110400095)(400006); SRVR:BY2PR03MB476; X-Forefront-PRVS: 05610E64EE X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR03MB476; 23:lHFe3xOTOhYrbCpUFLwEnZPn6Vbr2q1XXp/d2o6fgH?= =?us-ascii?Q?Kr5618DXObnpuM7LvIcadJDfgmVnIxdWkrQnuD9+TnhdbotxJoIhn/Z8TMiL?= =?us-ascii?Q?tH0Z97gKs6LmHKPARMLdikUYEHDSf0qCZQoHxtRgZ3NBhbzXsjlaPchEENuP?= =?us-ascii?Q?APZqY+El/e3y80N6fPwcFh/SyZwNrstzejB3gPpBCm3ACN70uuw4d4VQD1UP?= =?us-ascii?Q?X+KCxhyN/NNDLWlHWsR9DJgfjROi4CFil8nZeRyqe6cu5tiU1b4s+Bh9GXKN?= =?us-ascii?Q?80NiLf6tlW/mfFnVtARam/55BHnJTT3wO+v10FnQNOXn1A/bP7fzbN9n5LOs?= =?us-ascii?Q?cwGaIpTUUb8ZkeW/TECGqEn07mYS7061Oa/zorv0wYmlkGhl/wRaPeMWobS8?= =?us-ascii?Q?0N6Ox869tIuAADyyrIwUSFF3eNYlkQ8yfucJQfQEgpbyc28PpRcZ2PqpzTw9?= =?us-ascii?Q?BV2Asc6gEiD46bSIWjBDmb0gOPxy3+pkVSeRynN49PLW3NdvaQR+Pazx4KMz?= =?us-ascii?Q?zOCT1X9GiSwNGgHzbyJB2Cv/Ed0CKWGY+pdr4PYQnft/vHqKLFVTJzPOg+pr?= =?us-ascii?Q?RVSAR5jfsqcXn3jLRoSukmZauCeAg3aMC2+inQwS1XnJYlHm23MsKnXw+8Sp?= =?us-ascii?Q?e5vUW3ycoTWPDT0qmAK0lg7V1hvqDqUL8oP1noMXrvUaeq5UYCEBAuZDi1pp?= =?us-ascii?Q?+yUNRYfutEidgGCcHvieg1jjYj0iR5yxfLm92CO6ckgHCC3JqTMZC1whjkO2?= =?us-ascii?Q?NYS04ocJicrQK4rOam+o+jqrRmN0H44hPOLVuIO0eeCP1U+nZI+LTcs7RpHY?= =?us-ascii?Q?U+1v8w8F0Dsl6BreKD1RZB3YkptM+1ng3qGweMNZPBgpZlWh2EcWjlUO1Q/i?= =?us-ascii?Q?FWpVZDwZExSDmmVyRcQ9ypW0yLihxKyZOudBoV+C5k99aniDe56t/5c5LnA4?= =?us-ascii?Q?cGKpzQAvk/k/oiIQJhX9/10fHXgkHmCzyiDlR061NjyIXgekrqkrliJdfO2C?= =?us-ascii?Q?WJI5S/5VQYXtN5EqACKHl43Q5olG4uHwqgvXzW0UF8n6wT/BoVwyyv1vZ7cY?= =?us-ascii?Q?zXyF+7sz3KtWotp1hk2SCIzXuXDmaYXpaBDG59E+CZsjhnlw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB476; 6:3+cN2fRYb0DLm165n2l6MZSiRclM35T3d3NUntXOXHgILhyvwp0XBx1wdmW21U+aFIuU0iZ0uTQ6o+yut3iPSkhK/meqN9qb3sH4ilh0POEUOGi58wTHK82vpG0PX5/TSOEOntrYNrFga3CZFPSpU16EuHuVrB2FBg+2O1ag3cJbRcHIqkTTUO6Gg+vddDH3MWM+maXqIxc5W+tE0eEsboOADbf0ogPK0YNlTQv3GyrRgJ4gXbfw6RzqG2VknTIYvK8eLtzTeOhqjbJcGRsiBpe/zhTNM8GEi4qeloQq74MnFHPASI+avXEm/BJm9v7z824w5CS5JqpSJvwrwr3heEpnWqO8uwsqKAtYiQeHqHk=; 5:R21LpJoYlTsZHT7zobKKOQukj+nRRnW9HOvw7BklM3SiaVnLtKOZKIc//3cMfXWZtBARLQcZTIapDYs+9pLA2lf9hCjyh9a42Ta0df5UO0W6wRprA8bDqAln9Uk505Y9YAuZQojePjFsEy3ghWwFlLCKRRvTPyk5WNwbT7RzUgs=; 24:Yzbm8sgftRwCqMok0Mq+ryI1FByIfEDMy65nyyXsMAJU981I4IhGfkv2U8tNLFPUBdYkn3CAp5wOycVYGru8P6JZQP14Aewi3+VzB3CMgn8=; 7:KCjtSQWI2oPh5ipvFIhbg/BDYZGgQ4Y6L4kYGvZNu7HvQKrqMvUINCbK4yiz+PqudSrWIJIVSkBNsspekQ5WninSaATjGVv3S7Y2msj43ka002kvqYYqTv7h9VcINy/+ZMXWo+1IqJo9f7okkrMXkSOEMU4QorJ60+R3ovMPOr0KV48c/qSWiCnSxLdFFmN++v6GaBTu3pHPvOXkpJGA6K+g1wVrlQ7gRcdJp2DfJoKqiiKPtDqrtvXJP8cX/6nj SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2018 07:24:50.5662 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1b4d41d1-8f02-45c3-4aed-08d5623264d7 X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR03MB476 Subject: [dpdk-dev] [PATCH 7/7] net/dpaa: further push mode optimizations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jan 2018 07:24:56 -0000 This patch supports batch processing of multiple packets in the Rx side Signed-off-by: Nipun Gupta Signed-off-by: Hemant Agrawal --- drivers/bus/dpaa/base/qbman/qman.c | 89 ++++++++++++++++++------------------- drivers/bus/dpaa/include/fsl_qman.h | 10 +++++ drivers/net/dpaa/dpaa_ethdev.c | 9 +++- drivers/net/dpaa/dpaa_rxtx.c | 80 +++++++++++++++++++++++++++++---- drivers/net/dpaa/dpaa_rxtx.h | 9 ++-- 5 files changed, 136 insertions(+), 61 deletions(-) diff --git a/drivers/bus/dpaa/base/qbman/qman.c b/drivers/bus/dpaa/base/qbman/qman.c index 4d8bdae..2b97671 100644 --- a/drivers/bus/dpaa/base/qbman/qman.c +++ b/drivers/bus/dpaa/base/qbman/qman.c @@ -1055,64 +1055,63 @@ unsigned int qman_portal_poll_rx(unsigned int poll_limit, void **bufs, struct qman_portal *p) { - const struct qm_dqrr_entry *dq; - struct qman_fq *fq; - enum qman_cb_dqrr_result res; - unsigned int limit = 0; -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - struct qm_dqrr_entry *shadow; -#endif - unsigned int rx_number = 0; + struct qm_portal *portal = &p->p; + register struct qm_dqrr *dqrr = &portal->dqrr; + struct qm_dqrr_entry *dq[QM_DQRR_SIZE], *shadow[QM_DQRR_SIZE]; + struct qman_fq *fq[QM_DQRR_SIZE]; + unsigned int limit = 0, rx_number = 0; + uint32_t consume = 0; do { qm_dqrr_pvb_update(&p->p); - dq = qm_dqrr_current(&p->p); - if (unlikely(!dq)) + if (!dqrr->fill) break; + + dq[rx_number] = dqrr->cursor; + dqrr->cursor = DQRR_CARRYCLEAR(dqrr->cursor + 1); + /* Prefetch the next DQRR entry */ + rte_prefetch0(dqrr->cursor); + #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - /* If running on an LE system the fields of the - * dequeue entry must be swapper. Because the - * QMan HW will ignore writes the DQRR entry is - * copied and the index stored within the copy - */ - shadow = &p->shadow_dqrr[DQRR_PTR2IDX(dq)]; - *shadow = *dq; - dq = shadow; - shadow->fqid = be32_to_cpu(shadow->fqid); - shadow->contextB = be32_to_cpu(shadow->contextB); - shadow->seqnum = be16_to_cpu(shadow->seqnum); - hw_fd_to_cpu(&shadow->fd); + /* If running on an LE system the fields of the + * dequeue entry must be swapper. Because the + * QMan HW will ignore writes the DQRR entry is + * copied and the index stored within the copy + */ + shadow[rx_number] = + &p->shadow_dqrr[DQRR_PTR2IDX(dq[rx_number])]; + shadow[rx_number]->fd.opaque_addr = + dq[rx_number]->fd.opaque_addr; + shadow[rx_number]->fd.addr = + be40_to_cpu(dq[rx_number]->fd.addr); + shadow[rx_number]->fd.opaque = + be32_to_cpu(dq[rx_number]->fd.opaque); +#else + shadow = dq; #endif /* SDQCR: context_b points to the FQ */ #ifdef CONFIG_FSL_QMAN_FQ_LOOKUP - fq = get_fq_table_entry(dq->contextB); + fq[rx_number] = qman_fq_lookup_table[be32_to_cpu( + dq[rx_number]->contextB)]; #else - fq = (void *)(uintptr_t)dq->contextB; + fq[rx_number] = (void *)(uintptr_t)be32_to_cpu(dq->contextB); #endif - /* Now let the callback do its stuff */ - res = fq->cb.dqrr_dpdk_cb(NULL, p, fq, dq, &bufs[rx_number]); + fq[rx_number]->cb.dqrr_prepare(shadow[rx_number], + &bufs[rx_number]); + + consume |= (1 << (31 - DQRR_PTR2IDX(shadow[rx_number]))); rx_number++; - /* Interpret 'dq' from a driver perspective. */ - /* - * Parking isn't possible unless HELDACTIVE was set. NB, - * FORCEELIGIBLE implies HELDACTIVE, so we only need to - * check for HELDACTIVE to cover both. - */ - DPAA_ASSERT((dq->stat & QM_DQRR_STAT_FQ_HELDACTIVE) || - (res != qman_cb_dqrr_park)); - qm_dqrr_cdc_consume_1ptr(&p->p, dq, res == qman_cb_dqrr_park); - /* Move forward */ - qm_dqrr_next(&p->p); - /* - * Entry processed and consumed, increment our counter. The - * callback can request that we exit after consuming the - * entry, and we also exit if we reach our processing limit, - * so loop back only if neither of these conditions is met. - */ - } while (likely(++limit < poll_limit)); + --dqrr->fill; + } while (++limit < poll_limit); - return limit; + if (rx_number) + fq[0]->cb.dqrr_dpdk_pull_cb(fq, shadow, bufs, rx_number); + + /* Consume all the DQRR enries together */ + qm_out(DQRR_DCAP, (1 << 8) | consume); + + return rx_number; } u32 qman_portal_dequeue(struct rte_event ev[], unsigned int poll_limit, diff --git a/drivers/bus/dpaa/include/fsl_qman.h b/drivers/bus/dpaa/include/fsl_qman.h index 99e46e1..e9793f3 100644 --- a/drivers/bus/dpaa/include/fsl_qman.h +++ b/drivers/bus/dpaa/include/fsl_qman.h @@ -1131,6 +1131,14 @@ typedef enum qman_cb_dqrr_result (*qman_dpdk_cb_dqrr)(void *event, const struct qm_dqrr_entry *dqrr, void **bd); +/* This callback type is used when handling buffers in dpdk pull mode */ +typedef void (*qman_dpdk_pull_cb_dqrr)(struct qman_fq **fq, + struct qm_dqrr_entry **dqrr, + void **bufs, + int num_bufs); + +typedef void (*qman_dpdk_cb_prepare)(struct qm_dqrr_entry *dq, void **bufs); + /* * This callback type is used when handling ERNs, FQRNs and FQRLs via MR. They * are always consumed after the callback returns. @@ -1191,8 +1199,10 @@ enum qman_fq_state { struct qman_fq_cb { union { /* for dequeued frames */ qman_dpdk_cb_dqrr dqrr_dpdk_cb; + qman_dpdk_pull_cb_dqrr dqrr_dpdk_pull_cb; qman_cb_dqrr dqrr; }; + qman_dpdk_cb_prepare dqrr_prepare; qman_cb_mr ern; /* for s/w ERNs */ qman_cb_mr fqs; /* frame-queue state changes*/ }; diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c index b60ed3b..97679eb 100644 --- a/drivers/net/dpaa/dpaa_ethdev.c +++ b/drivers/net/dpaa/dpaa_ethdev.c @@ -503,7 +503,11 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, QM_FQCTRL_CTXASTASHING | QM_FQCTRL_PREFERINCACHE; opts.fqd.context_a.stashing.exclusive = 0; - opts.fqd.context_a.stashing.annotation_cl = + /* In muticore scenario stashing becomes a bottleneck on LS1046. + * So do not enable stashing in this case + */ + if (dpaa_svr_family != SVR_LS1046A_FAMILY) + opts.fqd.context_a.stashing.annotation_cl = DPAA_IF_RX_ANNOTATION_STASH; opts.fqd.context_a.stashing.data_cl = DPAA_IF_RX_DATA_STASH; opts.fqd.context_a.stashing.context_cl = @@ -526,7 +530,8 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, if (ret) DPAA_PMD_ERR("Channel/Queue association failed. fqid %d" " ret: %d", rxq->fqid, ret); - rxq->cb.dqrr_dpdk_cb = dpaa_rx_cb; + rxq->cb.dqrr_dpdk_pull_cb = dpaa_rx_cb; + rxq->cb.dqrr_prepare = dpaa_rx_cb_prepare; rxq->is_static = true; } dev->data->rx_queues[queue_idx] = rxq; diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c index f969ccf..89e7918 100644 --- a/drivers/net/dpaa/dpaa_rxtx.c +++ b/drivers/net/dpaa/dpaa_rxtx.c @@ -399,17 +399,79 @@ struct rte_mbuf * return mbuf; } -enum qman_cb_dqrr_result dpaa_rx_cb(void *event __always_unused, - struct qman_portal *qm __always_unused, - struct qman_fq *fq, - const struct qm_dqrr_entry *dqrr, - void **bufs) +void +dpaa_rx_cb(struct qman_fq **fq, struct qm_dqrr_entry **dqrr, + void **bufs, int num_bufs) { - const struct qm_fd *fd = &dqrr->fd; + struct rte_mbuf *mbuf; + struct dpaa_bp_info *bp_info; + const struct qm_fd *fd; + void *ptr; + struct dpaa_if *dpaa_intf; + uint16_t offset, i; + uint32_t length; + uint8_t format; + + if (dpaa_svr_family != SVR_LS1046A_FAMILY) { + bp_info = DPAA_BPID_TO_POOL_INFO(dqrr[0]->fd.bpid); + ptr = rte_dpaa_mem_ptov(qm_fd_addr(&dqrr[0]->fd)); + rte_prefetch0((void *)((uint8_t *)ptr + DEFAULT_RX_ICEOF)); + bufs[0] = (struct rte_mbuf *)((char *)ptr - + bp_info->meta_data_size); + } - *bufs = dpaa_eth_fd_to_mbuf(fd, - ((struct dpaa_if *)fq->dpaa_intf)->ifid); - return qman_cb_dqrr_consume; + for (i = 0; i < num_bufs; i++) { + if (dpaa_svr_family != SVR_LS1046A_FAMILY && i < num_bufs -1) { + bp_info = DPAA_BPID_TO_POOL_INFO(dqrr[i + 1]->fd.bpid); + ptr = rte_dpaa_mem_ptov(qm_fd_addr(&dqrr[i + 1]->fd)); + rte_prefetch0((void *)((uint8_t *)ptr + + DEFAULT_RX_ICEOF)); + bufs[i + 1] = (struct rte_mbuf *)((char *)ptr - + bp_info->meta_data_size); + } + + fd = &dqrr[i]->fd; + dpaa_intf = fq[i]->dpaa_intf; + + format = (fd->opaque & DPAA_FD_FORMAT_MASK) >> + DPAA_FD_FORMAT_SHIFT; + if (unlikely(format == qm_fd_sg)) { + bufs[i] = dpaa_eth_sg_to_mbuf(fd, dpaa_intf->ifid); + continue; + } + + offset = (fd->opaque & DPAA_FD_OFFSET_MASK) >> + DPAA_FD_OFFSET_SHIFT; + length = fd->opaque & DPAA_FD_LENGTH_MASK; + + mbuf = bufs[i]; + mbuf->data_off = offset; + mbuf->data_len = length; + mbuf->pkt_len = length; + mbuf->port = dpaa_intf->ifid; + + mbuf->nb_segs = 1; + mbuf->ol_flags = 0; + mbuf->next = NULL; + rte_mbuf_refcnt_set(mbuf, 1); + dpaa_eth_packet_info(mbuf, (uint64_t)mbuf->buf_addr); + } +} + +void dpaa_rx_cb_prepare(struct qm_dqrr_entry *dq, void **bufs) +{ + struct dpaa_bp_info *bp_info = DPAA_BPID_TO_POOL_INFO(dq->fd.bpid); + void *ptr = rte_dpaa_mem_ptov(qm_fd_addr(&(dq->fd))); + + /* In case of LS1046, annotation stashing is disabled due to L2 cache + * being bottleneck in case of multicore scanario for this platform. + * So we prefetch the annoation beforehand, so that it is available + * in cache when accessed. + */ + if (dpaa_svr_family == SVR_LS1046A_FAMILY) + rte_prefetch0((void *)((uint8_t *)ptr + DEFAULT_RX_ICEOF)); + + *bufs = (struct rte_mbuf *)((char *)ptr - bp_info->meta_data_size); } static uint16_t diff --git a/drivers/net/dpaa/dpaa_rxtx.h b/drivers/net/dpaa/dpaa_rxtx.h index 29d8f95..d3e6351 100644 --- a/drivers/net/dpaa/dpaa_rxtx.h +++ b/drivers/net/dpaa/dpaa_rxtx.h @@ -268,9 +268,8 @@ int dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf, struct qm_fd *fd, uint32_t bpid); -enum qman_cb_dqrr_result dpaa_rx_cb(void *event, - struct qman_portal *qm, - struct qman_fq *fq, - const struct qm_dqrr_entry *dqrr, - void **bd); +void dpaa_rx_cb(struct qman_fq **fq, + struct qm_dqrr_entry **dqrr, void **bufs, int num_bufs); + +void dpaa_rx_cb_prepare(struct qm_dqrr_entry *dq, void **bufs); #endif -- 1.9.1