From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09CC1A04C1 for ; Fri, 22 Nov 2019 07:50:18 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C281D58C3; Fri, 22 Nov 2019 07:50:17 +0100 (CET) Received: from NAM05-DM3-obe.outbound.protection.outlook.com (mail-eopbgr730048.outbound.protection.outlook.com [40.107.73.48]) by dpdk.org (Postfix) with ESMTP id B17EC2BF9; Fri, 22 Nov 2019 07:50:14 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=enBK7k0LVhYwXzsj5S9EXZ4nhO89ZPYKJ/Jf87IPTd+CLaZ3KnFgNXKt2v7a0vKSy7AhRdndHzCQcwM5VzZKmWuBeC4PDj7kyqVTW2BMPmPPEprO7wk5KU3W9C0KvLIh4B+eIlxaApFw15m6dm2bFIuzwVcbz03iW0o76fMPJkx1xaiqaENCyGtxZGgzZxhRA3OYuMk5Ml9iMiektawXf7ayyevMXM1NdPNzJUKEp1irzcsS16ZboPfWwgW03e1TeJmSHosHH0+CycLUn0AejMZ8xlmwtJgs33gTLhxbiIIkad8A/4filBPneFd1luDw0Oe2GA899ehaJ7y7RPrX/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sfEprZooloyjA6GRqGOKdne8aYscgsQ3XUOqiXg0sgw=; b=Lcmnwb8zkWGzWEHYFQ9UH7hs3qmcy0UstOdS7W6GKEpjz7vSf1mOvkWcHh75WzxqKuz9pFvoEwVyPZJtPfqYJtFcqEdr+cRVanackhgDNDO1Mhu4iTRCRiXUgv/oE9fg6rikBki3bfv4WGja2Lkkit3Gx1MuS1O8loyDrH3+D1nDDJiaD8RaK0DmBDuu+YaJSFVy5/gx/khqMDBbHYKuE5/Kwv88vpsUt/kHZlolGH7psP9dVlmLLj5G6Ys0FjP5t279jVvfj3pPPlG6wj5KOlXMNlO21wkqHmbJw1LDjG5XEmqOqY340FTYS8LdETGOL5kLpwK3bL8gqsZsK6/l1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sfEprZooloyjA6GRqGOKdne8aYscgsQ3XUOqiXg0sgw=; b=hS5L6+lIJJ3krxPUpDEZmMp5TdytG345MgNoR5hIZ39eQ2VjbQDNlQ9caMx6IABJ1XQ4HK1cZZ/guyh6RBlxM/Lb0Uxi3/velb4jLl9hSPa4zRU7K3SdX20DaJJiSt/dK1XFkAfUW5klZMgiIO2kZKffHpoWUROsxaPd7FjoOGA= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Amaranath.Somalapuram@amd.com; Received: from DM5PR1201MB2474.namprd12.prod.outlook.com (10.172.87.136) by DM5PR1201MB2473.namprd12.prod.outlook.com (10.172.87.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2474.17; Fri, 22 Nov 2019 06:50:13 +0000 Received: from DM5PR1201MB2474.namprd12.prod.outlook.com ([fe80::3da8:4a0d:72f9:d41d]) by DM5PR1201MB2474.namprd12.prod.outlook.com ([fe80::3da8:4a0d:72f9:d41d%9]) with mapi id 15.20.2451.031; Fri, 22 Nov 2019 06:50:13 +0000 From: asomalap@amd.com To: dev@dpdk.org Cc: stable@dpdk.org Date: Fri, 22 Nov 2019 12:17:17 +0530 Message-Id: <20191122064717.94223-1-asomalap@amd.com> X-Mailer: git-send-email 2.17.1 Content-Type: text/plain X-ClientProxiedBy: MAXPR01CA0112.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::30) To DM5PR1201MB2474.namprd12.prod.outlook.com (2603:10b6:3:e3::8) MIME-Version: 1.0 X-Mailer: git-send-email 2.17.1 X-Originating-IP: [165.204.157.251] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 5ba81450-a263-46c4-a4ed-08d76f183997 X-MS-TrafficTypeDiagnostic: DM5PR1201MB2473: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-Forefront-PRVS: 02296943FF X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(376002)(396003)(39860400002)(136003)(366004)(199004)(189003)(16586007)(316002)(305945005)(81156014)(99286004)(8676002)(7736002)(2361001)(2351001)(2906002)(2616005)(6512007)(3846002)(4326008)(81166006)(450100002)(9686003)(36756003)(6436002)(66066001)(478600001)(47776003)(6116002)(48376002)(6916009)(66476007)(66946007)(186003)(50466002)(66556008)(6506007)(386003)(51416003)(52116002)(1076003)(14454004)(6486002)(5660300002)(8936002)(50226002)(25786009)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR1201MB2473; H:DM5PR1201MB2474.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PcbRAgVDALHjMDwjiYTUO8rBgMmHBffzcNKtdWd/wG/a2rNV+QoSJnvk2egx945uKvwNDzEim561O7QNhashJCfxw1uRchCMfr1jy1yYcQHnrokqcbsmtjWPjerG36fzI+DzQqx7MoA/05cmZ2gzD2hprkicLfplN4tyzNd3dCTwqdhZbf8d9grmNBEY7mtcaM/W/PBrGQ7jWAwhcKgJqMZEZjzM4r3YmQvHSzl3iEEGncRj8Eqhxbl1YZTumXyc1S6H3Vb/4o1cTFBvPby5jXE5uxY5n4H3edybh7M4Ujb3chgoQesQm6HIzxoZew+bJu8o0MKrQxqrEMILYyn8y+utceDHKaZs3tU6YIkkhmEpxDs4yxBkduMVmS7+ELyBdY5mWj87MQlpforxgQGQGhBQj3TCTRliySwZ80NrD0jo1YBNJ4CQ/eopIbgJ5XA/ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ba81450-a263-46c4-a4ed-08d76f183997 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2019 06:50:13.4441 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mmSqtl6OW5mmiz5GS0funfPXHnrSqKMd2ayAHpy1KJ4hktHqBd25n3HFG/6RkK3svoZJEV+01Znmd+Ja9H31eQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB2473 Subject: [dpdk-stable] [PATCH v2 4/5] crypto/ccp: fix support scheduling CCP's on burst X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" From: Amaranath Somalapuram CCP driver was scheduling only one CCP in a single burst(enqueue). Effective throughput was limited to 1 CCP performance. Scheduling multiple ccp within one burst will increase the ccp performance. this changes will divide the enqueue packets equally among the multiple CCP Fixes: e0d88a394e ("crypto/ccp: support run-time CPU based auth") Cc: stable@dpdk.org Signed-off-by: Amaranath Somalapuram --- drivers/crypto/ccp/ccp_crypto.c | 22 +++++++---- drivers/crypto/ccp/ccp_crypto.h | 7 +++- drivers/crypto/ccp/ccp_dev.h | 2 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 + drivers/crypto/ccp/rte_ccp_pmd.c | 57 +++++++++++++++++++--------- 5 files changed, 63 insertions(+), 27 deletions(-) diff --git a/drivers/crypto/ccp/ccp_crypto.c b/drivers/crypto/ccp/ccp_crypto.c index 1837c8543..4256734d1 100644 --- a/drivers/crypto/ccp/ccp_crypto.c +++ b/drivers/crypto/ccp/ccp_crypto.c @@ -2680,7 +2680,9 @@ process_ops_to_enqueue(struct ccp_qp *qp, struct rte_crypto_op **op, struct ccp_queue *cmd_q, uint16_t nb_ops, - int slots_req) + uint16_t total_nb_ops, + int slots_req, + uint16_t b_idx) { int i, result = 0; struct ccp_batch_info *b_info; @@ -2701,6 +2703,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, /* populate batch info necessary for dequeue */ b_info->op_idx = 0; + b_info->b_idx = 0; b_info->lsb_buf_idx = 0; b_info->desccnt = 0; b_info->cmd_q = cmd_q; @@ -2710,7 +2713,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, b_info->head_offset = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); - for (i = 0; i < nb_ops; i++) { + for (i = b_idx; i < (nb_ops+b_idx); i++) { session = (struct ccp_session *)get_sym_session_private_data( op[i]->sym->session, ccp_cryptodev_driver_id); @@ -2762,6 +2765,8 @@ process_ops_to_enqueue(struct ccp_qp *qp, } b_info->opcnt = i; + b_info->b_idx = b_idx; + b_info->total_nb_ops = total_nb_ops; b_info->tail_offset = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); @@ -2776,7 +2781,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, rte_ring_enqueue(qp->processed_pkts, (void *)b_info); EVP_MD_CTX_destroy(auth_ctx); - return i; + return i-b_idx; } static inline void ccp_auth_dq_prepare(struct rte_crypto_op *op) @@ -2861,8 +2866,8 @@ ccp_prepare_ops(struct ccp_qp *qp, } min_ops = RTE_MIN(nb_ops, b_info->opcnt); - for (i = 0; i < min_ops; i++) { - op_d[i] = b_info->op[b_info->op_idx++]; + for (i = b_info->b_idx; i < min_ops; i++) { + op_d[i] = b_info->op[b_info->b_idx + b_info->op_idx++]; session = (struct ccp_session *)get_sym_session_private_data( op_d[i]->sym->session, ccp_cryptodev_driver_id); @@ -2903,7 +2908,8 @@ ccp_prepare_ops(struct ccp_qp *qp, int process_ops_to_dequeue(struct ccp_qp *qp, struct rte_crypto_op **op, - uint16_t nb_ops) + uint16_t nb_ops, + uint16_t *total_nb_ops) { struct ccp_batch_info *b_info; uint32_t cur_head_offset; @@ -2918,6 +2924,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, if (b_info->auth_ctr == b_info->opcnt) goto success; + *total_nb_ops = b_info->total_nb_ops; cur_head_offset = CCP_READ_REG(b_info->cmd_q->reg_base, CMD_Q_HEAD_LO_BASE); @@ -2927,7 +2934,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, qp->b_info = b_info; return 0; } - } else { + } else if (b_info->tail_offset != b_info->head_offset) { if ((cur_head_offset >= b_info->head_offset) || (cur_head_offset < b_info->tail_offset)) { qp->b_info = b_info; @@ -2937,6 +2944,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, success: + *total_nb_ops = b_info->total_nb_ops; nb_ops = ccp_prepare_ops(qp, op, b_info, nb_ops); rte_atomic64_add(&b_info->cmd_q->free_slots, b_info->desccnt); b_info->desccnt = 0; diff --git a/drivers/crypto/ccp/ccp_crypto.h b/drivers/crypto/ccp/ccp_crypto.h index 882b398ac..8e6d03efc 100644 --- a/drivers/crypto/ccp/ccp_crypto.h +++ b/drivers/crypto/ccp/ccp_crypto.h @@ -353,7 +353,9 @@ int process_ops_to_enqueue(struct ccp_qp *qp, struct rte_crypto_op **op, struct ccp_queue *cmd_q, uint16_t nb_ops, - int slots_req); + uint16_t total_nb_ops, + int slots_req, + uint16_t b_idx); /** * process crypto ops to be dequeued @@ -365,7 +367,8 @@ int process_ops_to_enqueue(struct ccp_qp *qp, */ int process_ops_to_dequeue(struct ccp_qp *qp, struct rte_crypto_op **op, - uint16_t nb_ops); + uint16_t nb_ops, + uint16_t *total_nb_ops); /** diff --git a/drivers/crypto/ccp/ccp_dev.h b/drivers/crypto/ccp/ccp_dev.h index de3e4bcc6..f4ad9eafd 100644 --- a/drivers/crypto/ccp/ccp_dev.h +++ b/drivers/crypto/ccp/ccp_dev.h @@ -59,7 +59,7 @@ #define CMD_Q_RUN 0x1 #define CMD_Q_SIZE 0x1F #define CMD_Q_SHIFT 3 -#define COMMANDS_PER_QUEUE 2048 +#define COMMANDS_PER_QUEUE 8192 #define QUEUE_SIZE_VAL ((ffs(COMMANDS_PER_QUEUE) - 2) & \ CMD_Q_SIZE) diff --git a/drivers/crypto/ccp/ccp_pmd_private.h b/drivers/crypto/ccp/ccp_pmd_private.h index 781050c31..1c4118ee3 100644 --- a/drivers/crypto/ccp/ccp_pmd_private.h +++ b/drivers/crypto/ccp/ccp_pmd_private.h @@ -50,8 +50,10 @@ struct ccp_batch_info { struct rte_crypto_op *op[CCP_MAX_BURST]; /**< optable populated at enque time from app*/ int op_idx; + uint16_t b_idx; struct ccp_queue *cmd_q; uint16_t opcnt; + uint16_t total_nb_ops; /**< no. of crypto ops in batch*/ int desccnt; /**< no. of ccp queue descriptors*/ diff --git a/drivers/crypto/ccp/rte_ccp_pmd.c b/drivers/crypto/ccp/rte_ccp_pmd.c index 4810d799c..38cb1fe3d 100644 --- a/drivers/crypto/ccp/rte_ccp_pmd.c +++ b/drivers/crypto/ccp/rte_ccp_pmd.c @@ -22,6 +22,7 @@ */ static unsigned int ccp_pmd_init_done; uint8_t ccp_cryptodev_driver_id; +uint8_t cryptodev_cnt; struct ccp_pmd_init_params { struct rte_cryptodev_pmd_init_params def_p; @@ -201,30 +202,46 @@ ccp_pmd_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, struct ccp_queue *cmd_q; struct rte_cryptodev *dev = qp->dev; uint16_t i, enq_cnt = 0, slots_req = 0; + uint16_t tmp_ops = nb_ops, b_idx, cur_ops = 0; if (nb_ops == 0) return 0; if (unlikely(rte_ring_full(qp->processed_pkts) != 0)) return 0; + if (tmp_ops >= cryptodev_cnt) + cur_ops = nb_ops / cryptodev_cnt + (nb_ops)%cryptodev_cnt; + else + cur_ops = tmp_ops; + while (tmp_ops) { + b_idx = nb_ops - tmp_ops; + slots_req = 0; + if (cur_ops <= tmp_ops) { + tmp_ops -= cur_ops; + } else { + cur_ops = tmp_ops; + tmp_ops = 0; + } + for (i = 0; i < cur_ops; i++) { + sess = get_ccp_session(qp, ops[i + b_idx]); + if (unlikely(sess == NULL) && (i == 0)) { + qp->qp_stats.enqueue_err_count++; + return 0; + } else if (sess == NULL) { + cur_ops = i; + break; + } + slots_req += ccp_compute_slot_count(sess); + } - for (i = 0; i < nb_ops; i++) { - sess = get_ccp_session(qp, ops[i]); - if (unlikely(sess == NULL) && (i == 0)) { - qp->qp_stats.enqueue_err_count++; + cmd_q = ccp_allot_queue(dev, slots_req); + if (unlikely(cmd_q == NULL)) return 0; - } else if (sess == NULL) { - nb_ops = i; - break; - } - slots_req += ccp_compute_slot_count(sess); + enq_cnt += process_ops_to_enqueue(qp, ops, cmd_q, cur_ops, + nb_ops, slots_req, b_idx); + i++; } - cmd_q = ccp_allot_queue(dev, slots_req); - if (unlikely(cmd_q == NULL)) - return 0; - - enq_cnt = process_ops_to_enqueue(qp, ops, cmd_q, nb_ops, slots_req); qp->qp_stats.enqueued_count += enq_cnt; return enq_cnt; } @@ -234,9 +251,16 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, uint16_t nb_ops) { struct ccp_qp *qp = queue_pair; - uint16_t nb_dequeued = 0, i; + uint16_t nb_dequeued = 0, i, total_nb_ops; + + nb_dequeued = process_ops_to_dequeue(qp, ops, nb_ops, &total_nb_ops); - nb_dequeued = process_ops_to_dequeue(qp, ops, nb_ops); + if (total_nb_ops) { + while (nb_dequeued != total_nb_ops) { + nb_dequeued = process_ops_to_dequeue(qp, + ops, nb_ops, &total_nb_ops); + } + } /* Free session if a session-less crypto op */ for (i = 0; i < nb_dequeued; i++) @@ -296,7 +320,6 @@ cryptodev_ccp_create(const char *name, { struct rte_cryptodev *dev; struct ccp_private *internals; - uint8_t cryptodev_cnt = 0; if (init_params->def_p.name[0] == '\0') strlcpy(init_params->def_p.name, name, -- 2.17.1