From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1F7D1A2EFC for ; Tue, 15 Oct 2019 09:03:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id ED4CC1D414; Tue, 15 Oct 2019 09:03:02 +0200 (CEST) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-eopbgr740051.outbound.protection.outlook.com [40.107.74.51]) by dpdk.org (Postfix) with ESMTP id 83B081C43F; Tue, 15 Oct 2019 09:02:59 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q8eARaTEFMLP4KUjIv2KUAl3Uy5tt28I6/32yIsiU5HjSpZl1R8Y33gCVVhQHa3vTbhcJIuTPd4PCcLEabpEsLD0OXGx+4DinZsx3Ag1+/eAv1IhaFXh/JAd7FtgXM2M5JynuIxsSJg/f3KoU5ONhf6vIRiDXOG6A6Nup34nLcfSaiMZgokLDtHvb/nRu4sx709ffo6mdjBmcTP7WHp1vlxw7hAgsJs7sSz5G3/IMU4c58mq+eQNNHTUzfrXNYW6XQEDYE+AbCrLSXcMgmEeatWQTeBc3LvSlA5XhbjFG39VhGOIVNV7carIj/yRkU/j3bi/QFaRTIxHas01LCYqxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3oEKzsoyR0xhgEVwy+QgmifmNqrDU/z4EfRy+EvMSec=; b=MsUumSMOB3q0IqHZPkSpya8fWHqS7xFLuV6TeG/VZNTqBS74WBlLkXCdQ81r5fY+qckRLQXm58nJtL6XRUCwURKM2LYVyCbxqNmPqj1ahzBFjbXX+zVOo5chUg0a+BB8ajkvzHudrKYiofWxr5cl8S5aBOa1Zswh4x2FlnmnICa2RZbLyTvdVzRYg8eIXaZVlTFVZZ7q+sqgrpijYc1BYrn0XrZJceG9N3cAeR53jvcJVVqZgvVlLkffGsfnAcSK75F7SsvbEonLywdK8yfnmI1QAhxOI12aBy/Txwgl2XP3R8l2R3+aK8XyGvOjOHlGGUfJlwZ7VvADOfb38/m0LA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3oEKzsoyR0xhgEVwy+QgmifmNqrDU/z4EfRy+EvMSec=; b=ZaMnykslE8nsFj2ovRTUBB8Sfe71CENqz7ZQvx9bmKQP+Ocd0Ep1PvIFf1yfOazJ/DJLojMX/zj2o4DSBSuYqjkup6wAZrmvVBzLNqL7cW1JHo7ATMRs6BLpMIE/vdl8tOEQV4koOC+WnQ9Q9Ls3eEsnXm6zilpBlJFVAjla0Us= Received: from DM5PR1201MB2474.namprd12.prod.outlook.com (10.172.87.136) by DM5PR1201MB2472.namprd12.prod.outlook.com (10.172.86.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.19; Tue, 15 Oct 2019 07:02:58 +0000 Received: from DM5PR1201MB2474.namprd12.prod.outlook.com ([fe80::f3:94a5:9dbc:1fd5]) by DM5PR1201MB2474.namprd12.prod.outlook.com ([fe80::f3:94a5:9dbc:1fd5%12]) with mapi id 15.20.2347.023; Tue, 15 Oct 2019 07:02:58 +0000 From: To: "dev@dpdk.org" CC: "stable@dpdk.org" Thread-Topic: [PATCH v1 6/6] crypto/ccp: scheduling multiple CCP within single burst Thread-Index: AQHVgyaTV1GS07M7VUSBm8yNMjhUFQ== Date: Tue, 15 Oct 2019 07:02:58 +0000 Message-ID: <20191015070239.93608-1-asomalap@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: MA1PR01CA0081.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00::21) To DM5PR1201MB2474.namprd12.prod.outlook.com (2603:10b6:3:e3::8) authentication-results: spf=none (sender IP is ) smtp.mailfrom=Amaranath.Somalapuram@amd.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [165.204.156.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1c9ed330-ebc3-44b7-188f-08d7513db5d0 x-ms-office365-filtering-ht: Tenant x-ms-traffictypediagnostic: DM5PR1201MB2472: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2733; x-forefront-prvs: 01917B1794 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(346002)(366004)(376002)(39860400002)(396003)(189003)(199004)(305945005)(450100002)(81156014)(1076003)(25786009)(7736002)(81166006)(1730700003)(66476007)(66446008)(50226002)(8936002)(64756008)(316002)(8676002)(2616005)(71190400001)(71200400001)(6916009)(2906002)(99286004)(36756003)(66556008)(6506007)(386003)(6486002)(102836004)(14454004)(2351001)(2501003)(256004)(52116002)(6436002)(186003)(486006)(66946007)(66066001)(5640700003)(476003)(5660300002)(26005)(4326008)(3846002)(6512007)(6116002)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR1201MB2472; H:DM5PR1201MB2474.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qmgkMLz9Cd9+420EiDXAX83+nCPHJgtOqZqPbwNZimqjy5b/Rp5PewE+5aqPDav7nhmdz7TFg+yF1SbtQD3Few6cTRGxMGHZkQ4CW00WnNa25Bji4Q9VPeRlV/dUSdHMbdv6iCAY0hSDtNzDuoCxOmiPzRUTWfJsxmv0oigD7iQWj5aLun6NFa57PjTv4FAf2ZRvrtvHubU3heRN0pyOT6F91zQTTQayms1h6JoJnTgqNGNeCk8AN+whFRsewSm2bI+ha535Glb9E9pXyynVNKlJm0G+93aVjYDc+PutBL6WvV1ifEOOMK9qV1wbYEO1e3ZeBBfC35AWl/74lqrQ3KDQU5EYCB3cwMxibQFv0AaTwuiUIHaZJAaPczGBEPx6T+qLg38eLCaYaexPmcl1ORSVuz//yJChQ5hQ8KdyxJ4= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1c9ed330-ebc3-44b7-188f-08d7513db5d0 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Oct 2019 07:02:58.3422 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 55lcPZNYbb1xfl+U4WBGw1amguVoQ0uyw2mmNNUNqaG2ReBgfPGEcR+8MSp254aTZhr2+t++H/kjZml4HJ6uGA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB2472 Subject: [dpdk-stable] [PATCH v1 6/6] crypto/ccp: scheduling multiple CCP within single burst X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" From: Amaranath Somalapuram ccp driver was scheduling only one CCP in a single burst. Effective throughput was limited to 1 CCP performance. Scheduling multiple ccp within one burst will increase the ccp performance. this changes will divide the enqueue packets equally among the multiple CCP Cc: stable@dpdk.org Signed-off-by: Amaranath Somalapuram --- drivers/crypto/ccp/ccp_crypto.c | 22 +++++++---- drivers/crypto/ccp/ccp_crypto.h | 7 +++- drivers/crypto/ccp/ccp_pmd_private.h | 2 + drivers/crypto/ccp/rte_ccp_pmd.c | 57 +++++++++++++++++++--------- 4 files changed, 62 insertions(+), 26 deletions(-) diff --git a/drivers/crypto/ccp/ccp_crypto.c b/drivers/crypto/ccp/ccp_crypt= o.c index 8862a1a84..23694bac6 100644 --- a/drivers/crypto/ccp/ccp_crypto.c +++ b/drivers/crypto/ccp/ccp_crypto.c @@ -2803,7 +2803,9 @@ process_ops_to_enqueue(struct ccp_qp *qp, struct rte_crypto_op **op, struct ccp_queue *cmd_q, uint16_t nb_ops, - int slots_req) + uint16_t total_nb_ops, + int slots_req, + uint16_t b_idx) { int i, result =3D 0; struct ccp_batch_info *b_info; @@ -2824,6 +2826,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, =20 /* populate batch info necessary for dequeue */ b_info->op_idx =3D 0; + b_info->b_idx =3D 0; b_info->lsb_buf_idx =3D 0; b_info->desccnt =3D 0; b_info->cmd_q =3D cmd_q; @@ -2839,7 +2842,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, =20 b_info->head_offset =3D (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); - for (i =3D 0; i < nb_ops; i++) { + for (i =3D b_idx; i < (nb_ops+b_idx); i++) { session =3D (struct ccp_session *)get_sym_session_private_data( op[i]->sym->session, ccp_cryptodev_driver_id); @@ -2891,6 +2894,8 @@ process_ops_to_enqueue(struct ccp_qp *qp, } =20 b_info->opcnt =3D i; + b_info->b_idx =3D b_idx; + b_info->total_nb_ops =3D total_nb_ops; b_info->tail_offset =3D (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); =20 @@ -2905,7 +2910,7 @@ process_ops_to_enqueue(struct ccp_qp *qp, rte_ring_enqueue(qp->processed_pkts, (void *)b_info); =20 EVP_MD_CTX_destroy(auth_ctx); - return i; + return i-b_idx; } =20 static inline void ccp_auth_dq_prepare(struct rte_crypto_op *op) @@ -2990,8 +2995,8 @@ ccp_prepare_ops(struct ccp_qp *qp, } min_ops =3D RTE_MIN(nb_ops, b_info->opcnt); =20 - for (i =3D 0; i < min_ops; i++) { - op_d[i] =3D b_info->op[b_info->op_idx++]; + for (i =3D b_info->b_idx; i < min_ops; i++) { + op_d[i] =3D b_info->op[b_info->b_idx + b_info->op_idx++]; session =3D (struct ccp_session *)get_sym_session_private_data( op_d[i]->sym->session, ccp_cryptodev_driver_id); @@ -3032,7 +3037,8 @@ ccp_prepare_ops(struct ccp_qp *qp, int process_ops_to_dequeue(struct ccp_qp *qp, struct rte_crypto_op **op, - uint16_t nb_ops) + uint16_t nb_ops, + uint16_t *total_nb_ops) { struct ccp_batch_info *b_info; uint32_t cur_head_offset; @@ -3047,6 +3053,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, =20 if (b_info->auth_ctr =3D=3D b_info->opcnt) goto success; + *total_nb_ops =3D b_info->total_nb_ops; cur_head_offset =3D CCP_READ_REG(b_info->cmd_q->reg_base, CMD_Q_HEAD_LO_BASE); =20 @@ -3056,7 +3063,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, qp->b_info =3D b_info; return 0; } - } else { + } else if (b_info->tail_offset !=3D b_info->head_offset) { if ((cur_head_offset >=3D b_info->head_offset) || (cur_head_offset < b_info->tail_offset)) { qp->b_info =3D b_info; @@ -3066,6 +3073,7 @@ process_ops_to_dequeue(struct ccp_qp *qp, =20 =20 success: + *total_nb_ops =3D b_info->total_nb_ops; nb_ops =3D ccp_prepare_ops(qp, op, b_info, nb_ops); rte_atomic64_add(&b_info->cmd_q->free_slots, b_info->desccnt); b_info->desccnt =3D 0; diff --git a/drivers/crypto/ccp/ccp_crypto.h b/drivers/crypto/ccp/ccp_crypt= o.h index 882b398ac..8e6d03efc 100644 --- a/drivers/crypto/ccp/ccp_crypto.h +++ b/drivers/crypto/ccp/ccp_crypto.h @@ -353,7 +353,9 @@ int process_ops_to_enqueue(struct ccp_qp *qp, struct rte_crypto_op **op, struct ccp_queue *cmd_q, uint16_t nb_ops, - int slots_req); + uint16_t total_nb_ops, + int slots_req, + uint16_t b_idx); =20 /** * process crypto ops to be dequeued @@ -365,7 +367,8 @@ int process_ops_to_enqueue(struct ccp_qp *qp, */ int process_ops_to_dequeue(struct ccp_qp *qp, struct rte_crypto_op **op, - uint16_t nb_ops); + uint16_t nb_ops, + uint16_t *total_nb_ops); =20 =20 /** diff --git a/drivers/crypto/ccp/ccp_pmd_private.h b/drivers/crypto/ccp/ccp_= pmd_private.h index 781050c31..1c4118ee3 100644 --- a/drivers/crypto/ccp/ccp_pmd_private.h +++ b/drivers/crypto/ccp/ccp_pmd_private.h @@ -50,8 +50,10 @@ struct ccp_batch_info { struct rte_crypto_op *op[CCP_MAX_BURST]; /**< optable populated at enque time from app*/ int op_idx; + uint16_t b_idx; struct ccp_queue *cmd_q; uint16_t opcnt; + uint16_t total_nb_ops; /**< no. of crypto ops in batch*/ int desccnt; /**< no. of ccp queue descriptors*/ diff --git a/drivers/crypto/ccp/rte_ccp_pmd.c b/drivers/crypto/ccp/rte_ccp_= pmd.c index a182c6a52..4807b580e 100644 --- a/drivers/crypto/ccp/rte_ccp_pmd.c +++ b/drivers/crypto/ccp/rte_ccp_pmd.c @@ -23,6 +23,7 @@ static unsigned int ccp_pmd_init_done; uint8_t ccp_cryptodev_driver_id; extern void *sha_ctx; +uint8_t cryptodev_cnt; =20 struct ccp_pmd_init_params { struct rte_cryptodev_pmd_init_params def_p; @@ -202,30 +203,45 @@ ccp_pmd_enqueue_burst(void *queue_pair, struct rte_cr= ypto_op **ops, struct ccp_queue *cmd_q; struct rte_cryptodev *dev =3D qp->dev; uint16_t i, enq_cnt =3D 0, slots_req =3D 0; + uint16_t tmp_ops =3D nb_ops, b_idx, cur_ops =3D 0; =20 if (nb_ops =3D=3D 0) return 0; =20 if (unlikely(rte_ring_full(qp->processed_pkts) !=3D 0)) return 0; + if (tmp_ops >=3D cryptodev_cnt) + cur_ops =3D nb_ops / cryptodev_cnt + (nb_ops)%cryptodev_cnt; + else + cur_ops =3D tmp_ops; + while (tmp_ops) { + b_idx =3D nb_ops - tmp_ops; + slots_req =3D 0; + if (cur_ops <=3D tmp_ops) { + tmp_ops -=3D cur_ops; + } else { + cur_ops =3D tmp_ops; + tmp_ops =3D 0; + } + for (i =3D 0; i < cur_ops; i++) { + sess =3D get_ccp_session(qp, ops[i + b_idx]); + if (unlikely(sess =3D=3D NULL) && (i =3D=3D 0)) { + qp->qp_stats.enqueue_err_count++; + return 0; + } else if (sess =3D=3D NULL) { + cur_ops =3D i; + break; + } + slots_req +=3D ccp_compute_slot_count(sess); + } =20 - for (i =3D 0; i < nb_ops; i++) { - sess =3D get_ccp_session(qp, ops[i]); - if (unlikely(sess =3D=3D NULL) && (i =3D=3D 0)) { - qp->qp_stats.enqueue_err_count++; + cmd_q =3D ccp_allot_queue(dev, slots_req); + if (unlikely(cmd_q =3D=3D NULL)) return 0; - } else if (sess =3D=3D NULL) { - nb_ops =3D i; - break; - } - slots_req +=3D ccp_compute_slot_count(sess); + enq_cnt +=3D process_ops_to_enqueue(qp, ops, cmd_q, cur_ops, + nb_ops, slots_req, b_idx); + i++; } - - cmd_q =3D ccp_allot_queue(dev, slots_req); - if (unlikely(cmd_q =3D=3D NULL)) - return 0; - - enq_cnt =3D process_ops_to_enqueue(qp, ops, cmd_q, nb_ops, slots_req); qp->qp_stats.enqueued_count +=3D enq_cnt; return enq_cnt; } @@ -235,9 +251,16 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_cry= pto_op **ops, uint16_t nb_ops) { struct ccp_qp *qp =3D queue_pair; - uint16_t nb_dequeued =3D 0, i; + uint16_t nb_dequeued =3D 0, i, total_nb_ops; + + nb_dequeued =3D process_ops_to_dequeue(qp, ops, nb_ops, &total_nb_ops); =20 - nb_dequeued =3D process_ops_to_dequeue(qp, ops, nb_ops); + if (total_nb_ops) { + while (nb_dequeued !=3D total_nb_ops) { + nb_dequeued =3D process_ops_to_dequeue(qp, + ops, nb_ops, &total_nb_ops); + } + } =20 /* Free session if a session-less crypto op */ for (i =3D 0; i < nb_dequeued; i++) --=20 2.17.1