From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AE55A0530; Mon, 3 Feb 2020 10:05:31 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 915411BFE2; Mon, 3 Feb 2020 10:05:30 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 903F31BF78 for ; Mon, 3 Feb 2020 10:05:29 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0138uoi1030478; Mon, 3 Feb 2020 01:05:28 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=N2T01xfyf3c+LL4XGJZyWUvUs0ksLQwf3Ky4noNptI0=; b=oj7MiahWX1xd4YxhOT5Bujs1JFyQdnXmLukKDaGNdWx17oCBpJCMSpECOliwPsCthWHN MgGlrk+CwgE6L4vS0OmeIfawq3ryprgP7JN8ZVzY6I15+VPSx95/N0m8QO8cFsllAoQU WKC3EidNcdbc9u6AacSvUDi4CxcVTnzQickIvNr3Hb/sJcNn+M3sP/8kdVE4FI5IUXJC jfYri2Cp/8vLKNukHnhQ+QvzAM/Ml6dn5+GVYZZPT7xbPUkofTD+I3RjQQdaVQkYduY7 BBvwkIsqgirhRyAmogrAPvjT76Q0Fqm2h0ieExB6jy+O1UX9UDqe5sGC0JBzNYEfr3XQ RA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2xw9que1aa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 03 Feb 2020 01:05:28 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 3 Feb 2020 01:05:25 -0800 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.50) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 3 Feb 2020 01:05:25 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AJd6CA7p75UbvQMnIcwGWOn8LDRcuS1XhfYvxxO0N+f/NgcezsqcLMH63DRG6eDHQVNnRl61TREBySjtBsv25evxGF3wMnOLeFLk5GNrH2t9UWnu59DSM09xbKMTQpVOkVBCu7kYOeMKvRmVkun03+0RrFFQGKixKwEry0DbZ0yGfjvAplJ5nmIVIe/h+KY43FqUUMSyq0I43Di1BWh+4nboheyrlP+Aee4azLBQMFSGWWslcxiyjO0nzPOd+kGUFxE0CqJ+/bQJFIZA956S39rDGqAIOQAgeAz0CYgQve35bqfQdVTgBa3m/5H88VVhXgstUaEjNKVgCYqddwBGpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N2T01xfyf3c+LL4XGJZyWUvUs0ksLQwf3Ky4noNptI0=; b=iiDnUz7Xo0J3JJp0vOr5/FySnZP91xFoOmpp3ccgZTk653f33mxb4tYdvfgGL6OZ7v4YIY+Mmz9AISH2j3vXtoLnR7cq9kky9ylba7RVIRVaXYXzveCwijamE+w+m675dKEEKQ7QmFmbYXCFvV1Y9qfUlIyB+woUXJqF7hVRKNi+zvhilJSW1NKqHwD5RhrGwpZBn1EOLfiYWjoN0WtNiv0CG3+u9y9TW+RhPeiiNF5RNmkIeZBaoF2CC4KZ1QgItPIPlDmvjiBds2Y/yfSOcS+qd7BZ03n5y8n1GyB1fYlscpViD3pSRI7ggFNN2qD5veSIAoG1IXvz2oJrLkrqAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N2T01xfyf3c+LL4XGJZyWUvUs0ksLQwf3Ky4noNptI0=; b=Wif769/2AXJ88dA6ulE699O/thihlsFaI3RWqUJ+xUTHgIcl6G5iseBhNY413NEWgqYhkmUwVZin4xBpZAg7cstxi5xO4idtlHi+oLtyt160BtMhaZTDUCVkN57KfHoJO3wBn+MUndpagZmnKve2PdwLZiRE1CB6diu1Bw2ZYzg= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (20.179.20.218) by MN2PR18MB2701.namprd18.prod.outlook.com (20.179.22.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2686.32; Mon, 3 Feb 2020 09:05:23 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::e48d:494:fc46:3572]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::e48d:494:fc46:3572%7]) with mapi id 15.20.2686.031; Mon, 3 Feb 2020 09:05:23 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" CC: Jerin Jacob Kollanukkaran , Lukas Bartosik , Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params Thread-Index: AQHVx8TYVDK8x5pEckiwirDv9/IQuKgCUt6AgAEuDPCAAZd9AIAABTbggAAhMACABBMEIA== Date: Mon, 3 Feb 2020 09:05:23 +0000 Message-ID: References: <1578667598-18942-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.140.231.66] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5b11e98f-6129-43b0-56f4-08d7a88833e7 x-ms-traffictypediagnostic: MN2PR18MB2701: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2449; x-forefront-prvs: 0302D4F392 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(376002)(136003)(366004)(39850400004)(346002)(189003)(199004)(81166006)(81156014)(52536014)(66946007)(66476007)(76116006)(5660300002)(8676002)(8936002)(66446008)(66556008)(64756008)(478600001)(7696005)(86362001)(26005)(316002)(55016002)(9686003)(186003)(4326008)(53546011)(33656002)(6506007)(54906003)(110136005)(55236004)(2906002)(71200400001); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2701; H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jjb75Vsh5+ubpADTGtMBjYa8Dzk5UBJ+683+/VPyfWqcizq+JLWdcHJD70I5fi/nyTZ0I9icdlGH2k5gRPdafgvnriAtb1aivmYF1z1fDevF06MBsE4PnvMLch2EWnUZGxxp+5gRod9FIwcA/4X/r/JSldcM76piBgy8jNLSIlq96f67sfBmImaEUVm/rIdrxXefYE9Et/jcjN1MH42OYll0ouRWs3i5pD0D5+DTHFUZQP84MMO6m+9/9MD2IL+H2c64rMqQT1X4Vwp0aaQVFo5Xesj4pOeLb7Ct/4Qy6D5mKDu/c61Ukc2zyBUOdAoJqnNu6UMwg0zNWlRX0+9W1V59Mtw9XX5FUChO4otBj/iTcqTie2liEFKDA2mDFPGakyRRhDHkp+784lZ7q+QIcwOLoXIJDlmjunba8kW8DJfh9GSgTQdmXH7NL8sfQY8Z x-ms-exchange-antispam-messagedata: sFG32e2XlfaHHFXLJuEmxvhYXVOrFejfTBjbyAwLjxhgoSXareEirdjyVG9uZ9aeH2TNbtotNzB0A6A2HbGdZ4wOloLczS+o3WSw7E3KfRu5SGZT6HnFCzVAgWc/vvGcqKuzKnvKWzIe2iKiEgpSrA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 5b11e98f-6129-43b0-56f4-08d7a88833e7 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Feb 2020 09:05:23.4093 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ui/IXaSldvNz4GWrHI9qVsK9SYa5FSWocSDZgunpusxgSOF2e21ehXQ7wM4soXbt3Jnmfct1FP4lW+wJeG7ZmA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2701 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-03_02:2020-02-02, 2020-02-03 signatures=0 Subject: Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Ananyev, Konstantin > Sent: Saturday, February 1, 2020 12:20 AM > To: Anoob Joseph ; Akhil Goyal > ; Nicolau, Radu > Cc: Jerin Jacob Kollanukkaran ; Lukas Bartosik > ; Narayana Prasad Raju Athreya > ; dev@dpdk.org > Subject: [EXT] RE: [PATCH] examples/ipsec-secgw: increase number of qps t= o > lcore_params >=20 > External Email >=20 > ---------------------------------------------------------------------- > > > > > > Currently only one qp will be used for one core. The number of > > > > > > qps can be increased to match the number of lcore params. > > > > > > > > > > I don't really understand the purpose of that patch.... > > > > > As I understand, all it does - unconditionally increases number > > > > > of crypto-queues mapped to the same lcore. > > > > > > > > [Anoob] With the current code, we will have only crypto qp mapped > > > > to one lcore. So even if you have large number of crypto qps, you > > > > would be only > > > using as much as the number of lcores. > > > > > > > > This is an inefficient model as a fat flow(say with large packet > > > > sizes) on one eth queue hitting one core can starve another flow > > > > which > > > happens to hit the same core, because both flows would get queued to > > > the same qp. > > > > And, we cannot just randomly submit to multiple qps from the same > > > > core as then the ordering would be messed up. > > > > > > No-one suggesting that one I believe. > > > > > > So the best possible usage model would be to map one eth queue to > > > one > > > > crypto qp. That way, the core wouldn't be unnecessarily pipeline > > > > the crypto > > > processing. > > > > > > I doubt it is really the 'best possible usage model'. > > > There could be cases when forcing lcore to use/manage more > > > crypto-queues will lead to negative effects: perf drop, not enough > > > crypto queues for all eth queues, etc. > > > If your HW/SW requires to match each eth queue with a separate > > > crypto-queue, then I think it should be an optional feature, while > > > keeping default behavior intact. > > > > [Anoob] I think the question here is more to do with s/w crypto PMDs > > vs h/w crypto PMDs. For s/w PMDs, having more queues doesn't really > make sense and for h/w PMDs it's better. >=20 > Not always. > If these queues belongs to the same device, sometimes it is faster to use= just > one queue for device per core. > HW descriptor status polling, etc. is not free. >=20 > > I'll see how we can make this an optional feature. Would you be okay > > with allowing such behavior if the underlying PMD can support as many > queues as lcore_params? > > > > As in, if the PMD doesn't support enough queues, we do 1 qp per core. > Would that work for you? >=20 > I am not fond of idea to change default mapping method silently. > My preference would be a new command-line option (--cdev-maping or so). > Another thought, make it more fine-grained and user-controlled by > extending eth-port-queue-lcore mapping option. > from current: ,, > to ,,,. >=20 > So let say with 2 cores , 2 eth ports/2 queues per port for current mappi= ng > user would do: > # use cdev queue 0 on all cdevs for lcore 6 # use cdev queue 1 on all cde= vs for > lcore 7 --lcores=3D"6,7" ... -- --config=3D"(0,0,6,0),(1,0,6,0),(0,1,7,1)= ,(1,1,7,1)" >=20 > for the mapping you'd like to have: > --lcores=3D"6,7" ... -- --config=3D"(0,0,6,0),(1,0,6,1),(0,1,7,2),(1,1,7,= 3)" [Anoob] I like this idea. This would work for inline case as well. @Akhil, do you have any comments? Also, I think we should make it ,,,, =20 >=20 > > Also, if we want to stick to 1 crypto qp per lcore, I don't understand > > why we should have the following macro defined such, > > > > #define MAX_QP_PER_LCORE 256 > > > > > > > > > > > > > > The question is what for? > > > > > All these extra queues woulnd't be used by current poll mode > > > > > data-path > > > anyway. > > > > > Plus in some cases hash_lookup() would fail to find existing map= ping. > > > > > > > > [Anoob] It was an issue with v1 that we have addressed with v2. > > > > I'll send v2 > > > shortly. Please do check that see if you have more comments. > > > > > > > > > I understand that for your eventdev implementation you need one > > > > > crypto queue for each eth device. > > > > > But I think it could be done in less intrusive way (see my > > > > > previous mail as one possible option). > > > > > > > > [Anoob] If this suggestion is agreeable, then we can solve both qp > > > > number requirement (for inline) and default nb_mbuf calculation in > > > > a much more sensible way. If this doesn't fly, I'll probably go > > > > back to your > > > suggestion, but then there would be more code just to handle these > > > cases. And the nb_mbuf calculation could turn slightly complicated. > > > > > > Don't see how it affects number of required mbufs... > > > Wouldn't it just be: > > > * + > > > * + > > > *+.... > > > > [Anoob] What would be num_of_crypto_queues? > > > > > ? > > > > > > > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > Signed-off-by: Anoob Joseph > > > > > > --- > > > > > > examples/ipsec-secgw/ipsec-secgw.c | 39 > > > > > > +++++++++++++++++++-------------- > > > > > ----- > > > > > > examples/ipsec-secgw/ipsec.h | 4 +++- > > > > > > 2 files changed, 23 insertions(+), 20 deletions(-) > > > > > > > > > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > > > > > b/examples/ipsec-secgw/ipsec-secgw.c > > > > > > index 3b5aaf6..d8c435e 100644 > > > > > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > > > > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > > > > > @@ -1709,6 +1709,8 @@ add_mapping(struct rte_hash *map, const > > > > > > char > > > > > *str, uint16_t cdev_id, > > > > > > unsigned long i; > > > > > > struct cdev_key key =3D { 0 }; > > > > > > > > > > > > + key.port_id =3D params->port_id; > > > > > > + key.queue_id =3D params->queue_id; > > > > > > key.lcore_id =3D params->lcore_id; > > > > > > if (cipher) > > > > > > key.cipher_algo =3D cipher->sym.cipher.algo; @@ - > 1721,23 > > > > > +1723,17 @@ > > > > > > add_mapping(struct rte_hash *map, const char *str, uint16_t > cdev_id, > > > > > > if (ret !=3D -ENOENT) > > > > > > return 0; > > > > > > > > > > > > - for (i =3D 0; i < ipsec_ctx->nb_qps; i++) > > > > > > - if (ipsec_ctx->tbl[i].id =3D=3D cdev_id) > > > > > > - break; > > > > > > - > > > > > > - if (i =3D=3D ipsec_ctx->nb_qps) { > > > > > > - if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > > > - printf("Maximum number of crypto devices > assigned to > > > > > " > > > > > > - "a core, increase > MAX_QP_PER_LCORE > > > > > value\n"); > > > > > > - return 0; > > > > > > - } > > > > > > - ipsec_ctx->tbl[i].id =3D cdev_id; > > > > > > - ipsec_ctx->tbl[i].qp =3D qp; > > > > > > - ipsec_ctx->nb_qps++; > > > > > > - printf("%s cdev mapping: lcore %u using cdev %u qp > %u " > > > > > > - "(cdev_id_qp %lu)\n", str, > key.lcore_id, > > > > > > - cdev_id, qp, i); > > > > > > + i =3D ipsec_ctx->nb_qps; > > > > > > + if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > > > + printf("Maximum number of crypto devices assigned > to a core, " > > > > > > + "increase MAX_QP_PER_LCORE value\n"); > > > > > > + return 0; > > > > > > } > > > > > > + ipsec_ctx->tbl[i].id =3D cdev_id; > > > > > > + ipsec_ctx->tbl[i].qp =3D qp; > > > > > > + ipsec_ctx->nb_qps++; > > > > > > + printf("%s cdev mapping: lcore %u using cdev %u qp %u " > > > > > > + "(cdev_id_qp %lu)\n", str, key.lcore_id, cdev_id, qp, > > > > > > +i); > > > > > > > > > > > > ret =3D rte_hash_add_key_data(map, &key, (void *)i); > > > > > > if (ret < 0) { > > > > > > @@ -1785,8 +1781,10 @@ add_cdev_mapping(struct > > > > > > rte_cryptodev_info > > > > > *dev_info, uint16_t cdev_id, > > > > > > continue; > > > > > > > > > > > > if (i->sym.xform_type =3D=3D > RTE_CRYPTO_SYM_XFORM_AEAD) { > > > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, > params, > > > > > > + ret =3D add_mapping(map, str, cdev_id, qp, > params, > > > > > > ipsec_ctx, NULL, NULL, i); > > > > > > + if (ret) > > > > > > + return ret; > > > > > > continue; > > > > > > } > > > > > > > > > > > > @@ -1801,12 +1799,15 @@ add_cdev_mapping(struct > > > > > > rte_cryptodev_info > > > > > *dev_info, uint16_t cdev_id, > > > > > > if (j->sym.xform_type !=3D > > > > > RTE_CRYPTO_SYM_XFORM_AUTH) > > > > > > continue; > > > > > > > > > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, > params, > > > > > > + ret =3D add_mapping(map, str, cdev_id, qp, > params, > > > > > > ipsec_ctx, i, j, NULL); > > > > > > + if (ret) > > > > > > + return ret; > > > > > > + continue; > > > > > > } > > > > > > } > > > > > > > > > > > > - return ret; > > > > > > + return 0; > > > > > > } > > > > > > > > > > > > /* Check if the device is enabled by cryptodev_mask */ diff > > > > > > --git a/examples/ipsec-secgw/ipsec.h > > > > > > b/examples/ipsec-secgw/ipsec.h index 8e07521..92fd5eb 100644 > > > > > > --- a/examples/ipsec-secgw/ipsec.h > > > > > > +++ b/examples/ipsec-secgw/ipsec.h > > > > > > @@ -200,7 +200,9 @@ struct ipsec_ctx { }; > > > > > > > > > > > > struct cdev_key { > > > > > > - uint16_t lcore_id; > > > > > > + uint16_t port_id; > > > > > > + uint8_t queue_id; > > > > > > + uint8_t lcore_id; > > > > > > uint8_t cipher_algo; > > > > > > uint8_t auth_algo; > > > > > > uint8_t aead_algo; > > > > > > -- > > > > > > 2.7.4