From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3A728A0524; Fri, 31 Jan 2020 18:05:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1C8381C12D; Fri, 31 Jan 2020 18:05:03 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id BADF71C12D for ; Fri, 31 Jan 2020 18:05:01 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00VGu9uY032202; Fri, 31 Jan 2020 09:05:01 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=DBhLUlB5e/eSV5aXbyt97vw+zGT5CcQPa0Su9myXtyY=; b=KlDrN0Le4ntbz7lzMw4hRfiGhprTABIxE5yW85ZQRIQavMR+SuNSUzRv1XYFYF6P+l3P q/yxqITESUSI/FGiZ37u+UWt33UPbsUexauXEwSzJKWGvGFJ5G9jKeAJaHOpOGT/7xIF rgqzh5PUnUILPag5TET6efJw4dqTPesJDd3Divq7422I779I8/tHIfPW7x5AzzFU0mLm vIOrkFSRqSxlKJMyHftSUe15HJeo7BegrKpYM4ofpJdDl4u9yFVf5lpYiPiPsWQUl9CS 5ztu74iYF0OepyvKqgIQiPERh0tFbURCNUKBWcZS8M1JEsePhK/cBcBmdeAARZXXvOkO uA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xvpas8gat-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 31 Jan 2020 09:05:00 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 31 Jan 2020 09:04:59 -0800 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.106) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Fri, 31 Jan 2020 09:04:59 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HHpD+c8Nh0XF5kqfdK2IEltJlq7vXWc0u4pKftDw97e4O9alraIVkzxIpHtl4zbJL1oA1QEm8A0WzBbaeEqPKnl5Y+O6raN6Z9IddGOHX3Oaq1gp6BfIX24yQQDkFq+Jd6hr3qzDT3bmpeUDwwjoAI/a5Fplqu3A+sLiW3NgiCKAO9lrVnF73uduasEmJIAOKooFtJ2g/DE4MCOVFCRChX4SpKeACeikmJq7m82XARMx2Mxw6bOi7RfP4U4G9o/gtyjsd2jrUJypaCNKCF2F8IcaYABHj7J+1XTu6SvN0Dsy0DdHU9TIb4qo97ifUi739Ra80uecVUNQiB+XEULGhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DBhLUlB5e/eSV5aXbyt97vw+zGT5CcQPa0Su9myXtyY=; b=azmhoAzM5TaaAx1NiRZLzET48ftSyi94bIkMz5C9mDjtAXSSAPdKBGKqLtC9Sa8iPrfXPlmUHjPACKWfbQuasTzdntot/zgeFXC3GlsjgxhyTW582+iTZgAQOzgE74ersoopggc+A9OisCEfySFoxc8eKtMZLPWD8rLd2gZ6zOGiuuqtFuxR1g6cn2Hj4wsODuFR/q1sliLgTcBlY2a4NPwCvXvsHbtr6lsKuas2lO/87hdrQTq9D3xQeTJQHa6wXFzwuSBOPrl48EpBiz7jbMk1PvWeY5w/vxkF7ykoN86AEtKr1tAkHEEKNbf7JPa/oinZPGAhxwiEr8hr7EeQGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DBhLUlB5e/eSV5aXbyt97vw+zGT5CcQPa0Su9myXtyY=; b=JxDb3K2EeXEHpMgXYKR2EYIV1zMh1J5sSOd8/Q3Wg1bsdxiZfci3ao8l3RAEhcNEnBwJfX7ejhsrocCzL3iZYsCACsJXj8ddE4AreapwGIzHOs8Oiw1wI51B+nh0avkxgxDQYJIj2dPmyfkyrFhF+2uKJGTMGN0HuFHvYZoaArA= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (20.179.20.218) by MN2PR18MB3280.namprd18.prod.outlook.com (10.255.236.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2665.24; Fri, 31 Jan 2020 17:04:58 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::e48d:494:fc46:3572]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::e48d:494:fc46:3572%7]) with mapi id 15.20.2686.028; Fri, 31 Jan 2020 17:04:58 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" CC: Jerin Jacob Kollanukkaran , Lukas Bartosik , Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params Thread-Index: AQHVx8TYVDK8x5pEckiwirDv9/IQuKgCUt6AgAEuDPCAAZd9AIAABTbg Date: Fri, 31 Jan 2020 17:04:58 +0000 Message-ID: References: <1578667598-18942-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [27.34.250.227] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d930c129-cefa-4a14-f049-08d7a66fb3c5 x-ms-traffictypediagnostic: MN2PR18MB3280: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4941; x-forefront-prvs: 029976C540 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(366004)(346002)(39860400002)(136003)(376002)(189003)(199004)(2906002)(66446008)(33656002)(64756008)(66556008)(316002)(71200400001)(76116006)(66476007)(66946007)(8936002)(110136005)(478600001)(5660300002)(81166006)(8676002)(186003)(81156014)(54906003)(53546011)(52536014)(86362001)(9686003)(55016002)(26005)(4326008)(7696005)(6506007); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3280; H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: o+J1onR5OE+/wBMCybR1TrUjk2+RHrcgCeImj2X5Zp0381S4OaLCGU3D0At28TBkIrtu56iwCc1JKyV3d+M3JM3gmoGg+4hrLXY2ARoBAoYaSdwDf5wORLWHzm2rH+pUufKv5FyPjPS3nwiH34C3A8bB+785VZ4DVM70Pq/vE+7r6WUay9HycKYPwJtIDa9Wx3VOk0msQ2UxyvGbMmNV39GPwW2u4mqw+630SwsMv1hYc2AvUwT9jULL5UYjGORuLy/dySn5dJ0w2DRryfb2l+biaWffM977VW+DK9A7LpGHCBqmyWcA6pHGEtkl73ZiaoClKf2Y/FiPRVJNzwjk2KJoUsJ4l37AHqFTecnfIz6ybH1Fn2YsAurr8Qt/ZcvTwA+JLrxHl5c06bk1qdXLGKF29ZcgRxF38ESryl/qg8MFzfQbQpfc8wFdB6PLw1fo x-ms-exchange-antispam-messagedata: McboMLpXnq8npZzkkgY6yuZrONmwcqdx92ofE4MVxuRWPnEdAXY10G7hdxRby6ofHlbq/ubrz+4ZFGBnCrMzKFTnUiejIatzPgC6xQ5qzow8tHHKUc2weJ36E9GgDfpLRtn8Xtcm+Kwan6eSjuqdvg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: d930c129-cefa-4a14-f049-08d7a66fb3c5 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jan 2020 17:04:58.2040 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: d2KgI79QO9RFthGClFWFlgsN10LLdLP+leSwaM8c3FV30I4bv8QPlhQtcl/j5M5UIfc/5goEsHvYqu360Re20w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3280 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-31_04:2020-01-31, 2020-01-31 signatures=0 Subject: Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: Ananyev, Konstantin > Sent: Friday, January 31, 2020 10:02 PM > To: Anoob Joseph ; Akhil Goyal ; > Nicolau, Radu > Cc: Jerin Jacob Kollanukkaran ; Lukas Bartosik > ; Narayana Prasad Raju Athreya > ; dev@dpdk.org > Subject: [EXT] RE: [PATCH] examples/ipsec-secgw: increase number of qps t= o > lcore_params >=20 > External Email >=20 > ---------------------------------------------------------------------- >=20 >=20 > > > > > > -------------------------------------------------------------------- > > > -- > > > > > > > > > > > > > > Currently only one qp will be used for one core. The number of qps > > > > can be increased to match the number of lcore params. > > > > > > I don't really understand the purpose of that patch.... > > > As I understand, all it does - unconditionally increases number of > > > crypto-queues mapped to the same lcore. > > > > [Anoob] With the current code, we will have only crypto qp mapped to > > one lcore. So even if you have large number of crypto qps, you would be= only > using as much as the number of lcores. > > > > This is an inefficient model as a fat flow(say with large packet > > sizes) on one eth queue hitting one core can starve another flow which > happens to hit the same core, because both flows would get queued to the > same qp. > > And, we cannot just randomly submit to multiple qps from the same core > > as then the ordering would be messed up. >=20 > No-one suggesting that one I believe. >=20 > So the best possible usage model would be to map one eth queue to one > > crypto qp. That way, the core wouldn't be unnecessarily pipeline the cr= ypto > processing. >=20 > I doubt it is really the 'best possible usage model'. > There could be cases when forcing lcore to use/manage more crypto-queues > will lead to negative effects: perf drop, not enough crypto queues for al= l eth > queues, etc. > If your HW/SW requires to match each eth queue with a separate crypto-que= ue, > then I think it should be an optional feature, while keeping default beha= vior > intact. [Anoob] I think the question here is more to do with s/w crypto PMDs vs h/w= crypto PMDs. For s/w PMDs, having more queues doesn't really make sense an= d for h/w PMDs it's better. I'll see how we can make this an optional featu= re. Would you be okay with allowing such behavior if the underlying PMD can= support as many queues as lcore_params? As in, if the PMD doesn't support enough queues, we do 1 qp per core. Would= that work for you?=20 Also, if we want to stick to 1 crypto qp per lcore, I don't understand why = we should have the following macro defined such, #define MAX_QP_PER_LCORE 256 =20 >=20 > > > > > The question is what for? > > > All these extra queues woulnd't be used by current poll mode data-pat= h > anyway. > > > Plus in some cases hash_lookup() would fail to find existing mapping= . > > > > [Anoob] It was an issue with v1 that we have addressed with v2. I'll se= nd v2 > shortly. Please do check that see if you have more comments. > > > > > I understand that for your eventdev implementation you need one > > > crypto queue for each eth device. > > > But I think it could be done in less intrusive way (see my previous > > > mail as one possible option). > > > > [Anoob] If this suggestion is agreeable, then we can solve both qp > > number requirement (for inline) and default nb_mbuf calculation in a > > much more sensible way. If this doesn't fly, I'll probably go back to y= our > suggestion, but then there would be more code just to handle these cases.= And > the nb_mbuf calculation could turn slightly complicated. >=20 > Don't see how it affects number of required mbufs... > Wouldn't it just be: > * + > * + > *+.... [Anoob] What would be num_of_crypto_queues?=20 > ? >=20 >=20 >=20 > > > > > Konstantin > > > > > > > > > > > Signed-off-by: Anoob Joseph > > > > --- > > > > examples/ipsec-secgw/ipsec-secgw.c | 39 > > > > +++++++++++++++++++-------------- > > > ----- > > > > examples/ipsec-secgw/ipsec.h | 4 +++- > > > > 2 files changed, 23 insertions(+), 20 deletions(-) > > > > > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > > > b/examples/ipsec-secgw/ipsec-secgw.c > > > > index 3b5aaf6..d8c435e 100644 > > > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > > > @@ -1709,6 +1709,8 @@ add_mapping(struct rte_hash *map, const char > > > *str, uint16_t cdev_id, > > > > unsigned long i; > > > > struct cdev_key key =3D { 0 }; > > > > > > > > + key.port_id =3D params->port_id; > > > > + key.queue_id =3D params->queue_id; > > > > key.lcore_id =3D params->lcore_id; > > > > if (cipher) > > > > key.cipher_algo =3D cipher->sym.cipher.algo; @@ -1721,23 > > > +1723,17 @@ > > > > add_mapping(struct rte_hash *map, const char *str, uint16_t cdev_id= , > > > > if (ret !=3D -ENOENT) > > > > return 0; > > > > > > > > - for (i =3D 0; i < ipsec_ctx->nb_qps; i++) > > > > - if (ipsec_ctx->tbl[i].id =3D=3D cdev_id) > > > > - break; > > > > - > > > > - if (i =3D=3D ipsec_ctx->nb_qps) { > > > > - if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > - printf("Maximum number of crypto devices assigned to > > > " > > > > - "a core, increase MAX_QP_PER_LCORE > > > value\n"); > > > > - return 0; > > > > - } > > > > - ipsec_ctx->tbl[i].id =3D cdev_id; > > > > - ipsec_ctx->tbl[i].qp =3D qp; > > > > - ipsec_ctx->nb_qps++; > > > > - printf("%s cdev mapping: lcore %u using cdev %u qp %u " > > > > - "(cdev_id_qp %lu)\n", str, key.lcore_id, > > > > - cdev_id, qp, i); > > > > + i =3D ipsec_ctx->nb_qps; > > > > + if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > + printf("Maximum number of crypto devices assigned to a core, " > > > > + "increase MAX_QP_PER_LCORE value\n"); > > > > + return 0; > > > > } > > > > + ipsec_ctx->tbl[i].id =3D cdev_id; > > > > + ipsec_ctx->tbl[i].qp =3D qp; > > > > + ipsec_ctx->nb_qps++; > > > > + printf("%s cdev mapping: lcore %u using cdev %u qp %u " > > > > + "(cdev_id_qp %lu)\n", str, key.lcore_id, cdev_id, qp, i); > > > > > > > > ret =3D rte_hash_add_key_data(map, &key, (void *)i); > > > > if (ret < 0) { > > > > @@ -1785,8 +1781,10 @@ add_cdev_mapping(struct rte_cryptodev_info > > > *dev_info, uint16_t cdev_id, > > > > continue; > > > > > > > > if (i->sym.xform_type =3D=3D RTE_CRYPTO_SYM_XFORM_AEAD) { > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, params, > > > > + ret =3D add_mapping(map, str, cdev_id, qp, params, > > > > ipsec_ctx, NULL, NULL, i); > > > > + if (ret) > > > > + return ret; > > > > continue; > > > > } > > > > > > > > @@ -1801,12 +1799,15 @@ add_cdev_mapping(struct rte_cryptodev_info > > > *dev_info, uint16_t cdev_id, > > > > if (j->sym.xform_type !=3D > > > RTE_CRYPTO_SYM_XFORM_AUTH) > > > > continue; > > > > > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, params, > > > > + ret =3D add_mapping(map, str, cdev_id, qp, params, > > > > ipsec_ctx, i, j, NULL); > > > > + if (ret) > > > > + return ret; > > > > + continue; > > > > } > > > > } > > > > > > > > - return ret; > > > > + return 0; > > > > } > > > > > > > > /* Check if the device is enabled by cryptodev_mask */ diff --git > > > > a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h > > > > index 8e07521..92fd5eb 100644 > > > > --- a/examples/ipsec-secgw/ipsec.h > > > > +++ b/examples/ipsec-secgw/ipsec.h > > > > @@ -200,7 +200,9 @@ struct ipsec_ctx { }; > > > > > > > > struct cdev_key { > > > > - uint16_t lcore_id; > > > > + uint16_t port_id; > > > > + uint8_t queue_id; > > > > + uint8_t lcore_id; > > > > uint8_t cipher_algo; > > > > uint8_t auth_algo; > > > > uint8_t aead_algo; > > > > -- > > > > 2.7.4