From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1FE49A0524; Fri, 31 Jan 2020 19:49:37 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 66AE11C0DD; Fri, 31 Jan 2020 19:49:36 +0100 (CET) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 092431C0DC for ; Fri, 31 Jan 2020 19:49:34 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 31 Jan 2020 10:49:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,386,1574150400"; d="scan'208";a="310085777" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga001.jf.intel.com with ESMTP; 31 Jan 2020 10:49:32 -0800 Received: from fmsmsx158.amr.corp.intel.com (10.18.116.75) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 31 Jan 2020 10:49:32 -0800 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx158.amr.corp.intel.com (10.18.116.75) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 31 Jan 2020 10:49:32 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.174) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 31 Jan 2020 10:49:32 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OyGvZEn7v/pEKMDsUqo5E7B+T8ZE7/sY/D0A6x/zjkvnzQuWeI+zLcLg+ZyurCUdBAQR69ARI3DVCBi/8ZoyCYMdl3i6/9Ua1VxMOqsgQp5H9IorfPpRT8lXRDO4f5foUZtBifRIoLwDnK/F2NxrQ3c0YpKeloLbItzB95t5iOfRZZ1V+xUZS3qYixYNanV3jfNfx0SpFJqog1BrNEfpnsHCyjv4sgDUOvrKpYfsSQSDreJXPvvtE7rMGCnpn0oflzdOXiupG8mMRwcI/yltdNwWU4bPPXu6qVjjraEpox3DMZcwtR3/wM4lAjiyVDjaqEe+bDISynmxTCyCPci0gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sNR0sbr1XSes3B/WXp7aokov00SGdtlP69NkcB0TQ4c=; b=XZUzKyL+HTXphIFp0V/6MvCcEYxmSD1MOCg4JOyUGWvBiQv8jOMk23CLJVIkzB6iU0w6m+3yP+zuIb0I3Ja7GCHn8DFRLWBgMUiiCWdcUCZZxw/y5DAcBifQ9/X1pNESOMDs3XJxQklaX1slunWNBtinbuKSe/LjxRnkU/DA50nwNhvb3Z6yB1QBJX/ELIClyQWE3btR0/WVp56DTk8rcfplpiDo7gFdSYzy/zfjZQoYlWwArZxofZNxLJiVGM7zR+bUur0oUan2ilXFMDBz8sSI9dfyb49TQSeFHL918FnRvo6T0Mktapt23mxDub9+3NbibTiaLnE1dfl/bW/WLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sNR0sbr1XSes3B/WXp7aokov00SGdtlP69NkcB0TQ4c=; b=Hw2uqF9QT3TdQjiLYiX2i/asdwwrD9eAEFSiY1ELer9JXzX7gUkFR4Zw9z2UU/8HkOj39U5jq3gsXhNHFUkXuSxLbg0Wu19zewIOjCclz1Xdpo0qItkooatSsj9NN3zgld0W2aHxTx6g0k9owezu7bz9syQXvNS0zLeiJR/DShM= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (52.135.94.19) by SN6PR11MB2718.namprd11.prod.outlook.com (52.135.97.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2665.19; Fri, 31 Jan 2020 18:49:30 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::4d86:362a:13c3:8386%7]) with mapi id 15.20.2665.027; Fri, 31 Jan 2020 18:49:30 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal , "Nicolau, Radu" CC: Jerin Jacob Kollanukkaran , Lukas Bartosik , Narayana Prasad Raju Athreya , "dev@dpdk.org" Thread-Topic: [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params Thread-Index: AQHVx8T0p6tfWl36AEO+n9Pp5WnH2agCUR9ggAEz9gCAAYtkUIAAER8AgAAWRCA= Date: Fri, 31 Jan 2020 18:49:30 +0000 Message-ID: References: <1578667598-18942-1-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMDViOTBhYjEtN2IwOC00MDhhLWE5ZGEtMzE3NmRlN2E5NmY1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiOVR0WW9na2tac2dCWnQyR2RrK1dSa2V2NmsxYkJWU1ZYUmZmVFBDcmxKQTlTY3F3MExxZDNTazhSRUp4akpkTSJ9 dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [46.7.38.224] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7d51f6fb-f6d7-473e-58f3-08d7a67e4e70 x-ms-traffictypediagnostic: SN6PR11MB2718: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3826; x-forefront-prvs: 029976C540 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(39860400002)(376002)(136003)(396003)(366004)(189003)(199004)(478600001)(4326008)(26005)(6506007)(55016002)(8676002)(81156014)(81166006)(2906002)(8936002)(316002)(9686003)(186003)(54906003)(110136005)(33656002)(7696005)(6636002)(71200400001)(66446008)(64756008)(86362001)(76116006)(66946007)(5660300002)(52536014)(66556008)(66476007); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2718; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SXou2Ew18CRwAE4H2f+lwYaR6Q9Z9E17XE/RB+8Ye8BAhYihnEG4ddNVBW98RAXq0l7jP9Rv5mF/wQMvgom+d4U2PvRmfadiJb5TSXPHGoJaoC5wl0UawdvGTsMuLOclVO38ccMXWgtSqhIy2d44swQ9NEWS8oRaENLT3qYqJdcvHaAekg8lco6J57iXDcY4o0ewVaYMDhHAmDOJloUUNLttiOWk+TFKwId06nPWian34UgEZxeUR4bNk+QcuLFXIsJKQY2NB75PVZi8NIAR9ulZaxX2zGbYDAuqTuPizrCFAsPYBJY2Xz73rk+sCWZpwDbif66r10d0KNPqwUvR96jDH3hC2lErhadGSSoRD2Q0RM+w35wRxuLEM4Relf7Ybx3dhEoatUooqd2ZOeywjJ19uC32qvzdiewIoT0CAaEG2nqGFEZCmFLCtYBxa+Mn x-ms-exchange-antispam-messagedata: wh3rD0qguL3fagfYBTlYsAci4ncuGSzHj2a82Fi5yxatl2RGfBQQJ1mRzJ6DpfEVPkPOQCprY84xm+wDRcKtNV7K5sNkW52G/EDiex5muzF/h5P4ZBjNgLOFTk/ihggq//F62VBiL5rjOeZgbxa3tg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 7d51f6fb-f6d7-473e-58f3-08d7a67e4e70 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jan 2020 18:49:30.6921 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: r9wrojikn1sPT//hFwvGhhJR4+9ZYYonZVuFjX7aVKP8ZdyRchuLD3kKFgWhkRR+/xkEIcmS6HaZodtZhqP4w1sxKy/xZQR1jlo4Img82tQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2718 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: increase number of qps to lcore_params X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > Currently only one qp will be used for one core. The number of qp= s > > > > > can be increased to match the number of lcore params. > > > > > > > > I don't really understand the purpose of that patch.... > > > > As I understand, all it does - unconditionally increases number of > > > > crypto-queues mapped to the same lcore. > > > > > > [Anoob] With the current code, we will have only crypto qp mapped to > > > one lcore. So even if you have large number of crypto qps, you would = be only > > using as much as the number of lcores. > > > > > > This is an inefficient model as a fat flow(say with large packet > > > sizes) on one eth queue hitting one core can starve another flow whic= h > > happens to hit the same core, because both flows would get queued to th= e > > same qp. > > > And, we cannot just randomly submit to multiple qps from the same cor= e > > > as then the ordering would be messed up. > > > > No-one suggesting that one I believe. > > > > So the best possible usage model would be to map one eth queue to one > > > crypto qp. That way, the core wouldn't be unnecessarily pipeline the = crypto > > processing. > > > > I doubt it is really the 'best possible usage model'. > > There could be cases when forcing lcore to use/manage more crypto-queue= s > > will lead to negative effects: perf drop, not enough crypto queues for = all eth > > queues, etc. > > If your HW/SW requires to match each eth queue with a separate crypto-q= ueue, > > then I think it should be an optional feature, while keeping default be= havior > > intact. >=20 > [Anoob] I think the question here is more to do with s/w crypto PMDs vs h= /w crypto PMDs. For s/w PMDs, having more queues doesn't > really make sense and for h/w PMDs it's better. Not always. If these queues belongs to the same device, sometimes it is faster to use j= ust one queue for device per core. HW descriptor status polling, etc. is not free. > I'll see how we can make this an optional feature. Would you be okay with= allowing such > behavior if the underlying PMD can support as many queues as lcore_params= ? >=20 > As in, if the PMD doesn't support enough queues, we do 1 qp per core. Wou= ld that work for you? I am not fond of idea to change default mapping method silently. My preference would be a new command-line option (--cdev-maping or so). Another thought, make it more fine-grained and user-controlled by extending eth-port-queue-lcore mapping option. from current: ,, to ,,,. So let say with 2 cores , 2 eth ports/2 queues per port for current mapping user would do: # use cdev queue 0 on all cdevs for lcore 6 # use cdev queue 1 on all cdevs for lcore 7 --lcores=3D"6,7" ... -- --config=3D"(0,0,6,0),(1,0,6,0),(0,1,7,1),(1,1,7,1)= " for the mapping you'd like to have: --lcores=3D"6,7" ... -- --config=3D"(0,0,6,0),(1,0,6,1),(0,1,7,2),(1,1,7,3)= " > Also, if we want to stick to 1 crypto qp per lcore, I don't understand wh= y we should have the following macro defined such, >=20 > #define MAX_QP_PER_LCORE 256 >=20 > > > > > > > > > The question is what for? > > > > All these extra queues woulnd't be used by current poll mode data-p= ath > > anyway. > > > > Plus in some cases hash_lookup() would fail to find existing mappi= ng. > > > > > > [Anoob] It was an issue with v1 that we have addressed with v2. I'll = send v2 > > shortly. Please do check that see if you have more comments. > > > > > > > I understand that for your eventdev implementation you need one > > > > crypto queue for each eth device. > > > > But I think it could be done in less intrusive way (see my previous > > > > mail as one possible option). > > > > > > [Anoob] If this suggestion is agreeable, then we can solve both qp > > > number requirement (for inline) and default nb_mbuf calculation in a > > > much more sensible way. If this doesn't fly, I'll probably go back to= your > > suggestion, but then there would be more code just to handle these case= s. And > > the nb_mbuf calculation could turn slightly complicated. > > > > Don't see how it affects number of required mbufs... > > Wouldn't it just be: > > * + > > * + > > *+.... >=20 > [Anoob] What would be num_of_crypto_queues? >=20 > > ? > > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > Signed-off-by: Anoob Joseph > > > > > --- > > > > > examples/ipsec-secgw/ipsec-secgw.c | 39 > > > > > +++++++++++++++++++-------------- > > > > ----- > > > > > examples/ipsec-secgw/ipsec.h | 4 +++- > > > > > 2 files changed, 23 insertions(+), 20 deletions(-) > > > > > > > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > > > > b/examples/ipsec-secgw/ipsec-secgw.c > > > > > index 3b5aaf6..d8c435e 100644 > > > > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > > > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > > > > @@ -1709,6 +1709,8 @@ add_mapping(struct rte_hash *map, const cha= r > > > > *str, uint16_t cdev_id, > > > > > unsigned long i; > > > > > struct cdev_key key =3D { 0 }; > > > > > > > > > > + key.port_id =3D params->port_id; > > > > > + key.queue_id =3D params->queue_id; > > > > > key.lcore_id =3D params->lcore_id; > > > > > if (cipher) > > > > > key.cipher_algo =3D cipher->sym.cipher.algo; @@ -1721,23 > > > > +1723,17 @@ > > > > > add_mapping(struct rte_hash *map, const char *str, uint16_t cdev_= id, > > > > > if (ret !=3D -ENOENT) > > > > > return 0; > > > > > > > > > > - for (i =3D 0; i < ipsec_ctx->nb_qps; i++) > > > > > - if (ipsec_ctx->tbl[i].id =3D=3D cdev_id) > > > > > - break; > > > > > - > > > > > - if (i =3D=3D ipsec_ctx->nb_qps) { > > > > > - if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > > - printf("Maximum number of crypto devices assigned to > > > > " > > > > > - "a core, increase MAX_QP_PER_LCORE > > > > value\n"); > > > > > - return 0; > > > > > - } > > > > > - ipsec_ctx->tbl[i].id =3D cdev_id; > > > > > - ipsec_ctx->tbl[i].qp =3D qp; > > > > > - ipsec_ctx->nb_qps++; > > > > > - printf("%s cdev mapping: lcore %u using cdev %u qp %u " > > > > > - "(cdev_id_qp %lu)\n", str, key.lcore_id, > > > > > - cdev_id, qp, i); > > > > > + i =3D ipsec_ctx->nb_qps; > > > > > + if (ipsec_ctx->nb_qps =3D=3D MAX_QP_PER_LCORE) { > > > > > + printf("Maximum number of crypto devices assigned to a core, " > > > > > + "increase MAX_QP_PER_LCORE value\n"); > > > > > + return 0; > > > > > } > > > > > + ipsec_ctx->tbl[i].id =3D cdev_id; > > > > > + ipsec_ctx->tbl[i].qp =3D qp; > > > > > + ipsec_ctx->nb_qps++; > > > > > + printf("%s cdev mapping: lcore %u using cdev %u qp %u " > > > > > + "(cdev_id_qp %lu)\n", str, key.lcore_id, cdev_id, qp, i)= ; > > > > > > > > > > ret =3D rte_hash_add_key_data(map, &key, (void *)i); > > > > > if (ret < 0) { > > > > > @@ -1785,8 +1781,10 @@ add_cdev_mapping(struct rte_cryptodev_info > > > > *dev_info, uint16_t cdev_id, > > > > > continue; > > > > > > > > > > if (i->sym.xform_type =3D=3D RTE_CRYPTO_SYM_XFORM_AEAD) { > > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, params, > > > > > + ret =3D add_mapping(map, str, cdev_id, qp, params, > > > > > ipsec_ctx, NULL, NULL, i); > > > > > + if (ret) > > > > > + return ret; > > > > > continue; > > > > > } > > > > > > > > > > @@ -1801,12 +1799,15 @@ add_cdev_mapping(struct rte_cryptodev_inf= o > > > > *dev_info, uint16_t cdev_id, > > > > > if (j->sym.xform_type !=3D > > > > RTE_CRYPTO_SYM_XFORM_AUTH) > > > > > continue; > > > > > > > > > > - ret |=3D add_mapping(map, str, cdev_id, qp, params, > > > > > + ret =3D add_mapping(map, str, cdev_id, qp, params, > > > > > ipsec_ctx, i, j, NULL); > > > > > + if (ret) > > > > > + return ret; > > > > > + continue; > > > > > } > > > > > } > > > > > > > > > > - return ret; > > > > > + return 0; > > > > > } > > > > > > > > > > /* Check if the device is enabled by cryptodev_mask */ diff --gi= t > > > > > a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h > > > > > index 8e07521..92fd5eb 100644 > > > > > --- a/examples/ipsec-secgw/ipsec.h > > > > > +++ b/examples/ipsec-secgw/ipsec.h > > > > > @@ -200,7 +200,9 @@ struct ipsec_ctx { }; > > > > > > > > > > struct cdev_key { > > > > > - uint16_t lcore_id; > > > > > + uint16_t port_id; > > > > > + uint8_t queue_id; > > > > > + uint8_t lcore_id; > > > > > uint8_t cipher_algo; > > > > > uint8_t auth_algo; > > > > > uint8_t aead_algo; > > > > > -- > > > > > 2.7.4