From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 2C9AEA04FA;
	Wed,  5 Feb 2020 14:42:22 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 0C5421C1E0;
	Wed,  5 Feb 2020 14:42:22 +0100 (CET)
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
 by dpdk.org (Postfix) with ESMTP id A09201C1B7
 for <dev@dpdk.org>; Wed,  5 Feb 2020 14:42:20 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga007.jf.intel.com ([10.7.209.58])
 by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 05 Feb 2020 05:42:19 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.70,405,1574150400"; d="scan'208";a="220105138"
Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6])
 by orsmga007.jf.intel.com with ESMTP; 05 Feb 2020 05:42:19 -0800
Received: from orsmsx158.amr.corp.intel.com (10.22.240.20) by
 ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 5 Feb 2020 05:42:19 -0800
Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by
 ORSMSX158.amr.corp.intel.com (10.22.240.20) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 5 Feb 2020 05:42:18 -0800
Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.171)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Wed, 5 Feb 2020 05:42:18 -0800
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=Wj3Bcxw2to1C6kLrwoAohpOAmdqSBSA7AFJ1OcBmViphctP5dlG94zv0r+56zUolYrHpycPFtPbYF15W+z6eZsEKeYalhJGS0Wc2ckHmQzvGf67yZ0Nv+fufSVD67XLzFTSRibHi4cSVkjqEdFIGihcZ56v2g2ASlmfFWqWEfGXkwcR2d0FaCIrG0TiVqeEsiIfv4QpTMcrRFhgd08SvJi3nOOueTEToXIGWhd/nWagYfROEJwCidH0vZqfigwjO56xUY256gLiSpYMdzlK7qvlDopzDvm6mZQnO6RbG2/BlQP8NpTl0YEkRKxc4FOLPbxhVxP5p8o7nKkCxipg7XQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VSdJFsqmHyK0UcSq7Ny/yCozbl42XMmb5iRoo5hkOmU=;
 b=QDrJODpEOcxqRIXOdRgPFvTI3yZUdW5jKVsrp/aGzSZhgisDlKp+lqYztLYtDUGmtU/vs5m7jj/Ax4dpHPxw+QXqeMowqxuV0YbM3Ui/tlqblhyq3hh6QaQvTaefB2uZixNkxqgA6J3kT3oO7YVsL88eGlpvMIHEwPmbZBwEbcSC1tTLTcHLiURxOx0NsdoOjEUgRc7n0fD2hIExrG9m18zMzBM72IlAVjZgf+274RHSDXRhA8Yg4sB26j/PFSKS6Wez/fHMlbv8XD147XyDB4reanYjdRRnxme9Td5IrELd1iCrSzgY1JEOOrCbPZLmXICFF1F45P8rMNdCVSXOWg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; 
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=VSdJFsqmHyK0UcSq7Ny/yCozbl42XMmb5iRoo5hkOmU=;
 b=KEiUMMt1UvzZ/N5lW2FGJ4sQpeJAXFUSXWjjVgcFsGomG+wdWtcW9VTyfGOCwF+iSjNA31Om8ggGSUzgL4Ol+lWDPFgWsyEdOZmji+aFOlH4A5TJQHXd4sMefd5BKppqfi1S1fKmr2bsDqKdKHuwU4a9EJR8KQZR8ayJReeZ3UA=
Received: from DM6PR11MB2556.namprd11.prod.outlook.com (20.176.99.10) by
 DM6PR11MB3113.namprd11.prod.outlook.com (20.177.218.83) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.2686.32; Wed, 5 Feb 2020 13:42:16 +0000
Received: from DM6PR11MB2556.namprd11.prod.outlook.com
 ([fe80::513e:4aed:18d0:9b35]) by DM6PR11MB2556.namprd11.prod.outlook.com
 ([fe80::513e:4aed:18d0:9b35%7]) with mapi id 15.20.2686.031; Wed, 5 Feb 2020
 13:42:16 +0000
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: Lukasz Bartosik <lbartosik@marvell.com>, Akhil Goyal
 <akhil.goyal@nxp.com>, "Nicolau, Radu" <radu.nicolau@intel.com>, "Thomas
 Monjalon" <thomas@monjalon.net>
CC: Jerin Jacob <jerinj@marvell.com>, Narayana Prasad <pathreya@marvell.com>, 
 Ankur Dwivedi <adwivedi@marvell.com>, Anoob Joseph <anoobj@marvell.com>,
 Archana Muniganti <marchana@marvell.com>, Tejasree Kondoj
 <ktejasree@marvell.com>, Vamsi Attunuru <vattunuru@marvell.com>,
 "dev@dpdk.org" <dev@dpdk.org>
Thread-Topic: [PATCH v3 13/13] examples/ipsec-secgw: make number of buffers
 dynamic
Thread-Index: AQHV22Nu4zafD+zrQUentPyG8EEeoqgMnCVg
Date: Wed, 5 Feb 2020 13:42:16 +0000
Message-ID: <DM6PR11MB2556361E5908DABC0BD3D4CA9A020@DM6PR11MB2556.namprd11.prod.outlook.com>
References: <1579527918-360-1-git-send-email-anoobj@marvell.com>
 <1580824721-21527-1-git-send-email-lbartosik@marvell.com>
 <1580824721-21527-14-git-send-email-lbartosik@marvell.com>
In-Reply-To: <1580824721-21527-14-git-send-email-lbartosik@marvell.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOWNlYjI0NGUtNjk3MS00ZDRmLTkwMDQtYjNkYmRhNTVmNmNlIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoibzkyWnhzbkw0YjMwZ3JZaUluXC9MWDVUelwvSGpNVzN4YjJ6UUVUVno4K2xqQzlIcjZhUzBWOFwvS1VlZExobm5IYyJ9
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.2.0.6
x-ctpclassification: CTP_NT
authentication-results: spf=none (sender IP is )
 smtp.mailfrom=konstantin.ananyev@intel.com; 
x-originating-ip: [192.198.151.186]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 77aea9ec-54b6-41ec-665b-08d7aa413706
x-ms-traffictypediagnostic: DM6PR11MB3113:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <DM6PR11MB3113E402561A07BF913BD7489A020@DM6PR11MB3113.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:182;
x-forefront-prvs: 0304E36CA3
x-forefront-antispam-report: SFV:NSPM;
 SFS:(10019020)(366004)(189003)(199004)(66556008)(66446008)(66476007)(7416002)(64756008)(52536014)(33656002)(66946007)(76116006)(5660300002)(9686003)(4326008)(110136005)(7696005)(54906003)(55016002)(186003)(26005)(8936002)(81166006)(81156014)(498600001)(86362001)(6506007)(71200400001)(8676002)(2906002);
 DIR:OUT; SFP:1102; SCL:1; SRVR:DM6PR11MB3113;
 H:DM6PR11MB2556.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en;
 PTR:InfoNoRecords; MX:1; A:1; 
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: eixvJKwEoFiqMFD2Ju5gfG+ZLAxyCylvMfqROVSuvXuGEgmJHoMJFSJygC54aYFtUkp3xgYIPKa+ep7CHU2+gRsOodKqlpj1SI5et6N5VK1fFxiVfopT/zD35riglADlCxXFGEi2Wh2IIcGtrs4FTT/JqrqRIOyRgfKxjwVxNKdrEZIVH3VsgqTlddgQ9mfS8d6paO42Crb93IGui/PdFHlecmHMOCbBP0aJoZ3c/juTLWxV4aUZ78dq924HXHWi3R4I+7sN+TWIvx6FvNkKzuaCnMZ2G2m6nBLoQEgX+zdk5uc9AdavdsaqJEHwGQ3PlVpMM2nTppAXj/nFF3tqmMSAraUcEiZXWaulJ1XWVwGlyljNcqDEJB729rOcabyyFjif305jVtWjBgvxaDLVv+IHEmnh5Vqtelx/MKH2zFc6UZx1mw0md4qwtPPIiGmJ
x-ms-exchange-antispam-messagedata: myIrGwj50v2l3WEFUZ0xbu53AQbWfsIJvt1Bx0mGpAsxdQleghTswW0ubGGvhmIKL4HvHPxRBWG8LlHlLTbcZpKFhXy2Bp5svoGAM7tBmOL8AnQeZHnz+OySlx4w2JfogxUXSJyk6T9rRZyxUrZyPg==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 77aea9ec-54b6-41ec-665b-08d7aa413706
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Feb 2020 13:42:16.7697 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Q7gz5e3ugVJat2+9T4kp91qwqHtHG6ZNlTqT1628lviC6r++YImDJvc2jC6Dz61qXOrEWenOnJ6Y73d2xvHoO22hr3XMJDbH3t0xrq33gq4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB3113
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH v3 13/13] examples/ipsec-secgw: make number
 of buffers dynamic
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>


Hi Lukasz,
=20
> Make number of buffers in a pool nb_mbuf_in_pool dependent on number
> of ports, cores and crypto queues. Add command line option -s which
> when used overrides dynamic calculation of number of buffers in a pool.
>=20
> Signed-off-by: Anoob Joseph <anoobj@marvell.com>
> Signed-off-by: Lukasz Bartosik <lbartosik@marvell.com>
> ---
>  examples/ipsec-secgw/ipsec-secgw.c | 59 +++++++++++++++++++++++++++++++-=
------
>  1 file changed, 48 insertions(+), 11 deletions(-)
>=20
> diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ip=
sec-secgw.c
> index 862a7f0..f7acb52 100644
> --- a/examples/ipsec-secgw/ipsec-secgw.c
> +++ b/examples/ipsec-secgw/ipsec-secgw.c
> @@ -59,8 +59,6 @@ volatile bool force_quit;
>=20
>  #define MEMPOOL_CACHE_SIZE 256
>=20
> -#define NB_MBUF	(32000)
> -
>  #define CDEV_QUEUE_DESC 2048
>  #define CDEV_MAP_ENTRIES 16384
>  #define CDEV_MP_NB_OBJS 1024
> @@ -163,6 +161,7 @@ static int32_t promiscuous_on =3D 1;
>  static int32_t numa_on =3D 1; /**< NUMA is enabled by default. */
>  static uint32_t nb_lcores;
>  static uint32_t single_sa;
> +static uint32_t nb_bufs_in_pool;
>=20
>  /*
>   * RX/TX HW offload capabilities to enable/use on ethernet ports.
> @@ -1259,6 +1258,7 @@ print_usage(const char *prgname)
>  		" [-w REPLAY_WINDOW_SIZE]"
>  		" [-e]"
>  		" [-a]"
> +		" [-s NUMBER_OF_MBUFS_IN_PKT_POOL]"
>  		" -f CONFIG_FILE"
>  		" --config (port,queue,lcore)[,(port,queue,lcore)]"
>  		" [--single-sa SAIDX]"
> @@ -1280,6 +1280,7 @@ print_usage(const char *prgname)
>  		"     size for each SA\n"
>  		"  -e enables ESN\n"
>  		"  -a enables SA SQN atomic behaviour\n"
> +		"  -s number of mbufs in packet pool (default 8192)\n"
>  		"  -f CONFIG_FILE: Configuration file\n"
>  		"  --config (port,queue,lcore): Rx queue configuration\n"
>  		"  --single-sa SAIDX: In poll mode use single SA index for\n"
> @@ -1479,7 +1480,7 @@ parse_args(int32_t argc, char **argv, struct eh_con=
f *eh_conf)
>=20
>  	argvopt =3D argv;
>=20
> -	while ((opt =3D getopt_long(argc, argvopt, "aelp:Pu:f:j:w:",
> +	while ((opt =3D getopt_long(argc, argvopt, "aelp:Pu:f:j:w:s:",
>  				lgopts, &option_index)) !=3D EOF) {
>=20
>  		switch (opt) {
> @@ -1513,6 +1514,19 @@ parse_args(int32_t argc, char **argv, struct eh_co=
nf *eh_conf)
>  			cfgfile =3D optarg;
>  			f_present =3D 1;
>  			break;
> +
> +		case 's':
> +			ret =3D parse_decimal(optarg);
> +			if (ret < 0) {
> +				printf("Invalid number of buffers in a pool: "
> +					"%s\n", optarg);
> +				print_usage(prgname);
> +				return -1;
> +			}
> +
> +			nb_bufs_in_pool =3D ret;
> +			break;
> +
>  		case 'j':
>  			ret =3D parse_decimal(optarg);
>  			if (ret < RTE_MBUF_DEFAULT_BUF_SIZE ||
> @@ -1876,12 +1890,12 @@ check_cryptodev_mask(uint8_t cdev_id)
>  	return -1;
>  }
>=20
> -static int32_t
> +static uint16_t
>  cryptodevs_init(void)
>  {
>  	struct rte_cryptodev_config dev_conf;
>  	struct rte_cryptodev_qp_conf qp_conf;
> -	uint16_t idx, max_nb_qps, qp, i;
> +	uint16_t idx, max_nb_qps, qp, total_nb_qps, i;
>  	int16_t cdev_id;
>  	struct rte_hash_parameters params =3D { 0 };
>=20
> @@ -1909,6 +1923,7 @@ cryptodevs_init(void)
>  	printf("lcore/cryptodev/qp mappings:\n");
>=20
>  	idx =3D 0;
> +	total_nb_qps =3D 0;
>  	for (cdev_id =3D 0; cdev_id < rte_cryptodev_count(); cdev_id++) {
>  		struct rte_cryptodev_info cdev_info;
>=20
> @@ -1942,6 +1957,7 @@ cryptodevs_init(void)
>  		if (qp =3D=3D 0)
>  			continue;
>=20
> +		total_nb_qps +=3D qp;
>  		dev_conf.socket_id =3D rte_cryptodev_socket_id(cdev_id);
>  		dev_conf.nb_queue_pairs =3D qp;
>  		dev_conf.ff_disable =3D RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO;
> @@ -1974,7 +1990,7 @@ cryptodevs_init(void)
>=20
>  	printf("\n");
>=20
> -	return 0;
> +	return total_nb_qps;
>  }
>=20
>  static void
> @@ -2607,16 +2623,18 @@ int32_t
>  main(int32_t argc, char **argv)
>  {
>  	int32_t ret;
> -	uint32_t lcore_id;
> +	uint32_t lcore_id, nb_txq, nb_rxq =3D 0;
>  	uint32_t cdev_id;
>  	uint32_t i;
>  	uint8_t socket_id;
> -	uint16_t portid;
> +	uint16_t portid, nb_crypto_qp, nb_ports =3D 0;
>  	uint64_t req_rx_offloads[RTE_MAX_ETHPORTS];
>  	uint64_t req_tx_offloads[RTE_MAX_ETHPORTS];
>  	struct eh_conf *eh_conf =3D NULL;
>  	size_t sess_sz;
>=20
> +	nb_bufs_in_pool =3D 0;
> +
>  	/* init EAL */
>  	ret =3D rte_eal_init(argc, argv);
>  	if (ret < 0)
> @@ -2665,6 +2683,26 @@ main(int32_t argc, char **argv)
>=20
>  	sess_sz =3D max_session_size();
>=20
> +	nb_crypto_qp =3D cryptodevs_init();
> +
> +	if (nb_bufs_in_pool =3D=3D 0) {
> +		RTE_ETH_FOREACH_DEV(portid) {
> +			if ((enabled_port_mask & (1 << portid)) =3D=3D 0)
> +				continue;
> +			nb_ports++;
> +			nb_rxq +=3D get_port_nb_rx_queues(portid);
> +		}
> +
> +		nb_txq =3D nb_lcores;
> +
> +		nb_bufs_in_pool =3D RTE_MAX((nb_rxq*nb_rxd +
> +					   nb_ports*nb_lcores*MAX_PKT_BURST +
> +					   nb_ports*nb_txq*nb_txd +
> +					   nb_lcores*MEMPOOL_CACHE_SIZE +
> +					   nb_crypto_qp*CDEV_QUEUE_DESC),

I think you forgot to take into account possible reassemble table:
@@ -2699,7 +2699,9 @@ main(int32_t argc, char **argv)
                                           nb_ports*nb_lcores*MAX_PKT_BURST=
 +
                                           nb_ports*nb_txq*nb_txd +
                                           nb_lcores*MEMPOOL_CACHE_SIZE +
-                                          nb_crypto_qp*CDEV_QUEUE_DESC),
+                                          nb_crypto_qp*CDEV_QUEUE_DESC +
+                                          nb_lcores * frag_tbl_sz *
+                                          FRAG_TBL_BUCKET_ENTRIES),
                                          8192U);


Also it might be worth for better readability to put code for nb_bufs_in_po=
ol calculation
in a separate  function (and add spaces between '*' and its' operands).
Apart from that - whole series LGTM.
Konstantin


> +					  8192U);
> +	}
> +
>  	for (lcore_id =3D 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
>  		if (rte_lcore_is_enabled(lcore_id) =3D=3D 0)
>  			continue;
> @@ -2678,11 +2716,12 @@ main(int32_t argc, char **argv)
>  		if (socket_ctx[socket_id].mbuf_pool)
>  			continue;
>=20
> -		pool_init(&socket_ctx[socket_id], socket_id, NB_MBUF);
> +		pool_init(&socket_ctx[socket_id], socket_id, nb_bufs_in_pool);
>  		session_pool_init(&socket_ctx[socket_id], socket_id, sess_sz);
>  		session_priv_pool_init(&socket_ctx[socket_id], socket_id,
>  			sess_sz);
>  	}
> +	printf("Number of mbufs in packet pool %d\n", nb_bufs_in_pool);
>=20
>  	RTE_ETH_FOREACH_DEV(portid) {
>  		if ((enabled_port_mask & (1 << portid)) =3D=3D 0)
> @@ -2694,8 +2733,6 @@ main(int32_t argc, char **argv)
>  				req_tx_offloads[portid]);
>  	}
>=20
> -	cryptodevs_init();
> -
>  	/*
>  	 * Set the enabled port mask in helper config for use by helper
>  	 * sub-system. This will be used while initializing devices using
> --
> 2.7.4