From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 57A7FA0A0C for ; Fri, 2 Jul 2021 00:08:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1A4AB40141; Fri, 2 Jul 2021 00:08:35 +0200 (CEST) Received: from APC01-HK2-obe.outbound.protection.outlook.com (mail-eopbgr1300081.outbound.protection.outlook.com [40.107.130.81]) by mails.dpdk.org (Postfix) with ESMTP id 203594003E for ; Fri, 2 Jul 2021 00:08:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tatacommunications.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9K5/DOP4B70puQT/e+g5oWfd8gwbyqa5dzHfdxSCsd4=; b=QodJfDVdqudVRGPCGL2LTkGLuxYUYfZ55XsDmwWwSqb16FF/X0RZppExufRESgaqdN/8ZlnDBfuAK+HFQkUfhNV0vv+qAtJobihIp3h5jy2YKBN3njOlUFK1BSNMpxs/KnsTKT7CQR+MmYXB/TqNDX1Oz7jfKfzanY/DtEMTNhAN6dDPWJZf+kBFogYAWsvPq6TTZkZott9XgaVOfLVTA/zH576JiVVbE/V25giMgTBuzIHncH2Hz4O1ryojdWQn8wRnH0G4+PHzcGHIND9QCc9GaJ4DrsazIeUiwhrT49x8Us1Rr4wkNBoHoLPvEFnAsJwd1UvmesaWqQh8WIIghA== Received: from SG2PR04CA0175.apcprd04.prod.outlook.com (2603:1096:4:14::13) by KL1PR0401MB2167.apcprd04.prod.outlook.com (2603:1096:802:5::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.23; Thu, 1 Jul 2021 22:08:31 +0000 Received: from SG2APC01FT053.eop-APC01.prod.protection.outlook.com (2603:1096:4:14:cafe::da) by SG2PR04CA0175.outlook.office365.com (2603:1096:4:14::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 22:08:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 54.79.123.149) smtp.mailfrom=tatacommunications.com; dpdk.org; dkim=pass (signature was verified) header.d=tatacommunications.com;dpdk.org; dmarc=pass action=none header.from=tatacommunications.com; Received-SPF: Pass (protection.outlook.com: domain of tatacommunications.com designates 54.79.123.149 as permitted sender) receiver=protection.outlook.com; client-ip=54.79.123.149; helo=sydapipop-haraka-inline01.vpc-a3a96cc4.ase2.shn; Received: from sydapipop-haraka-inline01.vpc-a3a96cc4.ase2.shn (54.79.123.149) by SG2APC01FT053.mail.protection.outlook.com (10.152.250.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Thu, 1 Jul 2021 22:08:30 +0000 Received: from APC01-HK2-obe.outbound.protection.outlook.com (mail-hk2apc01lp2050.outbound.protection.outlook.com [104.47.124.50]) by sydapipop-haraka-inline01.vpc-a3a96cc4.ase2.shn (Haraka/2.8.24) with ESMTPS id 96C09680-1288-471A-8A4D-EBF88C18FAE8.1 envelope-from (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 verify=FAIL); Thu, 01 Jul 2021 22:08:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BEj5j9lsG+hr3wLVthgeRnx8muQTuMPVx1VQScRy/gZtEH+RHL2SURBgMUyET79updycwzDIskAL8h/R3UNqWlJ4TMMY15V2swjrEPKUMleFjPKcF5cO9aJP7gV/KzDW6N0fbDXiWjqHAM+aWiFHw3A/EKJM1SL+lzeARSTy1uc5lz99GgLOMdTX1OEz+8BwGVDFT4hLUXRBpbPFoYSWBTH1Udm755ijKPFqOioryAIv0BR0789JBdYvSHGvUBP5/Xx07Sh/5LB2WMMwI3WPWWf0nLzLH8mXWq2Lg77BKupXQkLNf3d+yEdC3aEUTU0pRG6VY/malqBuy7CKV4+bAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FyfLMvnHhac3cfSs3TjKb5/w8D28cKemQWPxO0yEZWo=; b=TuXQSRHGejw+X+YhBzjyBXsFXEv9xLNp0dTUb+3Cr3OGU5uD0ofSulLRrFrJGYQFQhDolv8jRpgAGCMKMu/b4O4kLRUlgjnPae96UGvSxGWzgTDbu5K0H7dUxa9J05ftk62m5sEUIHGOjwsWa5idWGkvW3ceO5G1Jvbmp2bGcXtAeoTbVb6G3LfNkY1XWZklGNSJj60byYj8MEfCSUw9zn4zpmeTW5eT6ElcyhmEUc7ahPWIN6Kd/wVpyia54lVUYHRqUrYWxFx3wZC5UYxSoA44C+hEpsj5Y6NRM0wqc3jiqnqYpCIG5YMD/MA9eWFOsyvKvaw88MZGsHuk4OqKfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=tatacommunications.com; dmarc=pass action=none header.from=tatacommunications.com; dkim=pass header.d=tatacommunications.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tatacommunications.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FyfLMvnHhac3cfSs3TjKb5/w8D28cKemQWPxO0yEZWo=; b=t+HFHEg82wd6vdmWtR9YIklE1gucZMRtPcHZ0yuAGoUZxyn0AcIP4EIO0GkOp73r4T432hAlmG4UiNhrJpRU4ugIOWlhw3tLmgj4j8tVaMiujK4B/p9VoS/6/oE1AZZb6S3dIxioMGl/ROtBYl1H7l843ZxMI/vCr+kEGBChjoDYMllhsau3PbCaoWm+0hvLYbjUPNfGirPpgF+Iha1Gqen9mm6z2YvZLmCk0yTk2jNfYEBR9Vhr6wUOX6u1zyHd1qJ9RJ6+gfmS8SncrJSeNDO5iZLmzqEY20awUshgPpUyfHyTMswmOKWrsXDr9r9+jlfiQ0fZOkH0pqOMfA0DRw== Received: from SG2PR04MB3173.apcprd04.prod.outlook.com (2603:1096:4:69::22) by SG2PR04MB2857.apcprd04.prod.outlook.com (2603:1096:4:20::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.24; Thu, 1 Jul 2021 22:08:25 +0000 Received: from SG2PR04MB3173.apcprd04.prod.outlook.com ([fe80::8c36:5206:c7d9:8e99]) by SG2PR04MB3173.apcprd04.prod.outlook.com ([fe80::8c36:5206:c7d9:8e99%4]) with mapi id 15.20.4264.031; Thu, 1 Jul 2021 22:08:25 +0000 From: Vishal Mohan To: "users@dpdk.org" Thread-Topic: IPv4 Fragmentation - indirect pool gets exhausted Thread-Index: Adduw5Vm3uvS7y3jSNCJF8giBtrfXg== Date: Thu, 1 Jul 2021 22:08:24 +0000 Message-ID: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_ActionId=ea49fb2e-c2c5-40e1-b1df-d261b158d42b; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_ContentBits=0; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_Enabled=true; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_Method=Privileged; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_Name=Public; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_SetDate=2021-07-01T22:08:23Z; MSIP_Label_6e8c6461-2dc9-4610-ad74-a9c76638efab_SiteId=20210462-2c5e-4ec8-b3e2-0be950f292ca; Authentication-Results-Original: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=tatacommunications.com; x-originating-ip: [122.164.165.90] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 06bc330e-b574-4e88-49f3-08d93cdcc2ad x-ms-traffictypediagnostic: SG2PR04MB2857:|KL1PR0401MB2167: X-Microsoft-Antispam-PRVS: x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: yKNu//M3wMTxkkGmwuerbIoiiZB8jbpKZ77ZDrhJwAoMpDoLSVHU8nGitcu14DlHVhjQDH8VjCbZXErg5BaXis/28cElF+odBGbySMGikC8MDHnUWwxUrMz7QepY0M8BQdBo8TlhThXNighJ+L3MRniVAzbkJ7TUn2+Y0xOr/C42us9QmgzO7Xj7YpTdK3LHMHbaTZYrduFw/tcnk5zOHSKENRaw0V8vqIMh2ujG1AUpjnmCr58E5VMiNd6DXMP9oxWnbIcRsh0NbFFIBexKaY/m0VaOcIxQVFJ6AAVvl4MOSreDoIgf26IMqKi58FcZwmn0Vfnym33+Cibvqe9DnakOx+lJLQOfD2UjzZchaCy7VBrCXXI4lap+5BaJh/DYNbX8klIvBKlPqTbWoo39hE7xVBQfAxHhrw51dNgTspYqhqtI1uTgVcYp6Zq8BQmhLGFB7g3DDUgZwO7kIuK1QwNGiEyIB0qW5bYeRALSWE3FSgvvaO/nMLgQ8xQ346nqkZhPR2GFKs5ebkopElyEbOT/vxnWbwe8B/cF/ZgfurgbzTBKCaU62vZssrHwchMecwBwxJGy8XVYhfZoDuZuXokHYJ5m88NL1AtQnp8dRLW8Y0vxmubxxmceWnzl/mMTrguF2C7HPWzxx073jm0aOwLAyG7IjiOHv7SqNQ5Nd/zV35JVhGx4FmpcWjVOLt7vuA4A1OD38X0Vru7ZaJBis4XjjEf5xkJ6QOAI/gMmgHJkah4Pz2qE28bxNiSFtIitmqQ7HuViFTQV3UqnyY8skA== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SG2PR04MB3173.apcprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(39850400004)(366004)(346002)(396003)(122000001)(8676002)(66446008)(26005)(6916009)(64756008)(66946007)(33656002)(186003)(76116006)(316002)(66476007)(2906002)(5660300002)(9326002)(478600001)(66556008)(71200400001)(44832011)(55016002)(8936002)(7696005)(9686003)(166002)(86362001)(55236004)(83380400001)(6506007)(38100700002)(52536014); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?R9QYF9QMNmUnohvhHhmynsTfAVmrWpjSdn0m+lygaqDAB4LPTr6xWXCUCtDD?= =?us-ascii?Q?bir49AvGLDYzwUles2XlPMF3/xczZuv93ZjrI4kgffBilw+g2nTI6SI82k+z?= =?us-ascii?Q?Hup/dR/bhb8QZcQH+yaFNq2/kSLPZ0xuU4jmyejygTyEC9vN+Z2kJBUu7dA8?= =?us-ascii?Q?+qinwvp33PwrXMXPDxnqn6iIuLMuUYZWcuIMCU6lSqNH15eKwamD0GOonmX/?= =?us-ascii?Q?Hh4qlO+oCXchNaPMT/bQ7WQljcEojjCPouzuwpzoIJhrVwg6iSqc0qj+IgrI?= =?us-ascii?Q?r3w8G5lbOuGpRwt11/wyoeIBexkDJfcbN2vIJ7rSFqtAkEkfj1c40X0RDUsk?= =?us-ascii?Q?CoLs0KxTeE8d17lStf9Bo+2fcMyyJ6LWVQaU3L8rww1uFCSMPxpvXV/PlE7g?= =?us-ascii?Q?ciZ+rPHJMVWfn2lGwRQ0+8+vuZC0F4cJirYvPHIByIWx+n99W+oweYdgVfVP?= =?us-ascii?Q?MOlGpQScvqf764fYGagkKFElNYk+kP/34ZwUFAIdeRsbOvtGtScCliTqXg6R?= =?us-ascii?Q?MryRLoNzR8fmDojJtVrIYURMsefJjUIut+zKIDlnGOx78r6JqKa6EkMo/zL0?= =?us-ascii?Q?T4aib4Bqw1D75zZ+jqVKqIWRyJWl7S02kSEnxhNeWPqI9rRwt4g/VWTPYcqo?= =?us-ascii?Q?jIGGHqu4axhbSav/hGiFGWpUh3zvcW//Imy2klZvXUBOSvImICe2rcH40yv7?= =?us-ascii?Q?n/ydkAb9R/8cO5oRffKEMYNoKZvIKAQs8HECXQOCzNS3QXco+H1lCLrwq0zO?= =?us-ascii?Q?IpAjjiYDufCGAxb3v0lZ6tSW6kPsdZt6I1FBNcvejis8ZMBsUpIW/0/y495C?= =?us-ascii?Q?a7M4EmgxfYuwyo/2LUbd/o9oISigmp1mZlAl02tscCCYmPdhBE4/8zzGFlL7?= =?us-ascii?Q?bTJqVlAjtbDQx0bc8c0FnoDQIcUplucjqgnkFp3mBOqe1Zv1Ub4H9K9hDTas?= =?us-ascii?Q?vGTWXQwnhBZdIwDMDcyPFH0jf+nQQudJevuIzhakTNxq6jQw/H8HGB2NpaqL?= =?us-ascii?Q?LNqnzfQ7O48AGhJTnUZG8e6O4CupUhmS7b1nHziuUYq84/zX3aH96TxoTshH?= =?us-ascii?Q?myE8Npa1Fdy29kLlJBdjPtB8amNGKkNpD6G7GXohFEfndGJ86uqhSj/G8r+6?= =?us-ascii?Q?MCXxiDd6+BFixuQCVXWMson4vzfbQC44lO0T4WA1FqxCZzty/E1LRGL+QNOP?= =?us-ascii?Q?T22Ftazb4VX7I206K/4t3wV98rCkKPOa/WsF9TwXTdk4tfR4vQm94loZ57IW?= =?us-ascii?Q?2pVsFhX84S9oVZvzkMRXPNuYtSRAOq6TiS04GJgJX4MOydxa2UILlaGh3Y2O?= =?us-ascii?Q?4bI=3D?= x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR04MB2857 Original-Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=tatacommunications.com; X-SHN-DLP-SCAN: success X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: SG2APC01FT053.eop-APC01.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: b849bb18-2da3-4370-258d-08d93cdcbf62 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +G95JmnnN75r7Xz4jUd39bLrKl8XvMvnlL0r4gfSlGDrzbvwcIFMjFR0udB3z5xnZrVSa/JJpIaeuVEKyTuLDLXPoPnHJmcwA4HkW1f8Sm7m2z7reM8PGr3l2QLBVCQwTX97PAAD+rvH4bKPzDBt9FstoQBOWI8cSZoFIYWn1pcq93jzoLQNO9paeuP5jGvky0e0Wn3HPM4zEDJonaBW7OUtreJVAoYYk/j2/cSzFAx2FmV6skGyCmgnmt44ve+npUmyKeXtXmNDCQRd3+XGHjfhmIqR8ja6oDzaLk7U8e5IW57HVIEBaclwuoHLIWgheHiQoaaqh86igRaU1ePxna7vQK6sSoCarar8PFffDK8tFMSMPbxDDsUnkrvyT/twPyjuOQR/yMjh0Z241wfJf9gVoYGhCfOMwrYKEDiHq6LkHKOSdj4OklW9oEouxh5Lc2Pj404HcWrPHRLlSrIAmN/omCbUnm5i23haX+3uwHbF9MTaDWoKTHkhiAInZ0todtrDl3wZWkwqVfaaW0lFkmJJbWTzXhW0Uko1Pc+5nxD7swE0nCyM094R4InFKFl7HTgNpRmCHooFV4v8TSRj8iis0HNuYE3GkZGLv8ePE5F1+PBabbVxxChpJJ1grLSXINL/cHjay3FRFFQkVNc4EJKF0oxHbTILx2rZMzSx6gCbvxnEj69sQ5mbUJ2b4NQ6QXkDfq3ZwuBxFi3NaYFE4lbBUredbaeYbO54WSX1gfWWZ4yegpgkPzpe0GjzoKFK+mU9LhVtOUnm/mWegQIjuajo7qissB8dC+H1WnJAteQu4BJ4dfVyzOWC2qGJiqop3lolNfCs5C0ldZvjBrJqpg== X-Forefront-Antispam-Report: CIP:54.79.123.149; CTRY:AU; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:sydapipop-haraka-inline01.vpc-a3a96cc4.ase2.shn; PTR:send1.mail-inline.syd-pop.i-shn.net; CAT:NONE; SFS:(4636009)(39850400004)(376002)(346002)(136003)(396003)(36840700001)(46966006)(8936002)(44832011)(8676002)(52536014)(86362001)(36906005)(33656002)(7696005)(55016002)(36860700001)(316002)(9326002)(83380400001)(5660300002)(186003)(2906002)(82740400003)(6916009)(9686003)(70586007)(26005)(81166007)(478600001)(47076005)(82310400003)(356005)(55236004)(70206006)(166002)(6506007)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: tatacommunications.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jul 2021 22:08:30.2614 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06bc330e-b574-4e88-49f3-08d93cdcc2ad X-MS-Exchange-CrossTenant-Id: 20210462-2c5e-4ec8-b3e2-0be950f292ca X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=20210462-2c5e-4ec8-b3e2-0be950f292ca; Ip=[54.79.123.149]; Helo=[sydapipop-haraka-inline01.vpc-a3a96cc4.ase2.shn] X-MS-Exchange-CrossTenant-AuthSource: SG2APC01FT053.eop-APC01.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1PR0401MB2167 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [dpdk-users] IPv4 Fragmentation - indirect pool gets exhausted X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, I m trying to fragmentation an IPv4 packet using the below logic: ///////////////////////////////////////////////////////////////////// ~after pkts ingress~ struct rte_port_ring_writer *p =3D port_out->h_port; pool_direct =3D rte_mempool_lookup("MEMPOOL0"); pool_indirect =3D rte_mempool_lookup("MEMPOOL1"); printf("before frag mempool size d %d in %d\n",rte_mempool_avail_count(pool= _direct),rte_mempool_avail_count(pool_indirect)); struct rte_mbuf *frag_pkts[MAX_FRAG_SIZE]; int out_pkts =3D rte_ipv4_fragment_packet (m, frag_pkts, n_frags, ip_mtu= , = pool_direct, pool_indirect); printf("after frag mempool size d %d in %d\n",rte_mempool_avail_count(pool_= direct),rte_mempool_avail_count(pool_indirect)); if(out_pkts > 0) port_out->ops.f_tx_bulk(port_out->h_port,frag_pkts,RTE_LEN2MASK(out_pkts, u= int64_t)); else printf("frag failed\n"); rte_pktmbuf_free(m); ///////////////////////////////////////////////////////////////////// Now the problem here is the indirect mempool gets exhausted. As a result af= ter few burst of packets the fragmentation fails due to -ENOMEM. I quite ca= nnot understand why the PMD doesn't free and put back the mempool obj back = to MEMPOOL1. Please find the log below for the above snippet which prints t= he available slots in direct (d) and indirect (in) mempools: before frag mempool size d 2060457 in 2095988 after frag mempool size d 2060344 in 2095952 before frag mempool size d 2060361 in 2095945 after frag mempool size d 2060215 in 2095913 . . . before frag mempool size d 2045013 in 0 after frag mempool size d 2045013 in 0 before frag mempool size d 2045013 in 0 after frag mempool size d 2045013 in 0 before frag mempool size d 2045013 in 0 I can see the direct mempool reduce and increase as packets ingress and dro= p/egress as expected. I can also confirm I receive the initial burst of fra= gmented packets equal to MEMPOOL1 size. Any inputs towards understanding th= e cause of the problem is much appreciated. P.S: We had the same problem in dpdk17.11. We had to refractor the rte_ipv4= _fragment_packet() to not use indirect chaining of frags instead just gener= ate them. Thanks & Regards, Vishal Mohan Tata Communications - Public Tata Communications - Public