From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B6CEDA00BE for ; Tue, 15 Mar 2022 16:47:20 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 883704014F; Tue, 15 Mar 2022 16:47:20 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 1F57A4014F for ; Tue, 15 Mar 2022 16:47:19 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 22FCT7oQ007722 for ; Tue, 15 Mar 2022 08:47:18 -0700 Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2048.outbound.protection.outlook.com [104.47.66.48]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3et64agffc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 15 Mar 2022 08:47:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nlNri4Kgxgrf8ymVNRLL3jWVB70vZp1gWfuVk4tSOGAUdzOQ7J/TRgG91QyTmRx2VSmlRtUKdG4qaB4VaTEUdNBENto5wLfwf9c8R7CLP75zqMzXo2kll6mtyKJnbF1acQg+Wx9gV6oRLO8cAy+MPGJQvbRDqnfvmlaUvGUm/wEwCjWVMFLlVnZxdLQX6a63KBftcDB7pHhv+A5zfnS7RmGddawPRM6WjfXYh+xbd9jRRTNhnkPURzZUO0TXiAVzkK/CvUPqXT3QFvr/BDAhZv9fqPDTNhX39S6cakKG4XCoNnEz05NXg6AMu0yxB5AVVxfAhsaAXTGvzGNIZW5QYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UxVbbp7f/tUWSp9TCiJ1ymeNAJ2rQCVxjgtKtkZG6kg=; b=b250R+PRUvnwGMb1+RLj57gXg72xX1KYYbYLTJgRqMrE3gnq0jBbdTQk54HsMYJRO29F2BHBWAuIqapzmSnFJ0O0KyFoYueh8ddpcbj1iDsEf8AE+NAF5UYoQFE1wr3rb8ghY3wg6mDxeiuuWKzC9/H1uS2sX7CjDOr5G+rrArBQ92nmuPVqTIi3iMPb9y3g7MFVTaw7ViSksuc3t1bklpXK19iY2B7JQPv89u/ZVY+8DXtjAaVpDXFeZHEqBQLsTIWRmvWU0SH53X1I19+eh3U/bOaVWmKwbgN0oINIV4g9FsBcuRqBkg8kukF6fiHVyL6Vmn454xap0A43cWfElA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UxVbbp7f/tUWSp9TCiJ1ymeNAJ2rQCVxjgtKtkZG6kg=; b=gYfUqG4vsM832sdHiFIHDj3i9IA/PhHSPkDMRppDPtUff22WyrUbc/dNzCDmMTcshhKoIuISJlAX3c7UZs55SNhzh9RgtqWPiZrwdwIc+S5rpWSiTO5ODcr9SlI+zO5fRlzAq3GyF/u+6lGkl2EzGOZSL4xJzYOOlTPRwKeRd/8= Received: from BYAPR18MB2965.namprd18.prod.outlook.com (2603:10b6:a03:10c::30) by DM5PR18MB2294.namprd18.prod.outlook.com (2603:10b6:4:b3::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5061.28; Tue, 15 Mar 2022 15:47:13 +0000 Received: from BYAPR18MB2965.namprd18.prod.outlook.com ([fe80::ad2b:8bf7:2af1:3ee0]) by BYAPR18MB2965.namprd18.prod.outlook.com ([fe80::ad2b:8bf7:2af1:3ee0%4]) with mapi id 15.20.5081.014; Tue, 15 Mar 2022 15:47:13 +0000 From: Kiran Kumar Kokkilagadda To: Satheesh Paul , Jerin Jacob Kollanukkaran , Nithin Kumar Dabilpuram CC: "stable@dpdk.org" , Satheesh Paul Subject: RE: [PATCH 20.11] net/octeontx2: fix flow MCAM priority management Thread-Topic: [PATCH 20.11] net/octeontx2: fix flow MCAM priority management Thread-Index: AQHYOCN07Ocoyl7CP0Kx7cQ9Ua9hVKzAl4rw Date: Tue, 15 Mar 2022 15:47:13 +0000 Message-ID: References: <20220315041620.2118828-1-psatheesh@marvell.com> In-Reply-To: <20220315041620.2118828-1-psatheesh@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2f9cce5b-e451-4300-93f2-08da069b12d5 x-ms-traffictypediagnostic: DM5PR18MB2294:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zqKpfPQqhpToaxZywXZ4v9F+x10gB+s7R9aQcH5faHN08xEyAw+fgVMmplLBfU1Df2j6SaS37l9XG+2vi9/7DXNVGiy0trXCj/uEjr0V5/2v9Y2TV/tOPsjEwgFlfyhHOy+bCfHYgp6ktkCVM9Obmxh8vK5hMnFn2V+5+pjLcRBEB/5UgjOqjL4bO2HU3y/VpM3krOgN73uDDLfKS36axDkUXMV4a8x5jhrYchzwlWYIUTR/EPpANhinn3a01gs4Ya97QyEsD36QfeCeq42td2+tRRsO30tLL5whfRZTLPTn6tdYiivwIWEcm+4PuCoulVjlUgAST/dfFzX16FjU3oNLOd8BDHVy63BeEhGTxrdmjxr/s9j7G2Kb9GeL6VJXuR4+ATuXm3Nyj6dJLhw9aX7KjYE5WySHmSnbTlacOFbl/BPIzki4KPeN9lX3vUHzjRcm3haX9/k9sNwN4i1wUE8Hkt6TvJtevT9kfjD0f2SiUKBeWAbifhrI160eUqgIqEsenWJ5aE4gjuW8qXYXjuCUcQQHwAi3PI0zkNDmXVlZ4Spvx4/mQOQ1wx8S6upe5/vV0CqkIXbyQvnxKtf7ieZXI2xHzscrVrVecIyp/fGeaxRsUav9IZTRPRzzl7kz5fiGBBDXPkhwSh4cIYq1C9ltPbw/R5MzNYKOalIlAw3yUp02yxT829E9zNiHJPL+0qv5JhcCo7DJEaeFWG90WQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR18MB2965.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(55016003)(9686003)(107886003)(33656002)(30864003)(5660300002)(2906002)(83380400001)(122000001)(66446008)(71200400001)(38070700005)(66556008)(8936002)(66476007)(76116006)(4326008)(64756008)(52536014)(26005)(186003)(8676002)(38100700002)(66946007)(53546011)(6506007)(86362001)(7696005)(110136005)(316002)(54906003)(6636002)(508600001)(559001)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?b/36Cy0JvZCjmtHfxCdqeLKoC6KR7er9JbinxQ3eSOwFWCMm0Vg+yIQ13fG/?= =?us-ascii?Q?9QaYagdyQD9cTwTsymNCM79wM1sAOh9wbN7uycTUoBHdXog662g3fEtMxXIg?= =?us-ascii?Q?F1xKcK70FnD3Un6gYXFh+yOBzJeQkCv8VUdJpKN+F/8j8TeKKrJnWvGa4A09?= =?us-ascii?Q?IHTR7FygE9YD36ACwCJT/Z+5lafOS22Cmb09RkIJwqN36F75BkDY7XJA2VQY?= =?us-ascii?Q?K4x0X9oOFsC9PDAPx6cFy2D0CCJtIoWeGRuNg13uTNJLJrBqFThlvEk6wTCB?= =?us-ascii?Q?XWQDiz/kGxlbkfmdabhnaAGvSE6UZ8ZVd9Rgooxvddg3xAKo2GFykfffS/VV?= =?us-ascii?Q?FGUMLv/dCulHh++xrYiiVIbvk/a5bpy6VOk0veHtFOqXrJDdfJRaLKIVaCwD?= =?us-ascii?Q?VYWTozX91FFRtpTKiD01zN84sNQzFhWQ/4RkY4o3sQRqsR9GHbVnXDBaV42N?= =?us-ascii?Q?4atTy5uId6wSNKGvUEw290nV9Dz0mELBJM/HlDFJV2bUs1GVgCzx/L4jUdNH?= =?us-ascii?Q?Z3AcOCovp9BBFx3civ0ubNGpG2TVuKypLh/wWrQ0mzxBxNrT2nsAXmUATWXb?= =?us-ascii?Q?fxRMrTJj4c3zKQp4rZ4FWtsBPttCJGzj1kIkZ2OBr7tHxarqtBqBR6KwQRhj?= =?us-ascii?Q?qvznu0ASnWmRANHMvjEjOM+bjb6g+gGijJnHFJX6P387sbTawB6uj9G1RQNr?= =?us-ascii?Q?vv+JFrVh5/SOGjFbZtkNtQMbLD6dW0ifVSG+RYOUwE3RSdo4n0Fki+OsK4qK?= =?us-ascii?Q?R89xQRde7HHCvKawWL7vvElsopQ+2M8U8yA7l3pLv5I4TrZLqTmdyuW/rKeV?= =?us-ascii?Q?CMi4DkpSA0ZeTsRZX5heBzPSqO6WI/zd2ZkzIDhoC14zWN0wVw0E9MxyV8aM?= =?us-ascii?Q?AykxFXRxKQl899CkF9gasK8ZLGCK8uloNs7f+cyQtVRs/Nnsa5ugziQIUSoR?= =?us-ascii?Q?O+7zqoj694wJRMK1M9jQtNM5HLdJp/jvcYeykGVpY5Jj6glGM28vZKGxD2as?= =?us-ascii?Q?+2C/EuH0S63hFyRT9TsQZc95MQQlzV/8AaPpbVQetREA9uD+5ZcR/Cc2kYMd?= =?us-ascii?Q?UEHw7MuniUrcusN0qA0WOXXvqnUr4ajyJ34D1Nie3cvVGKxFaPmO0XBPOLqQ?= =?us-ascii?Q?LvQni7EnS2+Ruj8YZKK85icBpRAoEnLuggswalHxGT5q/HkNNznnjKf25pOn?= =?us-ascii?Q?9e+E/s1DtyaSelGnw3zoSJJyei/FUU0xmEnwPAVChnRqoIka7VnGiEb/2oIq?= =?us-ascii?Q?qyhuVdi1/50xmmXpfjlvfzFXzWaYaHGCcoYnqaw9Rn4TLTbFCMSg9CN60+fB?= =?us-ascii?Q?qaNMVvcL3Dx+sCtTDviKMtwnni7VAS5vSjRIFFo7OcywTGIOIaH349eFZvyq?= =?us-ascii?Q?EkK6gOV+X+ypbtOw2FoKcoMtgs37ml6aqYIK7PU84/p9cBusBRH0g1+bfAdy?= =?us-ascii?Q?pNvDiHk00qQR+PcxqP9Tgz527IEc73fX?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR18MB2965.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f9cce5b-e451-4300-93f2-08da069b12d5 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Mar 2022 15:47:13.0937 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UZJ047FKdNVKT7jUMC+ALWsG4fM7HqdKEDGN3H+VQ1oCE3LYLeveIfdKyv8TuiWeVDxZJZgbsDryNJqcXTOMUw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR18MB2294 X-Proofpoint-ORIG-GUID: X9tTe1IhZEH2hioXjjSLhRNMmaS5opiR X-Proofpoint-GUID: X9tTe1IhZEH2hioXjjSLhRNMmaS5opiR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.850,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-03-15_03,2022-03-15_01,2022-02-23_01 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org > -----Original Message----- > From: psatheesh@marvell.com > Sent: Tuesday, March 15, 2022 9:46 AM > To: Jerin Jacob Kollanukkaran ; Nithin Kumar Dabilpur= am > ; Kiran Kumar Kokkilagadda > > Cc: stable@dpdk.org; Satheesh Paul > Subject: [PATCH 20.11] net/octeontx2: fix flow MCAM priority management >=20 > From: Satheesh Paul >=20 > This patch fixes issues in rearranging the MCAM entries when user is crea= ting > flows with priority levels. > The MCAM preallocation scheme and the free entry cache are removed. For > every flow created, an MCAM allocation request is made to the kernel. Eac= h > priority level has a list of MCAM entries. For every flow rule added, the= MCAM > entry obtained from kernel is checked if it is at the correct user specif= ied priority. > If not, the existing rules are moved across MCAM entries so that the user > specified priority is maintained. >=20 > Fixes: 29a2017c70 ("net/octeontx2: add flow mbox utility functions") >=20 > Signed-off-by: Satheesh Paul Acked-by: Kiran Kumar Kokkilagadda > --- > drivers/net/octeontx2/otx2_flow.c | 145 +---- > drivers/net/octeontx2/otx2_flow.h | 33 +- > drivers/net/octeontx2/otx2_flow_utils.c | 717 ++++++++++++------------ > 3 files changed, 405 insertions(+), 490 deletions(-) >=20 > diff --git a/drivers/net/octeontx2/otx2_flow.c > b/drivers/net/octeontx2/otx2_flow.c > index 5ff6a66abb..54dfff8f14 100644 > --- a/drivers/net/octeontx2/otx2_flow.c > +++ b/drivers/net/octeontx2/otx2_flow.c > @@ -13,38 +13,27 @@ otx2_flow_free_all_resources(struct otx2_eth_dev > *hw) { > struct otx2_npc_flow_info *npc =3D &hw->npc_flow; > struct otx2_mbox *mbox =3D hw->mbox; > - struct otx2_mcam_ents_info *info; > - struct rte_bitmap *bmap; > struct rte_flow *flow; > - int entry_count =3D 0; > int rc, idx; >=20 > - for (idx =3D 0; idx < npc->flow_max_priority; idx++) { > - info =3D &npc->flow_entry_info[idx]; > - entry_count +=3D info->live_ent; > - } > - > - if (entry_count =3D=3D 0) > - return 0; > - > /* Free all MCAM entries allocated */ > rc =3D otx2_flow_mcam_free_all_entries(mbox); >=20 > /* Free any MCAM counters and delete flow list */ > for (idx =3D 0; idx < npc->flow_max_priority; idx++) { > while ((flow =3D TAILQ_FIRST(&npc->flow_list[idx])) !=3D NULL) { > - if (flow->ctr_id !=3D NPC_COUNTER_NONE) > + if (flow->ctr_id !=3D NPC_COUNTER_NONE) { > + rc |=3D otx2_flow_mcam_clear_counter(mbox, > + flow->ctr_id); > rc |=3D otx2_flow_mcam_free_counter(mbox, > flow->ctr_id); > + } > + > + otx2_delete_prio_list_entry(npc, flow); >=20 > TAILQ_REMOVE(&npc->flow_list[idx], flow, next); > rte_free(flow); > - bmap =3D npc->live_entries[flow->priority]; > - rte_bitmap_clear(bmap, flow->mcam_id); > } > - info =3D &npc->flow_entry_info[idx]; > - info->free_ent =3D 0; > - info->live_ent =3D 0; > } > return rc; > } > @@ -661,7 +650,6 @@ otx2_flow_destroy(struct rte_eth_dev *dev, > struct otx2_eth_dev *hw =3D dev->data->dev_private; > struct otx2_npc_flow_info *npc =3D &hw->npc_flow; > struct otx2_mbox *mbox =3D hw->mbox; > - struct rte_bitmap *bmap; > uint16_t match_id; > int rc; >=20 > @@ -708,8 +696,7 @@ otx2_flow_destroy(struct rte_eth_dev *dev, >=20 > TAILQ_REMOVE(&npc->flow_list[flow->priority], flow, next); >=20 > - bmap =3D npc->live_entries[flow->priority]; > - rte_bitmap_clear(bmap, flow->mcam_id); > + otx2_delete_prio_list_entry(npc, flow); >=20 > rte_free(flow); > return 0; > @@ -977,9 +964,9 @@ static int otx2_mcam_tot_entries(struct otx2_eth_dev > *dev) int otx2_flow_init(struct otx2_eth_dev *hw) { > - uint8_t *mem =3D NULL, *nix_mem =3D NULL, *npc_mem =3D NULL; > struct otx2_npc_flow_info *npc =3D &hw->npc_flow; > - uint32_t bmap_sz, tot_mcam_entries =3D 0; > + uint32_t bmap_sz, tot_mcam_entries =3D 0, sz =3D 0; > + uint8_t *nix_mem =3D NULL; > int rc =3D 0, idx; >=20 > rc =3D flow_fetch_kex_cfg(hw); > @@ -993,60 +980,6 @@ otx2_flow_init(struct otx2_eth_dev *hw) >=20 > tot_mcam_entries =3D otx2_mcam_tot_entries(hw); > npc->mcam_entries =3D tot_mcam_entries >> npc- > >keyw[NPC_MCAM_RX]; > - /* Free, free_rev, live and live_rev entries */ > - bmap_sz =3D rte_bitmap_get_memory_footprint(npc->mcam_entries); > - mem =3D rte_zmalloc(NULL, 4 * bmap_sz * npc->flow_max_priority, > - RTE_CACHE_LINE_SIZE); > - if (mem =3D=3D NULL) { > - otx2_err("Bmap alloc failed"); > - rc =3D -ENOMEM; > - return rc; > - } > - > - npc->flow_entry_info =3D rte_zmalloc(NULL, npc->flow_max_priority > - * sizeof(struct > otx2_mcam_ents_info), > - 0); > - if (npc->flow_entry_info =3D=3D NULL) { > - otx2_err("flow_entry_info alloc failed"); > - rc =3D -ENOMEM; > - goto err; > - } > - > - npc->free_entries =3D rte_zmalloc(NULL, npc->flow_max_priority > - * sizeof(struct rte_bitmap *), > - 0); > - if (npc->free_entries =3D=3D NULL) { > - otx2_err("free_entries alloc failed"); > - rc =3D -ENOMEM; > - goto err; > - } > - > - npc->free_entries_rev =3D rte_zmalloc(NULL, npc->flow_max_priority > - * sizeof(struct rte_bitmap *), > - 0); > - if (npc->free_entries_rev =3D=3D NULL) { > - otx2_err("free_entries_rev alloc failed"); > - rc =3D -ENOMEM; > - goto err; > - } > - > - npc->live_entries =3D rte_zmalloc(NULL, npc->flow_max_priority > - * sizeof(struct rte_bitmap *), > - 0); > - if (npc->live_entries =3D=3D NULL) { > - otx2_err("live_entries alloc failed"); > - rc =3D -ENOMEM; > - goto err; > - } > - > - npc->live_entries_rev =3D rte_zmalloc(NULL, npc->flow_max_priority > - * sizeof(struct rte_bitmap *), > - 0); > - if (npc->live_entries_rev =3D=3D NULL) { > - otx2_err("live_entries_rev alloc failed"); > - rc =3D -ENOMEM; > - goto err; > - } >=20 > npc->flow_list =3D rte_zmalloc(NULL, npc->flow_max_priority > * sizeof(struct otx2_flow_list), > @@ -1057,30 +990,17 @@ otx2_flow_init(struct otx2_eth_dev *hw) > goto err; > } >=20 > - npc_mem =3D mem; > + sz =3D npc->flow_max_priority * sizeof(struct otx2_prio_flow_list_head)= ; > + npc->prio_flow_list =3D rte_zmalloc(NULL, sz, 0); > + if (npc->prio_flow_list =3D=3D NULL) { > + otx2_err("prio_flow_list alloc failed"); > + rc =3D -ENOMEM; > + goto err; > + } > + > for (idx =3D 0; idx < npc->flow_max_priority; idx++) { > TAILQ_INIT(&npc->flow_list[idx]); > - > - npc->free_entries[idx] =3D > - rte_bitmap_init(npc->mcam_entries, mem, bmap_sz); > - mem +=3D bmap_sz; > - > - npc->free_entries_rev[idx] =3D > - rte_bitmap_init(npc->mcam_entries, mem, bmap_sz); > - mem +=3D bmap_sz; > - > - npc->live_entries[idx] =3D > - rte_bitmap_init(npc->mcam_entries, mem, bmap_sz); > - mem +=3D bmap_sz; > - > - npc->live_entries_rev[idx] =3D > - rte_bitmap_init(npc->mcam_entries, mem, bmap_sz); > - mem +=3D bmap_sz; > - > - npc->flow_entry_info[idx].free_ent =3D 0; > - npc->flow_entry_info[idx].live_ent =3D 0; > - npc->flow_entry_info[idx].max_id =3D 0; > - npc->flow_entry_info[idx].min_id =3D ~(0); > + TAILQ_INIT(&npc->prio_flow_list[idx]); > } >=20 > npc->rss_grps =3D NIX_RSS_GRPS; > @@ -1105,18 +1025,8 @@ otx2_flow_init(struct otx2_eth_dev *hw) > err: > if (npc->flow_list) > rte_free(npc->flow_list); > - if (npc->live_entries_rev) > - rte_free(npc->live_entries_rev); > - if (npc->live_entries) > - rte_free(npc->live_entries); > - if (npc->free_entries_rev) > - rte_free(npc->free_entries_rev); > - if (npc->free_entries) > - rte_free(npc->free_entries); > - if (npc->flow_entry_info) > - rte_free(npc->flow_entry_info); > - if (npc_mem) > - rte_free(npc_mem); > + if (npc->prio_flow_list) > + rte_free(npc->prio_flow_list); > return rc; > } >=20 > @@ -1134,16 +1044,11 @@ otx2_flow_fini(struct otx2_eth_dev *hw) >=20 > if (npc->flow_list) > rte_free(npc->flow_list); > - if (npc->live_entries_rev) > - rte_free(npc->live_entries_rev); > - if (npc->live_entries) > - rte_free(npc->live_entries); > - if (npc->free_entries_rev) > - rte_free(npc->free_entries_rev); > - if (npc->free_entries) > - rte_free(npc->free_entries); > - if (npc->flow_entry_info) > - rte_free(npc->flow_entry_info); > + > + if (npc->prio_flow_list) { > + rte_free(npc->prio_flow_list); > + npc->prio_flow_list =3D NULL; > + } >=20 > return 0; > } > diff --git a/drivers/net/octeontx2/otx2_flow.h > b/drivers/net/octeontx2/otx2_flow.h > index efc9bd1a56..7a62f4469f 100644 > --- a/drivers/net/octeontx2/otx2_flow.h > +++ b/drivers/net/octeontx2/otx2_flow.h > @@ -140,14 +140,6 @@ struct npc_get_datax_cfg { > struct npc_xtract_info flag_xtract[NPC_MAX_LD][NPC_MAX_LT]; > }; >=20 > -struct otx2_mcam_ents_info { > - /* Current max & min values of mcam index */ > - uint32_t max_id; > - uint32_t min_id; > - uint32_t free_ent; > - uint32_t live_ent; > -}; > - > struct rte_flow { > uint8_t nix_intf; > uint32_t mcam_id; > @@ -163,6 +155,13 @@ struct rte_flow { >=20 > TAILQ_HEAD(otx2_flow_list, rte_flow); >=20 > +struct otx2_prio_flow_entry { > + struct rte_flow *flow; > + TAILQ_ENTRY(otx2_prio_flow_entry) next; }; > + > +TAILQ_HEAD(otx2_prio_flow_list_head, otx2_prio_flow_entry); > + > /* Accessed from ethdev private - otx2_eth_dev */ struct otx2_npc_flow_= info { > rte_atomic32_t mark_actions; > @@ -175,22 +174,9 @@ struct otx2_npc_flow_info { > otx2_dxcfg_t prx_dxcfg; /* intf, lid, lt, extract */ > otx2_fxcfg_t prx_fxcfg; /* Flag extract */ > otx2_ld_flags_t prx_lfcfg; /* KEX LD_Flags CFG */ > - /* mcam entry info per priority level: both free & in-use */ > - struct otx2_mcam_ents_info *flow_entry_info; > - /* Bitmap of free preallocated entries in ascending index & > - * descending priority > - */ > - struct rte_bitmap **free_entries; > - /* Bitmap of free preallocated entries in descending index & > - * ascending priority > - */ > - struct rte_bitmap **free_entries_rev; > - /* Bitmap of live entries in ascending index & descending priority */ > - struct rte_bitmap **live_entries; > - /* Bitmap of live entries in descending index & ascending priority */ > - struct rte_bitmap **live_entries_rev; > /* Priority bucket wise tail queue of all rte_flow resources */ > struct otx2_flow_list *flow_list; > + struct otx2_prio_flow_list_head *prio_flow_list; > uint32_t rss_grps; /* rss groups supported */ > struct rte_bitmap *rss_grp_entries; > uint16_t channel; /*rx channel */ > @@ -400,4 +386,7 @@ int otx2_flow_parse_actions(struct rte_eth_dev *dev, > int otx2_flow_free_all_resources(struct otx2_eth_dev *hw); >=20 > int otx2_flow_parse_mpls(struct otx2_parse_state *pst, int lid); > + > +void otx2_delete_prio_list_entry(struct otx2_npc_flow_info *flow_info, > + struct rte_flow *flow); > #endif /* __OTX2_FLOW_H__ */ > diff --git a/drivers/net/octeontx2/otx2_flow_utils.c > b/drivers/net/octeontx2/otx2_flow_utils.c > index 31277adcb4..4f4be7d69f 100644 > --- a/drivers/net/octeontx2/otx2_flow_utils.c > +++ b/drivers/net/octeontx2/otx2_flow_utils.c > @@ -451,435 +451,455 @@ otx2_flow_keyx_compress(uint64_t *data, > uint32_t nibble_mask) } >=20 > static int > -flow_first_set_bit(uint64_t slab) > +otx2_initialise_mcam_entry(struct otx2_mbox *mbox, > + struct otx2_npc_flow_info *flow_info, > + struct rte_flow *flow, int mcam_id) > { > - int num =3D 0; > + struct npc_mcam_write_entry_req *req; > + struct npc_mcam_write_entry_rsq *rsp; > + int rc =3D 0, idx; >=20 > - if ((slab & 0xffffffff) =3D=3D 0) { > - num +=3D 32; > - slab >>=3D 32; > - } > - if ((slab & 0xffff) =3D=3D 0) { > - num +=3D 16; > - slab >>=3D 16; > - } > - if ((slab & 0xff) =3D=3D 0) { > - num +=3D 8; > - slab >>=3D 8; > - } > - if ((slab & 0xf) =3D=3D 0) { > - num +=3D 4; > - slab >>=3D 4; > + req =3D otx2_mbox_alloc_msg_npc_mcam_write_entry(mbox); > + if (req =3D=3D NULL) > + return -ENOSPC; > + req->set_cntr =3D 0; > + req->cntr =3D 0; > + req->entry =3D mcam_id; > + > + req->intf =3D (flow->nix_intf =3D=3D NIX_INTF_RX) ? NPC_MCAM_RX : > NPC_MCAM_TX; > + req->enable_entry =3D 1; > + req->entry_data.action =3D flow->npc_action; > + req->entry_data.vtag_action =3D flow->vtag_action; > + > + for (idx =3D 0; idx < OTX2_MAX_MCAM_WIDTH_DWORDS; idx++) { > + req->entry_data.kw[idx] =3D 0x0; > + req->entry_data.kw_mask[idx] =3D 0x0; > } > - if ((slab & 0x3) =3D=3D 0) { > - num +=3D 2; > - slab >>=3D 2; > + > + if (flow->nix_intf =3D=3D NIX_INTF_RX) { > + req->entry_data.kw[0] |=3D (uint64_t)flow_info->channel; > + req->entry_data.kw_mask[0] |=3D (BIT_ULL(12) - 1); > + } else { > + uint16_t pf_func =3D (flow->npc_action >> 4) & 0xffff; > + > + pf_func =3D rte_cpu_to_be_16(pf_func); > + req->entry_data.kw[0] |=3D ((uint64_t)pf_func << 32); > + req->entry_data.kw_mask[0] |=3D ((uint64_t)0xffff << 32); > } > - if ((slab & 0x1) =3D=3D 0) > - num +=3D 1; >=20 > - return num; > + rc =3D otx2_mbox_process_msg(mbox, (void *)&rsp); > + if (rc !=3D 0) { > + otx2_err("npc: mcam initialisation write failed"); > + return rc; > + } > + return 0; > } >=20 > static int > -flow_shift_lv_ent(struct otx2_mbox *mbox, struct rte_flow *flow, > - struct otx2_npc_flow_info *flow_info, > - uint32_t old_ent, uint32_t new_ent) > +otx2_shift_mcam_entry(struct otx2_mbox *mbox, uint16_t old_ent, > + uint16_t new_ent) > { > struct npc_mcam_shift_entry_req *req; > struct npc_mcam_shift_entry_rsp *rsp; > - struct otx2_flow_list *list; > - struct rte_flow *flow_iter; > - int rc =3D 0; > - > - otx2_npc_dbg("Old ent:%u new ent:%u priority:%u", old_ent, new_ent, > - flow->priority); > - > - list =3D &flow_info->flow_list[flow->priority]; > + int rc =3D -ENOSPC; >=20 > /* Old entry is disabled & it's contents are moved to new_entry, > * new entry is enabled finally. > */ > req =3D otx2_mbox_alloc_msg_npc_mcam_shift_entry(mbox); > + if (req =3D=3D NULL) > + return rc; > req->curr_entry[0] =3D old_ent; > req->new_entry[0] =3D new_ent; > req->shift_count =3D 1; >=20 > - otx2_mbox_msg_send(mbox, 0); > - rc =3D otx2_mbox_get_rsp(mbox, 0, (void *)&rsp); > + rc =3D otx2_mbox_process_msg(mbox, (void *)&rsp); > if (rc) > return rc; >=20 > - /* Remove old node from list */ > - TAILQ_FOREACH(flow_iter, list, next) { > - if (flow_iter->mcam_id =3D=3D old_ent) > - TAILQ_REMOVE(list, flow_iter, next); > - } > - > - /* Insert node with new mcam id at right place */ > - TAILQ_FOREACH(flow_iter, list, next) { > - if (flow_iter->mcam_id > new_ent) > - TAILQ_INSERT_BEFORE(flow_iter, flow, next); > - } > - return rc; > + return 0; > } >=20 > -/* Exchange all required entries with a given priority level */ > +enum SHIFT_DIR { > + SLIDE_ENTRIES_TO_LOWER_INDEX, > + SLIDE_ENTRIES_TO_HIGHER_INDEX, > +}; > + > static int > -flow_shift_ent(struct otx2_mbox *mbox, struct rte_flow *flow, > - struct otx2_npc_flow_info *flow_info, > - struct npc_mcam_alloc_entry_rsp *rsp, int dir, int prio_lvl) > +otx2_slide_mcam_entries(struct otx2_mbox *mbox, > + struct otx2_npc_flow_info *flow_info, int prio, > + uint16_t *free_mcam_id, int dir) > { > - struct rte_bitmap *fr_bmp, *fr_bmp_rev, *lv_bmp, *lv_bmp_rev, *bmp; > - uint32_t e_fr =3D 0, e_lv =3D 0, e, e_id =3D 0, mcam_entries; > - uint64_t fr_bit_pos =3D 0, lv_bit_pos =3D 0, bit_pos =3D 0; > - /* Bit position within the slab */ > - uint32_t sl_fr_bit_off =3D 0, sl_lv_bit_off =3D 0; > - /* Overall bit position of the start of slab */ > - /* free & live entry index */ > - int rc_fr =3D 0, rc_lv =3D 0, rc =3D 0, idx =3D 0; > - struct otx2_mcam_ents_info *ent_info; > - /* free & live bitmap slab */ > - uint64_t sl_fr =3D 0, sl_lv =3D 0, *sl; > - > - fr_bmp =3D flow_info->free_entries[prio_lvl]; > - fr_bmp_rev =3D flow_info->free_entries_rev[prio_lvl]; > - lv_bmp =3D flow_info->live_entries[prio_lvl]; > - lv_bmp_rev =3D flow_info->live_entries_rev[prio_lvl]; > - ent_info =3D &flow_info->flow_entry_info[prio_lvl]; > - mcam_entries =3D flow_info->mcam_entries; > - > - > - /* New entries allocated are always contiguous, but older entries > - * already in free/live bitmap can be non-contiguous: so return > - * shifted entries should be in non-contiguous format. > - */ > - while (idx <=3D rsp->count) { > - if (!sl_fr && !sl_lv) { > - /* Lower index elements to be exchanged */ > - if (dir < 0) { > - rc_fr =3D rte_bitmap_scan(fr_bmp, &e_fr, &sl_fr); > - rc_lv =3D rte_bitmap_scan(lv_bmp, &e_lv, &sl_lv); > - otx2_npc_dbg("Fwd slab rc fr %u rc lv %u " > - "e_fr %u e_lv %u", rc_fr, rc_lv, > - e_fr, e_lv); > - } else { > - rc_fr =3D rte_bitmap_scan(fr_bmp_rev, > - &sl_fr_bit_off, > - &sl_fr); > - rc_lv =3D rte_bitmap_scan(lv_bmp_rev, > - &sl_lv_bit_off, > - &sl_lv); > - > - otx2_npc_dbg("Rev slab rc fr %u rc lv %u " > - "e_fr %u e_lv %u", rc_fr, rc_lv, > - e_fr, e_lv); > - } > - } > - > - if (rc_fr) { > - fr_bit_pos =3D flow_first_set_bit(sl_fr); > - e_fr =3D sl_fr_bit_off + fr_bit_pos; > - otx2_npc_dbg("Fr_bit_pos 0x%" PRIx64, fr_bit_pos); > - } else { > - e_fr =3D ~(0); > - } > - > - if (rc_lv) { > - lv_bit_pos =3D flow_first_set_bit(sl_lv); > - e_lv =3D sl_lv_bit_off + lv_bit_pos; > - otx2_npc_dbg("Lv_bit_pos 0x%" PRIx64, lv_bit_pos); > - } else { > - e_lv =3D ~(0); > - } > + uint16_t to_mcam_id =3D 0, from_mcam_id =3D 0; > + struct otx2_prio_flow_list_head *list; > + struct otx2_prio_flow_entry *curr =3D 0; > + int rc =3D 0; >=20 > - /* First entry is from free_bmap */ > - if (e_fr < e_lv) { > - bmp =3D fr_bmp; > - e =3D e_fr; > - sl =3D &sl_fr; > - bit_pos =3D fr_bit_pos; > - if (dir > 0) > - e_id =3D mcam_entries - e - 1; > - else > - e_id =3D e; > - otx2_npc_dbg("Fr e %u e_id %u", e, e_id); > - } else { > - bmp =3D lv_bmp; > - e =3D e_lv; > - sl =3D &sl_lv; > - bit_pos =3D lv_bit_pos; > - if (dir > 0) > - e_id =3D mcam_entries - e - 1; > - else > - e_id =3D e; > - > - otx2_npc_dbg("Lv e %u e_id %u", e, e_id); > - if (idx < rsp->count) > - rc =3D > - flow_shift_lv_ent(mbox, flow, > - flow_info, e_id, > - rsp->entry + idx); > + list =3D &flow_info->prio_flow_list[prio]; > + > + to_mcam_id =3D *free_mcam_id; > + if (dir =3D=3D SLIDE_ENTRIES_TO_HIGHER_INDEX) > + curr =3D TAILQ_LAST(list, otx2_prio_flow_list_head); > + else if (dir =3D=3D SLIDE_ENTRIES_TO_LOWER_INDEX) > + curr =3D TAILQ_FIRST(list); > + > + while (curr) { > + from_mcam_id =3D curr->flow->mcam_id; > + if ((dir =3D=3D SLIDE_ENTRIES_TO_HIGHER_INDEX && > + from_mcam_id < to_mcam_id) || > + (dir =3D=3D SLIDE_ENTRIES_TO_LOWER_INDEX && > + from_mcam_id > to_mcam_id)) { > + /* Newly allocated entry and the source entry given to > + * npc_mcam_shift_entry_req will be in disabled state. > + * Initialise and enable before moving an entry into > + * this mcam. > + */ > + rc =3D otx2_initialise_mcam_entry(mbox, flow_info, > + curr->flow, > to_mcam_id); > + if (rc) > + return rc; > + rc =3D otx2_shift_mcam_entry(mbox, from_mcam_id, > + to_mcam_id); > + if (rc) > + return rc; > + > + curr->flow->mcam_id =3D to_mcam_id; > + to_mcam_id =3D from_mcam_id; > } >=20 > - rte_bitmap_clear(bmp, e); > - rte_bitmap_set(bmp, rsp->entry + idx); > - /* Update entry list, use non-contiguous > - * list now. > - */ > - rsp->entry_list[idx] =3D e_id; > - *sl &=3D ~(1 << bit_pos); > + if (dir =3D=3D SLIDE_ENTRIES_TO_HIGHER_INDEX) > + curr =3D TAILQ_PREV(curr, otx2_prio_flow_list_head, > next); > + else if (dir =3D=3D SLIDE_ENTRIES_TO_LOWER_INDEX) > + curr =3D TAILQ_NEXT(curr, next); > + } >=20 > - /* Update min & max entry identifiers in current > - * priority level. > - */ > - if (dir < 0) { > - ent_info->max_id =3D rsp->entry + idx; > - ent_info->min_id =3D e_id; > - } else { > - ent_info->max_id =3D e_id; > - ent_info->min_id =3D rsp->entry; > - } > + *free_mcam_id =3D from_mcam_id; >=20 > - idx++; > - } > - return rc; > + return 0; > } >=20 > -/* Validate if newly allocated entries lie in the correct priority zone > - * since NPC_MCAM_LOWER_PRIO & NPC_MCAM_HIGHER_PRIO don't ensure > zone accuracy. > - * If not properly aligned, shift entries to do so > +/* > + * The mcam_alloc request is first made with NPC_MCAM_LOWER_PRIO with > +the last > + * entry in the requested priority level as the reference entry. If it > +fails, > + * the alloc request is retried with NPC_MCAM_HIGHER_PRIO with the > +first entry > + * in the next lower priority level as the reference entry. After > +obtaining > + * the free MCAM from kernel, we check if it is at the right user > +requested > + * priority level. If not, the flow rules are moved across MCAM entries > +till > + * the user requested priority levels are met. > + * The MCAM sorting algorithm works as below. > + * For any given free MCAM obtained from the kernel, there are 3 possibi= lities. > + * Case 1: > + * There are entries belonging to higher user priority > +level(numerically > + * lesser) in higher mcam indices. In this case, the entries with > +higher user > + * priority are slided towards lower indices and a free entry is > +created in the > + * higher indices. > + * Example: > + * Assume free entry =3D 1610, user requested priority =3D 2 and > + * max user priority levels =3D 5 with below entries in respective > +priority > + * levels. > + * 0: 1630, 1635, 1641 > + * 1: 1646, 1650, 1651 > + * 2: 1652, 1655, 1660 > + * 3: 1661, 1662, 1663, 1664 > + * 4: 1665, 1667, 1670 > + * > + * Entries (1630, 1635, 1641, 1646, 1650, 1651) have to be slided down > +towards > + * lower indices. > + * Shifting sequence will be as below: > + * 1610 <- 1630 <- 1635 <- 1641 <- 1646 <- 1650 <- 1651 > + * Entry 1651 will be free-ed for writing the new flow. This entry will > +now > + * become the head of priority level 2. > + * > + * Case 2: > + * There are entries belonging to lower user priority level > +(numerically > + * bigger) in lower mcam indices. In this case, the entries with lower > +user > + * priority are slided towards higher indices and a free entry is > +created in the > + * lower indices. > + * > + * Example: > + * free entry =3D 1653, user requested priority =3D 0 > + * 0: 1630, 1635, 1641 > + * 1: 1646, 1650, 1651 > + * 2: 1652, 1655, 1660 > + * 3: 1661, 1662, 1663, 1664 > + * 4: 1665, 1667, 1670 > + * > + * Entries (1646, 1650, 1651, 1652) have to be slided up towards higher > + * indices. > + * Shifting sequence will be as below: > + * 1646 -> 1650 -> 1651 -> 1652 -> 1653 > + * Entry 1646 will be free-ed for writing the new flow. This entry will > +now > + * become the last element in priority level 0. > + * > + * Case 3: > + * Free mcam is at the right place, ie, all higher user priority level > + * mcams lie in lower indices and all lower user priority level mcams > +lie in > + * higher mcam indices. > + * > + * The priority level lists are scanned first for case (1) and if the > + * condition is found true, case(2) is skipped because they are > +mutually > + * exclusive. For example, consider below state. > + * 0: 1630, 1635, 1641 > + * 1: 1646, 1650, 1651 > + * 2: 1652, 1655, 1660 > + * 3: 1661, 1662, 1663, 1664 > + * 4: 1665, 1667, 1670 > + * free entry =3D 1610, user requested priority =3D 2 > + * > + * Case 1: Here the condition is; > + * "if (requested_prio > prio_idx && free_mcam < tail->flow->mcam_id ){}= " > + * If this condition is true, it means at some higher priority level > +than > + * requested priority level, there are entries at lower indices than > +the given > + * free mcam. That is, we have found in levels 0,1 there is an mcam X > +which is > + * greater than 1610. > + * If, for any free entry and user req prio, the above condition is > +true, then > + * the below case(2) condition will always be false since the lists are > +kept > + * sorted. The case(2) condition is; > + * "if (requested_prio < prio_idx && free_mcam > head->flow->mcam_id){}= " > + * There can't be entries at lower indices at priority level higher > + * than the requested priority level. That is, here, at levels 3 & 4 > +there > + * cannot be any entry greater than 1610. Because all entries in 3 & 4 > +must be > + * greater than X which was found to be greater than 1610 earlier. > */ > + > static int > -flow_validate_and_shift_prio_ent(struct otx2_mbox *mbox, struct rte_flow > *flow, > - struct otx2_npc_flow_info *flow_info, > - struct npc_mcam_alloc_entry_rsp *rsp, > - int req_prio) > +otx2_sort_mcams_by_user_prio_level(struct otx2_mbox *mbox, > + struct otx2_prio_flow_entry *flow_list_entry, > + struct otx2_npc_flow_info *flow_info, > + struct npc_mcam_alloc_entry_rsp *rsp) > { > - int prio_idx =3D 0, rc =3D 0, needs_shift =3D 0, idx, prio =3D flow->pr= iority; > - struct otx2_mcam_ents_info *info =3D flow_info->flow_entry_info; > - int dir =3D (req_prio =3D=3D NPC_MCAM_HIGHER_PRIO) ? 1 : -1; > - uint32_t tot_ent =3D 0; > - > - otx2_npc_dbg("Dir %d, priority =3D %d", dir, prio); > - > - if (dir < 0) > - prio_idx =3D flow_info->flow_max_priority - 1; > - > - /* Only live entries needs to be shifted, free entries can just be > - * moved by bits manipulation. > - */ > - > - /* For dir =3D -1(NPC_MCAM_LOWER_PRIO), when shifting, > - * NPC_MAX_PREALLOC_ENT are exchanged with adjoining higher > priority > - * level entries(lower indexes). > - * > - * For dir =3D +1(NPC_MCAM_HIGHER_PRIO), during shift, > - * NPC_MAX_PREALLOC_ENT are exchanged with adjoining lower > priority > - * level entries(higher indexes) with highest indexes. > - */ > - do { > - tot_ent =3D info[prio_idx].free_ent + info[prio_idx].live_ent; > - > - if (dir < 0 && prio_idx !=3D prio && > - rsp->entry > info[prio_idx].max_id && tot_ent) { > - otx2_npc_dbg("Rsp entry %u prio idx %u " > - "max id %u", rsp->entry, prio_idx, > - info[prio_idx].max_id); > - > - needs_shift =3D 1; > - } else if ((dir > 0) && (prio_idx !=3D prio) && > - (rsp->entry < info[prio_idx].min_id) && tot_ent) { > - otx2_npc_dbg("Rsp entry %u prio idx %u " > - "min id %u", rsp->entry, prio_idx, > - info[prio_idx].min_id); > - needs_shift =3D 1; > + int requested_prio =3D flow_list_entry->flow->priority; > + struct otx2_prio_flow_entry *head, *tail; > + struct otx2_prio_flow_list_head *list; > + uint16_t free_mcam =3D rsp->entry; > + bool do_reverse_scan =3D true; > + int prio_idx =3D 0, rc =3D 0; > + > + while (prio_idx <=3D flow_info->flow_max_priority - 1) { > + list =3D &flow_info->prio_flow_list[prio_idx]; > + tail =3D TAILQ_LAST(list, otx2_prio_flow_list_head); > + > + /* requested priority is lower than current level > + * ie, numerically req prio is higher > + */ > + if (requested_prio > prio_idx && tail) { > + /* but there are some mcams in current level > + * at higher indices, ie, at priority lower > + * than free_mcam. > + */ > + if (free_mcam < tail->flow->mcam_id) { > + rc =3D otx2_slide_mcam_entries(mbox, > flow_info, > + prio_idx, &free_mcam, > + > SLIDE_ENTRIES_TO_LOWER_INDEX); > + if (rc) > + return rc; > + do_reverse_scan =3D false; > + } > } > + prio_idx++; > + } >=20 > - otx2_npc_dbg("Needs_shift =3D %d", needs_shift); > - if (needs_shift) { > - needs_shift =3D 0; > - rc =3D flow_shift_ent(mbox, flow, flow_info, rsp, dir, > - prio_idx); > - } else { > - for (idx =3D 0; idx < rsp->count; idx++) > - rsp->entry_list[idx] =3D rsp->entry + idx; > - } > - } while ((prio_idx !=3D prio) && (prio_idx +=3D dir)); > + prio_idx =3D flow_info->flow_max_priority - 1; > + while (prio_idx && do_reverse_scan) { > + list =3D &flow_info->prio_flow_list[prio_idx]; > + head =3D TAILQ_FIRST(list); >=20 > + /* requested priority is higher than current level > + * ie, numerically req prio is lower > + */ > + if (requested_prio < prio_idx && head) { > + /* but free mcam is higher than lowest priority > + * mcam in current level > + */ > + if (free_mcam > head->flow->mcam_id) { > + rc =3D otx2_slide_mcam_entries(mbox, > flow_info, > + prio_idx, &free_mcam, > + > SLIDE_ENTRIES_TO_HIGHER_INDEX); > + if (rc) > + return rc; > + } > + } > + prio_idx--; > + } > + rsp->entry =3D free_mcam; > return rc; > } >=20 > -static int > -flow_find_ref_entry(struct otx2_npc_flow_info *flow_info, int *prio, > - int prio_lvl) > +static void > +otx2_insert_into_flow_list(struct otx2_npc_flow_info *flow_info, > + struct otx2_prio_flow_entry *entry) > { > - struct otx2_mcam_ents_info *info =3D flow_info->flow_entry_info; > - int step =3D 1; > - > - while (step < flow_info->flow_max_priority) { > - if (((prio_lvl + step) < flow_info->flow_max_priority) && > - info[prio_lvl + step].live_ent) { > - *prio =3D NPC_MCAM_HIGHER_PRIO; > - return info[prio_lvl + step].min_id; > - } > + struct otx2_prio_flow_list_head *list; > + struct otx2_prio_flow_entry *curr; > + > + list =3D &flow_info->prio_flow_list[entry->flow->priority]; > + curr =3D TAILQ_FIRST(list); >=20 > - if (((prio_lvl - step) >=3D 0) && > - info[prio_lvl - step].live_ent) { > - otx2_npc_dbg("Prio_lvl %u live %u", prio_lvl - step, > - info[prio_lvl - step].live_ent); > - *prio =3D NPC_MCAM_LOWER_PRIO; > - return info[prio_lvl - step].max_id; > + if (curr) { > + while (curr) { > + if (entry->flow->mcam_id > curr->flow->mcam_id) > + curr =3D TAILQ_NEXT(curr, next); > + else > + break; > } > - step++; > + if (curr) > + TAILQ_INSERT_BEFORE(curr, entry, next); > + else > + TAILQ_INSERT_TAIL(list, entry, next); > + } else { > + TAILQ_INSERT_HEAD(list, entry, next); > } > - *prio =3D NPC_MCAM_ANY_PRIO; > - return 0; > } >=20 > static int > -flow_fill_entry_cache(struct otx2_mbox *mbox, struct rte_flow *flow, > - struct otx2_npc_flow_info *flow_info, uint32_t *free_ent) > +otx2_allocate_mcam_entry(struct otx2_mbox *mbox, int prio, > + struct npc_mcam_alloc_entry_rsp *rsp_local, > + int ref_entry) > { > - struct rte_bitmap *free_bmp, *free_bmp_rev, *live_bmp, > *live_bmp_rev; > - struct npc_mcam_alloc_entry_rsp rsp_local; > struct npc_mcam_alloc_entry_rsp *rsp_cmd; > struct npc_mcam_alloc_entry_req *req; > struct npc_mcam_alloc_entry_rsp *rsp; > - struct otx2_mcam_ents_info *info; > - uint16_t ref_ent, idx; > - int rc, prio; > - > - info =3D &flow_info->flow_entry_info[flow->priority]; > - free_bmp =3D flow_info->free_entries[flow->priority]; > - free_bmp_rev =3D flow_info->free_entries_rev[flow->priority]; > - live_bmp =3D flow_info->live_entries[flow->priority]; > - live_bmp_rev =3D flow_info->live_entries_rev[flow->priority]; > - > - ref_ent =3D flow_find_ref_entry(flow_info, &prio, flow->priority); > + int rc =3D -ENOSPC; >=20 > req =3D otx2_mbox_alloc_msg_npc_mcam_alloc_entry(mbox); > + if (req =3D=3D NULL) > + return rc; > req->contig =3D 1; > - req->count =3D flow_info->flow_prealloc_size; > + req->count =3D 1; > req->priority =3D prio; > - req->ref_entry =3D ref_ent; > + req->ref_entry =3D ref_entry; >=20 > - otx2_npc_dbg("Fill cache ref entry %u prio %u", ref_ent, prio); > - > - otx2_mbox_msg_send(mbox, 0); > - rc =3D otx2_mbox_get_rsp(mbox, 0, (void *)&rsp_cmd); > + rc =3D otx2_mbox_process_msg(mbox, (void *)&rsp_cmd); > if (rc) > return rc; >=20 > - rsp =3D &rsp_local; > - memcpy(rsp, rsp_cmd, sizeof(*rsp)); > + if (!rsp_cmd->count) > + return -ENOSPC; >=20 > - otx2_npc_dbg("Alloc entry %u count %u , prio =3D %d", rsp->entry, > - rsp->count, prio); > + memcpy(rsp_local, rsp_cmd, sizeof(*rsp)); >=20 > - /* Non-first ent cache fill */ > - if (prio !=3D NPC_MCAM_ANY_PRIO) { > - flow_validate_and_shift_prio_ent(mbox, flow, flow_info, rsp, > - prio); > - } else { > - /* Copy into response entry list */ > - for (idx =3D 0; idx < rsp->count; idx++) > - rsp->entry_list[idx] =3D rsp->entry + idx; > - } > - > - otx2_npc_dbg("Fill entry cache rsp count %u", rsp->count); > - /* Update free entries, reverse free entries list, > - * min & max entry ids. > - */ > - for (idx =3D 0; idx < rsp->count; idx++) { > - if (unlikely(rsp->entry_list[idx] < info->min_id)) > - info->min_id =3D rsp->entry_list[idx]; > - > - if (unlikely(rsp->entry_list[idx] > info->max_id)) > - info->max_id =3D rsp->entry_list[idx]; > + return 0; > +} >=20 > - /* Skip entry to be returned, not to be part of free > - * list. > - */ > - if (prio =3D=3D NPC_MCAM_HIGHER_PRIO) { > - if (unlikely(idx =3D=3D (rsp->count - 1))) { > - *free_ent =3D rsp->entry_list[idx]; > - continue; > +static void > +otx2_find_mcam_ref_entry(struct rte_flow *flow, > + struct otx2_npc_flow_info *flow_info, int *prio, > + int *ref_entry, int dir) > +{ > + struct otx2_prio_flow_entry *head, *tail; > + struct otx2_prio_flow_list_head *list; > + int prio_idx =3D flow->priority; > + > + if (dir =3D=3D NPC_MCAM_LOWER_PRIO) { > + while (prio_idx >=3D 0) { > + list =3D &flow_info->prio_flow_list[prio_idx]; > + head =3D TAILQ_FIRST(list); > + if (head) { > + *prio =3D NPC_MCAM_LOWER_PRIO; > + *ref_entry =3D head->flow->mcam_id; > + return; > } > - } else { > - if (unlikely(!idx)) { > - *free_ent =3D rsp->entry_list[idx]; > - continue; > + prio_idx--; > + } > + } else if (dir =3D=3D NPC_MCAM_HIGHER_PRIO) { > + prio_idx =3D flow->priority; > + while (prio_idx <=3D flow_info->flow_max_priority - 1) { > + list =3D &flow_info->prio_flow_list[prio_idx]; > + tail =3D TAILQ_LAST(list, otx2_prio_flow_list_head); > + if (tail) { > + *prio =3D NPC_MCAM_HIGHER_PRIO; > + *ref_entry =3D tail->flow->mcam_id; > + return; > } > + prio_idx++; > } > - info->free_ent++; > - rte_bitmap_set(free_bmp, rsp->entry_list[idx]); > - rte_bitmap_set(free_bmp_rev, flow_info->mcam_entries - > - rsp->entry_list[idx] - 1); > - > - otx2_npc_dbg("Final rsp entry %u rsp entry rev %u", > - rsp->entry_list[idx], > - flow_info->mcam_entries - rsp->entry_list[idx] - 1); > } > + *prio =3D NPC_MCAM_ANY_PRIO; > + *ref_entry =3D 0; > +} >=20 > - otx2_npc_dbg("Cache free entry %u, rev =3D %u", *free_ent, > - flow_info->mcam_entries - *free_ent - 1); > - info->live_ent++; > - rte_bitmap_set(live_bmp, *free_ent); > - rte_bitmap_set(live_bmp_rev, flow_info->mcam_entries - *free_ent - > 1); > +static int > +otx2_alloc_mcam_by_ref_entry(struct otx2_mbox *mbox, struct rte_flow > *flow, > + struct otx2_npc_flow_info *flow_info, > + struct npc_mcam_alloc_entry_rsp *rsp_local) { > + int prio, ref_entry =3D 0, rc =3D 0, dir =3D NPC_MCAM_LOWER_PRIO; > + bool retry_done =3D false; > + > +retry: > + otx2_find_mcam_ref_entry(flow, flow_info, &prio, &ref_entry, dir); > + rc =3D otx2_allocate_mcam_entry(mbox, prio, rsp_local, ref_entry); > + if (rc && !retry_done) { > + otx2_info("npc: Lower priority entry not available. " > + "Retrying for higher priority"); > + > + dir =3D NPC_MCAM_HIGHER_PRIO; > + retry_done =3D true; > + goto retry; > + } else if (rc && retry_done) { > + return rc; > + } >=20 > return 0; > } >=20 > static int > -flow_check_preallocated_entry_cache(struct otx2_mbox *mbox, > - struct rte_flow *flow, > - struct otx2_npc_flow_info *flow_info) > +otx2_get_free_mcam_entry(struct otx2_mbox *mbox, struct rte_flow *flow, > + struct otx2_npc_flow_info *flow_info) > { > - struct rte_bitmap *free, *free_rev, *live, *live_rev; > - uint32_t pos =3D 0, free_ent =3D 0, mcam_entries; > - struct otx2_mcam_ents_info *info; > - uint64_t slab =3D 0; > - int rc; > + struct npc_mcam_alloc_entry_rsp rsp_local; > + struct otx2_prio_flow_entry *new_entry; > + int rc =3D 0; >=20 > - otx2_npc_dbg("Flow priority %u", flow->priority); > + rc =3D otx2_alloc_mcam_by_ref_entry(mbox, flow, flow_info, > &rsp_local); >=20 > - info =3D &flow_info->flow_entry_info[flow->priority]; > + if (rc) > + return rc; >=20 > - free_rev =3D flow_info->free_entries_rev[flow->priority]; > - free =3D flow_info->free_entries[flow->priority]; > - live_rev =3D flow_info->live_entries_rev[flow->priority]; > - live =3D flow_info->live_entries[flow->priority]; > - mcam_entries =3D flow_info->mcam_entries; > + new_entry =3D rte_zmalloc("otx2_rte_flow", sizeof(*new_entry), 0); > + if (!new_entry) > + return -ENOSPC; >=20 > - if (info->free_ent) { > - rc =3D rte_bitmap_scan(free, &pos, &slab); > - if (rc) { > - /* Get free_ent from free entry bitmap */ > - free_ent =3D pos + __builtin_ctzll(slab); > - otx2_npc_dbg("Allocated from cache entry %u", > free_ent); > - /* Remove from free bitmaps and add to live ones */ > - rte_bitmap_clear(free, free_ent); > - rte_bitmap_set(live, free_ent); > - rte_bitmap_clear(free_rev, > - mcam_entries - free_ent - 1); > - rte_bitmap_set(live_rev, > - mcam_entries - free_ent - 1); > - > - info->free_ent--; > - info->live_ent++; > - return free_ent; > - } > + new_entry->flow =3D flow; >=20 > - otx2_npc_dbg("No free entry:its a mess"); > - return -1; > - } > + otx2_npc_dbg("kernel allocated MCAM entry %d", rsp_local.entry); >=20 > - rc =3D flow_fill_entry_cache(mbox, flow, flow_info, &free_ent); > + rc =3D otx2_sort_mcams_by_user_prio_level(mbox, new_entry, flow_info, > + &rsp_local); > if (rc) > - return rc; > + goto err; > + > + otx2_npc_dbg("allocated MCAM entry after sorting %d", > rsp_local.entry); > + flow->mcam_id =3D rsp_local.entry; > + otx2_insert_into_flow_list(flow_info, new_entry); > + > + return rsp_local.entry; > +err: > + rte_free(new_entry); > + return rc; > +} > + > +void > +otx2_delete_prio_list_entry(struct otx2_npc_flow_info *flow_info, > + struct rte_flow *flow) > +{ > + struct otx2_prio_flow_list_head *list; > + struct otx2_prio_flow_entry *curr; >=20 > - return free_ent; > + list =3D &flow_info->prio_flow_list[flow->priority]; > + curr =3D TAILQ_FIRST(list); > + > + if (!curr) > + return; > + > + while (curr) { > + if (flow->mcam_id =3D=3D curr->flow->mcam_id) { > + TAILQ_REMOVE(list, curr, next); > + rte_free(curr); > + break; > + } > + curr =3D TAILQ_NEXT(curr, next); > + } > } >=20 > int > @@ -902,10 +922,11 @@ otx2_flow_mcam_alloc_and_write(struct rte_flow > *flow, struct otx2_mbox *mbox, > return rc; > } >=20 > - entry =3D flow_check_preallocated_entry_cache(mbox, flow, flow_info); > + entry =3D otx2_get_free_mcam_entry(mbox, flow, flow_info); > if (entry < 0) { > - otx2_err("Prealloc failed"); > - otx2_flow_mcam_free_counter(mbox, ctr); > + otx2_err("MCAM allocation failed"); > + if (use_ctr) > + otx2_flow_mcam_free_counter(mbox, ctr); > return NPC_MCAM_ALLOC_FAILED; > } >=20 > -- > 2.25.4