From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D30F2A0560; Tue, 18 Oct 2022 13:27:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C697415D7; Tue, 18 Oct 2022 13:27:39 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2100.outbound.protection.outlook.com [40.107.236.100]) by mails.dpdk.org (Postfix) with ESMTP id D876A41144 for ; Tue, 18 Oct 2022 13:27:37 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VuFjboOfx2SPbh9AX5sPgPgtai9a6dJ1FYHAR/NmEAB8D0QX2YaU6qCvOol1ZIrqzDqUneBGQl1Yw8qRrRgq6uwknHI1tle17eiJh3ox6uZi6FFF0mSK5guYZCSecYP7lY4qDTVGW9p33mMC0KwSsdqPx1SI8cvud592fbE8i8atOr6cD1mRqQA58RDR/KgMQ0usm3JFJc3bVLfqkMcvpWBPt7h/9Izkwg2WXRbN1SD+0x0ipNS8ctweQt1mS0Nxd2XrYpe3KsCgHZ5Uxg80oi+YR+aSi6poKJa8yJAoBRRc081BsyIcEFf04rdP2s86JbyTvoJQL5XimqVgUdRYzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mA2BXif1Y0aOEAlbc2Gk2cxtaK7+XNywgDDknlU4K48=; b=gePp5rHUjc+E/37OZquobVM1qdfCdxtWDt+TIRsdFr5n5zeGI7UadOc6wHTKQsilgBr1TEZH4HAkUbMuMU7U34fVkvq0HwpMQsgh3EECooHuqt94b2+qfe9LsJS+jw+3dLjT2UJc3A40IYbEcKy2ppYeGebEvMJ5HvqhwFzljPUptdeBLYCYr9cRRP2wdCCRi4h68KgKkypquemnCduG5dhUDIo7hHB7sLkRkSQk7Ue+YGj0l9klzDllCsFDEIQFkYdPasR7V12MADrk210ocDthECe7W8+lzTtvyVlfknpw4TTq53NNrATcrvjC3+KxhVdlu6/iu52BzKXumKIvWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mA2BXif1Y0aOEAlbc2Gk2cxtaK7+XNywgDDknlU4K48=; b=Ebm20quxE4M3LZ8V0QUjqDx3zh2qAQX0hI6hz/tOXuXgG5zpKQopvTvhKWq0H/Kt1c2khEMqjK40G6xIbfdBO1knkui8aaO4+GNnvyw7TpYizyIKkQhHRqkoC5Z044czCBM4Fn91A3TXzBETl9q8LbFGvj0WyYcvKvhkUARVi9Q= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by CO3PR13MB5686.namprd13.prod.outlook.com (2603:10b6:303:179::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5746.16; Tue, 18 Oct 2022 11:27:36 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::7c26:1a0b:2825:6f4b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::7c26:1a0b:2825:6f4b%4]) with mapi id 15.20.5723.014; Tue, 18 Oct 2022 11:27:36 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v4 02/25] net/nfp: add the structures and functions for flow offload Date: Tue, 18 Oct 2022 19:26:51 +0800 Message-Id: <1666092434-10357-3-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1666092434-10357-1-git-send-email-chaoyong.he@corigine.com> References: <1666092434-10357-1-git-send-email-chaoyong.he@corigine.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SG2PR02CA0052.apcprd02.prod.outlook.com (2603:1096:4:54::16) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|CO3PR13MB5686:EE_ X-MS-Office365-Filtering-Correlation-Id: 9b2fa656-6cb5-468f-ba15-08dab0fbc1d3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hL02HPtGrEoL6ilsl0g4J/MghPRzmWJyJaYJdsTF4M9q/GMlPLPrRNrggRxrrK8JXoKX8Q7yh6e68ucPgH4ylFjo3S1y4ImRSG0lyRRUCt8lKy2p5n+YcxiaCd49wGZ0DwNKCuXbgzj89xUoTZ/cUKXaPVPbq1vQdGLqtKSVLeHIH3dXlG7d96EQxFC0JPPWV60fWjFGW8xYpPHw0N0AE72JYFR1GUrt/CnCatnbsW76X/Nuz5Jbtmhjov6n4ycQWExo2ljI0zX44sMjVYDlQ0nzR28Pb5VnGefiFnlAgAW8oVSarcUqnvR5G/oGqT76m+1MOmkXj7TpG7R0Wx3EqwR5RbBEHcxcrONg8iQuNwih1+wsXkVnDF6Vt7QYYtchB46cuTmRDq5n8iuVEwPtzWlyoua4I8MjMggM+Il8KJWOT4EJaPKR94ygXsVs4AC08RwJ4WcqNMFZBvUMAg5tTnaUuASRIA2ZI/E1Mbu6sRERMF6knzPKBPsF+cZEWhw8GYwtpjVa0xOi+fHZCkqs1htgARRb/KulsGsYvPSHDCPaAsR6Dy4YQAXvwWKIkUlgJpAGlaabmtNWVN3KP3qOU0M09DRKD8ZSdllWw9t2GhN//plJwM5lUwdzIzvABd6IBmVgmo2aXedBSzW48r64CiSHRh3ZVbQFKRZsyDTpoVGe3hwQ2efpbWRm/vltBzc5E0BEbJ3f9s6XdGo+IbiHx14QXqAb6F9w/8UIP6+u4V2ZTX3MhueOcv2sxH97VQQjtl1hTZC0VBTNRGtTk586SNQNkKAUIVovv62ncNQMBaV3JlAPXGCnnRt4XF8U5JxB X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(346002)(366004)(39840400004)(396003)(136003)(376002)(451199015)(38350700002)(38100700002)(2906002)(66574015)(41300700001)(316002)(66946007)(30864003)(66556008)(66476007)(83380400001)(4326008)(8676002)(6916009)(86362001)(44832011)(2616005)(8936002)(186003)(36756003)(5660300002)(6506007)(107886003)(6666004)(6486002)(478600001)(26005)(6512007)(52116002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S2k3NHh6ZlBaYTNZUkUraDgzZzI2Z2dTZXRSTERnekoxM25IY2hyQngwdmo2?= =?utf-8?B?dzV5eTIwVXM3TlkxeXRXdUpqaUFOOUhjWHdLZzVYNE1LWndwd3F6L2x2N3Y2?= =?utf-8?B?R2hla0xacEdQZ3FmcU1OMUR2cEFyMnpHV0lXM1AvVktqZTZkVHhrd1BhTC9u?= =?utf-8?B?QW1wMWVzeTRrLzRjT1dQdzhSVFM4TnVTbGVNVFZzN2t6ekIveHNsRTI5WGh0?= =?utf-8?B?cWNMYndKSnc5L0h2TFl2SzZNQ0d0SGdRQ0tPWmZsNnE0enVudTFZekZ2TkZs?= =?utf-8?B?STFrbldJWVhkSm52MnZqZ3ZQNnZOVnB2dzFDM3p2Mk9qNVVqMndTSVdLVTQ0?= =?utf-8?B?ZUtKZVFtV2FidUpiLzh0cXJxM25yZDZGOFViMEh5M0N5ZSsyK1lBOGNzSWs0?= =?utf-8?B?bzhtSXdKZDZQU1ZpRXdVYTZqa2Fic0h1N3ROanRZcmFPWGVoNklGUGlhdWJT?= =?utf-8?B?M3hXeCtRN2FQQzFWakhjTktnVENHU28wcVFHS1pWbmJqYWZaelpnbXFuRHpP?= =?utf-8?B?VkJFM3FkQVM4ZzNWbThmWkxsR3R6MjJuMTlNR3NtbG9WeElzSWtvZkpFdlcy?= =?utf-8?B?VnRITk9DdThSOFd5RTJPSHllcWV1VWRMMCtOTEw2aFM5eVNGcFBLKzY1U2JU?= =?utf-8?B?TTdTalJzWk5ITkYzeGo1L3V4VHpSdm1TT0FkZnAwRC8zUy9uWWJ4YUVvVzZ5?= =?utf-8?B?eit4YUJ4ZjlQYUZ2Y1UzUGQvbXVYcEZaUFFWcC9MOXlVK2hIK3IxVUdwNGpm?= =?utf-8?B?Y2c1dEFyZG1NcHJuTzZQeFNoZGZjbGowYXJaQ0gwTnlQYjQzU25SOGJ6T29F?= =?utf-8?B?V3A1aXVxMmI1N2s0NU5JdEVDUWpraThCbHdyclZVUDVIcnFNZjZXZllGeUk2?= =?utf-8?B?eDE4T1RpZ1pOWWZyczJCRmtxZXBhWWEzbFQ4T0RacXpNMjZ1OUFBcjhPdmhh?= =?utf-8?B?TEh0dXNQSjRkaWtQMmpsYlF6V0p2RUk3aEluWVFtT0tNQmZid25GcFNoL1Bo?= =?utf-8?B?Ti9WV3VWY21jNEJvdGE5SU11ajlka1JXaERmUTFGNWRXdzY3MVJaZFBSU0dW?= =?utf-8?B?Mk5STDRJazlIWXNjalJjaFl5cWVobE9jVjd3VXVXcTBzTU1iWjEyNDFEa09J?= =?utf-8?B?VnFTOUZHOFpDNldaN1hONVdvaXlPazh1cU15VDIycC8zUHpjNFN0bHpseFY3?= =?utf-8?B?S0VmNUlXVDh0TjBORmFUWElzdmtIY1Z4NWVQa0FxWmQyK3FPcWwwNkdEeVFm?= =?utf-8?B?N1hOMDNTNDVJb2hKeVYvcXg5NVI3Q25ndWVxS0RoMnYzU2t1UlFCUzdkR2JR?= =?utf-8?B?R0JlZ00vazdxOGtvaVlDeWNOL1ZPaDJ0NFBMeFVrdG5uVmY1cjEya2orWlVZ?= =?utf-8?B?b3JSbTRJcGhrZmpiSWN2bk5FenppZjNTOG0zZnYwYVl1UnUzRGtLd3pNYlhK?= =?utf-8?B?YWtUSytyVm1qOFcwODRyUlVZeGtBOWs5QzZnZmp3eFhnSS9JL09ZR3R1OFpC?= =?utf-8?B?RkVKWEwrWVhRNUpNWEJEWktzMkJJYTk4UnVHMzNRUjVTTi9LM3dqVFc1SFhN?= =?utf-8?B?OU9JNnVzMDZlRlQzZkhZc0Q2em1LcXY5MllXZzluWmtoZDdkQjRFOWM3TFhB?= =?utf-8?B?cjFNMTl3UVlCOFVEalJFY213K3Jlam1kTjlHeTBWU29iRmdHd1pJOFd0eVMy?= =?utf-8?B?bEhRa0xvRzVWbFIxWHVBVlpuQ2Vkd1Y0bWxjVTREejRaM2g3S1NIYTZCQzZs?= =?utf-8?B?ODdXKzZtZzhXZUFNZFdtNk1FMU41eS9RNE51eEpNTFZoZi9OaUtZSUdEYzlv?= =?utf-8?B?THdJdnhyVDJBVWFMdXJFSVZVT01XdmtjUzVpTDRkZWxDY0ZZM3NYWHhhby8x?= =?utf-8?B?dUx0MXpidmNwVjFVZmFoMU9qU0pPK2tBZzQ3dVFCbHJMQmZ5aENPQkhvRUFl?= =?utf-8?B?aGpEYk5ObG1idGx6aWtNQkNBT2pxcFJuMXM0dWVQUnVBWFFteWNkYkJVUTcy?= =?utf-8?B?dzdYUGtEcEI2b0JzMGxsMXdJSStWU2Q4MkpBT29WNm8vN0xDUUpjK011VGdB?= =?utf-8?B?cEFEU2ZCamFNcklFSHlGM0o5NkdyT2l0V0NzcHBQU0xUSVczeHJIWGVxbzJr?= =?utf-8?B?ZmpQT1NTQzdKQnRhaGFnVDRxd0tRcGZsV0xCVTZOWWxkMzhSNXR0bWhPdWNp?= =?utf-8?B?K0E9PQ==?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b2fa656-6cb5-468f-ba15-08dab0fbc1d3 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2022 11:27:36.2588 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0MM1INWYyop1gfPoMEFusZpdC2eVBSrJgP1zyxj2wSMY1HaqylVQZloKcJ1lqW2x/Nfaa5VahDHOwU4OPPw1JVIP64QLzTIY+BkPC2HFIbA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO3PR13MB5686 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the structures and functions to process mask table, flow table, and flow stats id, which are used in the rte_flow offload logics. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund --- drivers/net/nfp/flower/nfp_flower.c | 11 +- drivers/net/nfp/flower/nfp_flower.h | 2 + drivers/net/nfp/meson.build | 3 + drivers/net/nfp/nfp_flow.c | 507 ++++++++++++++++++++++++++++++++++++ drivers/net/nfp/nfp_flow.h | 101 +++++++ 5 files changed, 623 insertions(+), 1 deletion(-) create mode 100644 drivers/net/nfp/nfp_flow.c create mode 100644 drivers/net/nfp/nfp_flow.h diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c index 3e97f5c..168bf0c 100644 --- a/drivers/net/nfp/flower/nfp_flower.c +++ b/drivers/net/nfp/flower/nfp_flower.c @@ -15,6 +15,7 @@ #include "../nfp_ctrl.h" #include "../nfp_cpp_bridge.h" #include "../nfp_rxtx.h" +#include "../nfp_flow.h" #include "../nfpcore/nfp_mip.h" #include "../nfpcore/nfp_rtsym.h" #include "../nfpcore/nfp_nsp.h" @@ -1089,13 +1090,19 @@ pf_dev->app_fw_priv = app_fw_flower; + ret = nfp_flow_priv_init(pf_dev); + if (ret != 0) { + PMD_INIT_LOG(ERR, "init flow priv failed"); + goto app_cleanup; + } + /* Allocate memory for the PF AND ctrl vNIC here (hence the * 2) */ pf_hw = rte_zmalloc_socket("nfp_pf_vnic", 2 * sizeof(struct nfp_net_adapter), RTE_CACHE_LINE_SIZE, numa_node); if (pf_hw == NULL) { PMD_INIT_LOG(ERR, "Could not malloc nfp pf vnic"); ret = -ENOMEM; - goto app_cleanup; + goto flow_priv_cleanup; } /* Map the PF ctrl bar */ @@ -1173,6 +1180,8 @@ nfp_cpp_area_free(pf_dev->ctrl_area); vnic_cleanup: rte_free(pf_hw); +flow_priv_cleanup: + nfp_flow_priv_uninit(pf_dev); app_cleanup: rte_free(app_fw_flower); diff --git a/drivers/net/nfp/flower/nfp_flower.h b/drivers/net/nfp/flower/nfp_flower.h index 48f597a..b90391c 100644 --- a/drivers/net/nfp/flower/nfp_flower.h +++ b/drivers/net/nfp/flower/nfp_flower.h @@ -51,6 +51,8 @@ struct nfp_app_fw_flower { /* PF representor */ struct nfp_flower_representor *pf_repr; + + struct nfp_flow_priv *flow_priv; }; int nfp_init_app_fw_flower(struct nfp_pf_dev *pf_dev); diff --git a/drivers/net/nfp/meson.build b/drivers/net/nfp/meson.build index 8a63979..7416fd3 100644 --- a/drivers/net/nfp/meson.build +++ b/drivers/net/nfp/meson.build @@ -27,4 +27,7 @@ sources = files( 'nfp_cpp_bridge.c', 'nfp_ethdev_vf.c', 'nfp_ethdev.c', + 'nfp_flow.c', ) + +deps += ['hash'] diff --git a/drivers/net/nfp/nfp_flow.c b/drivers/net/nfp/nfp_flow.c new file mode 100644 index 0000000..97f5d1b --- /dev/null +++ b/drivers/net/nfp/nfp_flow.c @@ -0,0 +1,507 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#include +#include +#include +#include + +#include "nfp_common.h" +#include "nfp_flow.h" +#include "nfp_logs.h" +#include "flower/nfp_flower.h" +#include "nfpcore/nfp_mip.h" +#include "nfpcore/nfp_rtsym.h" + +struct nfp_mask_id_entry { + uint32_t hash_key; + uint32_t ref_cnt; + uint8_t mask_id; +}; + +static int +nfp_mask_id_alloc(struct nfp_flow_priv *priv, uint8_t *mask_id) +{ + uint8_t temp_id; + uint8_t freed_id; + struct circ_buf *ring; + + /* Checking for unallocated entries first. */ + if (priv->mask_ids.init_unallocated > 0) { + *mask_id = priv->mask_ids.init_unallocated; + priv->mask_ids.init_unallocated--; + return 0; + } + + /* Checking if buffer is empty. */ + freed_id = NFP_FLOWER_MASK_ENTRY_RS - 1; + ring = &priv->mask_ids.free_list; + if (ring->head == ring->tail) { + *mask_id = freed_id; + return -ENOENT; + } + + rte_memcpy(&temp_id, &ring->buf[ring->tail], NFP_FLOWER_MASK_ELEMENT_RS); + *mask_id = temp_id; + + rte_memcpy(&ring->buf[ring->tail], &freed_id, NFP_FLOWER_MASK_ELEMENT_RS); + ring->tail = (ring->tail + NFP_FLOWER_MASK_ELEMENT_RS) % + (NFP_FLOWER_MASK_ENTRY_RS * NFP_FLOWER_MASK_ELEMENT_RS); + + return 0; +} + +static int +nfp_mask_id_free(struct nfp_flow_priv *priv, uint8_t mask_id) +{ + struct circ_buf *ring; + + ring = &priv->mask_ids.free_list; + + /* Checking if buffer is full. */ + if (CIRC_SPACE(ring->head, ring->tail, NFP_FLOWER_MASK_ENTRY_RS) == 0) + return -ENOBUFS; + + rte_memcpy(&ring->buf[ring->head], &mask_id, NFP_FLOWER_MASK_ELEMENT_RS); + ring->head = (ring->head + NFP_FLOWER_MASK_ELEMENT_RS) % + (NFP_FLOWER_MASK_ENTRY_RS * NFP_FLOWER_MASK_ELEMENT_RS); + + return 0; +} + +static int +nfp_mask_table_add(struct nfp_flow_priv *priv, + char *mask_data, + uint32_t mask_len, + uint8_t *id) +{ + int ret; + uint8_t mask_id; + uint32_t hash_key; + struct nfp_mask_id_entry *mask_entry; + + mask_entry = rte_zmalloc("mask_entry", sizeof(struct nfp_mask_id_entry), 0); + if (mask_entry == NULL) { + ret = -ENOMEM; + goto exit; + } + + ret = nfp_mask_id_alloc(priv, &mask_id); + if (ret != 0) + goto mask_entry_free; + + hash_key = rte_jhash(mask_data, mask_len, priv->hash_seed); + mask_entry->mask_id = mask_id; + mask_entry->hash_key = hash_key; + mask_entry->ref_cnt = 1; + PMD_DRV_LOG(DEBUG, "hash_key=%#x id=%u ref=%u", hash_key, + mask_id, mask_entry->ref_cnt); + + ret = rte_hash_add_key_data(priv->mask_table, &hash_key, mask_entry); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Add to mask table failed."); + goto mask_id_free; + } + + *id = mask_id; + return 0; + +mask_id_free: + nfp_mask_id_free(priv, mask_id); +mask_entry_free: + rte_free(mask_entry); +exit: + return ret; +} + +static int +nfp_mask_table_del(struct nfp_flow_priv *priv, + char *mask_data, + uint32_t mask_len, + uint8_t id) +{ + int ret; + uint32_t hash_key; + + hash_key = rte_jhash(mask_data, mask_len, priv->hash_seed); + ret = rte_hash_del_key(priv->mask_table, &hash_key); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Delete from mask table failed."); + return ret; + } + + ret = nfp_mask_id_free(priv, id); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Free mask id failed."); + return ret; + } + + return 0; +} + +static struct nfp_mask_id_entry * +nfp_mask_table_search(struct nfp_flow_priv *priv, + char *mask_data, + uint32_t mask_len) +{ + int index; + uint32_t hash_key; + struct nfp_mask_id_entry *entry; + + hash_key = rte_jhash(mask_data, mask_len, priv->hash_seed); + index = rte_hash_lookup_data(priv->mask_table, &hash_key, (void **)&entry); + if (index < 0) { + PMD_DRV_LOG(DEBUG, "Data NOT found in the mask table."); + return NULL; + } + + return entry; +} + +__rte_unused static bool +nfp_check_mask_add(struct nfp_flow_priv *priv, + char *mask_data, + uint32_t mask_len, + uint8_t *meta_flags, + uint8_t *mask_id) +{ + int ret; + struct nfp_mask_id_entry *mask_entry; + + mask_entry = nfp_mask_table_search(priv, mask_data, mask_len); + if (mask_entry == NULL) { + /* mask entry does not exist, let's create one */ + ret = nfp_mask_table_add(priv, mask_data, mask_len, mask_id); + if (ret != 0) + return false; + + *meta_flags |= NFP_FL_META_FLAG_MANAGE_MASK; + } else { + /* mask entry already exist */ + mask_entry->ref_cnt++; + *mask_id = mask_entry->mask_id; + } + + return true; +} + +__rte_unused static bool +nfp_check_mask_remove(struct nfp_flow_priv *priv, + char *mask_data, + uint32_t mask_len, + uint8_t *meta_flags) +{ + int ret; + struct nfp_mask_id_entry *mask_entry; + + mask_entry = nfp_mask_table_search(priv, mask_data, mask_len); + if (mask_entry == NULL) + return false; + + mask_entry->ref_cnt--; + if (mask_entry->ref_cnt == 0) { + ret = nfp_mask_table_del(priv, mask_data, mask_len, + mask_entry->mask_id); + if (ret != 0) + return false; + + rte_free(mask_entry); + if (meta_flags) + *meta_flags &= ~NFP_FL_META_FLAG_MANAGE_MASK; + } + + return true; +} + +__rte_unused static int +nfp_flow_table_add(struct nfp_flow_priv *priv, + struct rte_flow *nfp_flow) +{ + int ret; + char *hash_data; + uint32_t hash_key; + + hash_data = (char *)(nfp_flow->payload.unmasked_data); + hash_key = rte_jhash(hash_data, nfp_flow->length, priv->hash_seed); + ret = rte_hash_add_key_data(priv->flow_table, &hash_key, nfp_flow); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Add to flow table failed."); + return ret; + } + + return 0; +} + +__rte_unused static int +nfp_flow_table_delete(struct nfp_flow_priv *priv, + struct rte_flow *nfp_flow) +{ + int ret; + char *hash_data; + uint32_t hash_key; + + hash_data = (char *)(nfp_flow->payload.unmasked_data); + hash_key = rte_jhash(hash_data, nfp_flow->length, priv->hash_seed); + ret = rte_hash_del_key(priv->flow_table, &hash_key); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Delete from flow table failed."); + return ret; + } + + return 0; +} + +__rte_unused static struct rte_flow * +nfp_flow_table_search(struct nfp_flow_priv *priv, + struct rte_flow *nfp_flow) +{ + int index; + char *hash_data; + uint32_t hash_key; + struct rte_flow *flow_find; + + hash_data = (char *)(nfp_flow->payload.unmasked_data); + hash_key = rte_jhash(hash_data, nfp_flow->length, priv->hash_seed); + index = rte_hash_lookup_data(priv->flow_table, &hash_key, (void **)&flow_find); + if (index < 0) { + PMD_DRV_LOG(DEBUG, "Data NOT found in the flow table."); + return NULL; + } + + return flow_find; +} + +__rte_unused static struct rte_flow * +nfp_flow_alloc(struct nfp_fl_key_ls *key_layer) +{ + char *tmp; + size_t len; + struct rte_flow *nfp_flow; + struct nfp_fl_payload *payload; + + nfp_flow = rte_zmalloc("nfp_flow", sizeof(struct rte_flow), 0); + if (nfp_flow == NULL) + goto exit; + + len = key_layer->key_size + key_layer->key_size + key_layer->act_size; + tmp = rte_zmalloc("nfp_flow_payload", len + sizeof(struct nfp_fl_rule_metadata), 0); + if (tmp == NULL) + goto free_flow; + + nfp_flow->length = len; + + payload = &nfp_flow->payload; + payload->meta = (struct nfp_fl_rule_metadata *)tmp; + payload->unmasked_data = tmp + sizeof(struct nfp_fl_rule_metadata); + payload->mask_data = payload->unmasked_data + key_layer->key_size; + payload->action_data = payload->mask_data + key_layer->key_size; + + return nfp_flow; + +free_flow: + rte_free(nfp_flow); +exit: + return NULL; +} + +__rte_unused static void +nfp_flow_free(struct rte_flow *nfp_flow) +{ + rte_free(nfp_flow->payload.meta); + rte_free(nfp_flow); +} + +__rte_unused static int +nfp_stats_id_alloc(struct nfp_flow_priv *priv, uint32_t *ctx) +{ + struct circ_buf *ring; + uint32_t freed_stats_id; + uint32_t temp_stats_id; + + /* Check for unallocated entries first. */ + if (priv->stats_ids.init_unallocated > 0) { + *ctx = ((priv->stats_ids.init_unallocated - 1) & NFP_FL_STAT_ID_STAT) | + (priv->active_mem_unit & NFP_FL_STAT_ID_MU_NUM); + if (++priv->active_mem_unit == priv->total_mem_units) { + priv->stats_ids.init_unallocated--; + priv->active_mem_unit = 0; + } + return 0; + } + + /* Check if buffer is empty */ + ring = &priv->stats_ids.free_list; + freed_stats_id = priv->stats_ring_size; + if (ring->head == ring->tail) { + *ctx = freed_stats_id; + return -ENOENT; + } + + memcpy(&temp_stats_id, &ring->buf[ring->tail], NFP_FL_STATS_ELEM_RS); + *ctx = temp_stats_id; + memcpy(&ring->buf[ring->tail], &freed_stats_id, NFP_FL_STATS_ELEM_RS); + ring->tail = (ring->tail + NFP_FL_STATS_ELEM_RS) % + (priv->stats_ring_size * NFP_FL_STATS_ELEM_RS); + + return 0; +} + +__rte_unused static int +nfp_stats_id_free(struct nfp_flow_priv *priv, uint32_t ctx) +{ + struct circ_buf *ring; + + /* Check if buffer is full */ + ring = &priv->stats_ids.free_list; + if (!CIRC_SPACE(ring->head, ring->tail, priv->stats_ring_size * + NFP_FL_STATS_ELEM_RS - NFP_FL_STATS_ELEM_RS + 1)) + return -ENOBUFS; + + memcpy(&ring->buf[ring->head], &ctx, NFP_FL_STATS_ELEM_RS); + ring->head = (ring->head + NFP_FL_STATS_ELEM_RS) % + (priv->stats_ring_size * NFP_FL_STATS_ELEM_RS); + + return 0; +} + +int +nfp_flow_priv_init(struct nfp_pf_dev *pf_dev) +{ + int ret = 0; + uint64_t ctx_count; + uint64_t ctx_split; + size_t stats_size; + struct nfp_flow_priv *priv; + struct nfp_app_fw_flower *app_fw_flower; + + struct rte_hash_parameters mask_hash_params = { + .name = "mask_hash_table", + .entries = NFP_MASK_TABLE_ENTRIES, + .hash_func = rte_jhash, + .socket_id = rte_socket_id(), + .key_len = sizeof(uint32_t), + .extra_flag = RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY, + }; + + struct rte_hash_parameters flow_hash_params = { + .name = "flow_hash_table", + .hash_func = rte_jhash, + .socket_id = rte_socket_id(), + .key_len = sizeof(uint32_t), + .extra_flag = RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY, + }; + + ctx_count = nfp_rtsym_read_le(pf_dev->sym_tbl, + "CONFIG_FC_HOST_CTX_COUNT", &ret); + if (ret < 0) { + PMD_INIT_LOG(ERR, "Read CTX_COUNT from symbol table failed"); + goto exit; + } + + ctx_split = nfp_rtsym_read_le(pf_dev->sym_tbl, + "CONFIG_FC_HOST_CTX_SPLIT", &ret); + if (ret < 0) { + PMD_INIT_LOG(ERR, "Read CTX_SPLIT from symbol table failed"); + goto exit; + } + + priv = rte_zmalloc("nfp_app_flow_priv", sizeof(struct nfp_flow_priv), 0); + if (priv == NULL) { + PMD_INIT_LOG(ERR, "nfp app flow priv creation failed"); + ret = -ENOMEM; + goto exit; + } + + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + app_fw_flower->flow_priv = priv; + priv->hash_seed = (uint32_t)rte_rand(); + priv->stats_ring_size = ctx_count; + priv->total_mem_units = ctx_split; + + /* Init ring buffer and unallocated mask_ids. */ + priv->mask_ids.init_unallocated = NFP_FLOWER_MASK_ENTRY_RS - 1; + priv->mask_ids.free_list.buf = rte_zmalloc("nfp_app_mask_ids", + NFP_FLOWER_MASK_ENTRY_RS * NFP_FLOWER_MASK_ELEMENT_RS, 0); + if (priv->mask_ids.free_list.buf == NULL) { + PMD_INIT_LOG(ERR, "mask id free list creation failed"); + ret = -ENOMEM; + goto free_priv; + } + + /* Init ring buffer and unallocated stats_ids. */ + priv->stats_ids.init_unallocated = ctx_count / ctx_split; + priv->stats_ids.free_list.buf = rte_zmalloc("nfp_app_stats_ids", + priv->stats_ring_size * NFP_FL_STATS_ELEM_RS, 0); + if (priv->stats_ids.free_list.buf == NULL) { + PMD_INIT_LOG(ERR, "stats id free list creation failed"); + ret = -ENOMEM; + goto free_mask_id; + } + + /* flow stats */ + rte_spinlock_init(&priv->stats_lock); + stats_size = (ctx_count & NFP_FL_STAT_ID_STAT) | + ((ctx_split - 1) & NFP_FL_STAT_ID_MU_NUM); + PMD_INIT_LOG(INFO, "ctx_count:%0lx, ctx_split:%0lx, stats_size:%0lx ", + ctx_count, ctx_split, stats_size); + priv->stats = rte_zmalloc("nfp_flow_stats", + stats_size * sizeof(struct nfp_fl_stats), 0); + if (priv->stats == NULL) { + PMD_INIT_LOG(ERR, "flow stats creation failed"); + ret = -ENOMEM; + goto free_stats_id; + } + + /* mask table */ + mask_hash_params.hash_func_init_val = priv->hash_seed; + priv->mask_table = rte_hash_create(&mask_hash_params); + if (priv->mask_table == NULL) { + PMD_INIT_LOG(ERR, "mask hash table creation failed"); + ret = -ENOMEM; + goto free_stats; + } + + /* flow table */ + flow_hash_params.hash_func_init_val = priv->hash_seed; + flow_hash_params.entries = ctx_count; + priv->flow_table = rte_hash_create(&flow_hash_params); + if (priv->flow_table == NULL) { + PMD_INIT_LOG(ERR, "flow hash table creation failed"); + ret = -ENOMEM; + goto free_mask_table; + } + + return 0; + +free_mask_table: + rte_free(priv->mask_table); +free_stats: + rte_free(priv->stats); +free_stats_id: + rte_free(priv->stats_ids.free_list.buf); +free_mask_id: + rte_free(priv->mask_ids.free_list.buf); +free_priv: + rte_free(priv); +exit: + return ret; +} + +void +nfp_flow_priv_uninit(struct nfp_pf_dev *pf_dev) +{ + struct nfp_app_fw_flower *app_fw_flower; + struct nfp_flow_priv *priv; + + app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev->app_fw_priv); + priv = app_fw_flower->flow_priv; + + rte_hash_free(priv->flow_table); + rte_hash_free(priv->mask_table); + rte_free(priv->stats); + rte_free(priv->stats_ids.free_list.buf); + rte_free(priv->mask_ids.free_list.buf); + rte_free(priv); +} diff --git a/drivers/net/nfp/nfp_flow.h b/drivers/net/nfp/nfp_flow.h new file mode 100644 index 0000000..142d7d5 --- /dev/null +++ b/drivers/net/nfp/nfp_flow.h @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 Corigine, Inc. + * All rights reserved. + */ + +#ifndef _NFP_FLOW_H_ +#define _NFP_FLOW_H_ + +#define NFP_FL_META_FLAG_MANAGE_MASK (1 << 7) + +#define NFP_MASK_TABLE_ENTRIES 1024 + +enum nfp_flower_tun_type { + NFP_FL_TUN_NONE = 0, + NFP_FL_TUN_GRE = 1, + NFP_FL_TUN_VXLAN = 2, + NFP_FL_TUN_GENEVE = 4, +}; + +struct nfp_fl_key_ls { + uint32_t key_layer_two; + uint8_t key_layer; + int key_size; + int act_size; + uint32_t port; + uint16_t vlan; + enum nfp_flower_tun_type tun_type; +}; + +struct nfp_fl_rule_metadata { + uint8_t key_len; + uint8_t mask_len; + uint8_t act_len; + uint8_t flags; + rte_be32_t host_ctx_id; + rte_be64_t host_cookie __rte_packed; + rte_be64_t flow_version __rte_packed; + rte_be32_t shortcut; +}; + +struct nfp_fl_payload { + struct nfp_fl_rule_metadata *meta; + char *unmasked_data; + char *mask_data; + char *action_data; +}; + +#define CIRC_CNT(head, tail, size) (((head) - (tail)) & ((size) - 1)) +#define CIRC_SPACE(head, tail, size) CIRC_CNT((tail), ((head) + 1), (size)) +struct circ_buf { + uint32_t head; + uint32_t tail; + char *buf; +}; + +#define NFP_FLOWER_MASK_ENTRY_RS 256 +#define NFP_FLOWER_MASK_ELEMENT_RS sizeof(uint8_t) +struct nfp_fl_mask_id { + struct circ_buf free_list; + uint8_t init_unallocated; +}; + +#define NFP_FL_STATS_ELEM_RS sizeof(uint32_t) +struct nfp_fl_stats_id { + struct circ_buf free_list; + uint32_t init_unallocated; +}; + +#define NFP_FL_STAT_ID_MU_NUM 0xffc00000 +#define NFP_FL_STAT_ID_STAT 0x003fffff +struct nfp_fl_stats { + uint64_t pkts; + uint64_t bytes; +}; + +struct nfp_flow_priv { + uint32_t hash_seed; /**< Hash seed for hash tables in this structure. */ + /* mask hash table */ + struct nfp_fl_mask_id mask_ids; /**< Entry for mask hash table */ + struct rte_hash *mask_table; /**< Hash table to store mask ids. */ + /* flow hash table */ + struct rte_hash *flow_table; /**< Hash table to store flow rules. */ + /* flow stats */ + uint32_t active_mem_unit; /**< The size of active mem units. */ + uint32_t total_mem_units; /**< The size of total mem units. */ + uint32_t stats_ring_size; /**< The size of stats id ring. */ + struct nfp_fl_stats_id stats_ids; /**< The stats id ring. */ + struct nfp_fl_stats *stats; /**< Store stats of flow. */ + rte_spinlock_t stats_lock; /** < Lock the update of 'stats' field. */ +}; + +struct rte_flow { + struct nfp_fl_payload payload; + size_t length; + bool install_flag; +}; + +int nfp_flow_priv_init(struct nfp_pf_dev *pf_dev); +void nfp_flow_priv_uninit(struct nfp_pf_dev *pf_dev); + +#endif /* _NFP_FLOW_H_ */ -- 1.8.3.1