From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0D5A0A0032; Sat, 15 Oct 2022 09:23:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A37DC40E25; Sat, 15 Oct 2022 09:23:18 +0200 (CEST) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2130.outbound.protection.outlook.com [40.107.100.130]) by mails.dpdk.org (Postfix) with ESMTP id 68B0F40042 for ; Sat, 15 Oct 2022 09:23:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eHKgKbqqsDkKxqLTPYdNoYgVofOk1QMxUkEey+FH6KLvD2g/BNEhjmjQ1fbSW/T3S2Yoyk4AaFdMFCpp8Cm/ITqAMuWcid0zzveLra9jVgNzF3WOebJkidnA+1x8mvTcqIy+fifoBD/eNA+DQWqHRfvpwMn/hNcgsslkZ1tN6Q1ZbVi/hcQpNzTl1+j5f8Y9pvfJo3EK6ZZQBNs70yr/1WNSG1po1Tuljn+bikHhNFdaVaahoyYYH8CR1/oGQm4ufBiplzXwn2EQP3HX234MYT8lYNMvreskehgGZPFQEvW9x0ygDFC8PEqZq3BZgmyjWKnXpZ0oasBGyHGIZjB5Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EB8nOL+tAVRaczU+sHfKrAd74rf3QfAz9O62g9hVGoM=; b=PxZuRCTtRnx7uKC+YGL3hVzmHiuqbyyVl7Kg8dSPMxk9gzLTaGVWicTyZFkRU17KNF2XELEYF0kY/UMOh2mAY5cGHKjW3NlDO5F0m6bxQd89BXRI6okZcKY02MFfoToA0iTEUzHNHxY4TUqXuuYHKQw3v0uwHdAKEGhc4S825kFkmM245AkKWvgjNulxkAx63nGJMnMcvTRAaBdjUgJN31L0wkFzU3kQV57w2y/2DHhJ9KIKizgVFouMeUMHfRNYhyJTh4IUbJNu9h8KK2yxe15OuIy9oYZx2s5AaaMDlDL/mw5wM547b2L6MfbpHXvsbe7AKOFQzcnRE26lhISt3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EB8nOL+tAVRaczU+sHfKrAd74rf3QfAz9O62g9hVGoM=; b=bDmO7p4FzBdPiOb4roFNlmG3Ml9TxTDYj/4HDYAjD52m7ldRpoRMjjHIDLZvm56uNhmR/YjDqU4clu11ubzp+HKu8W2O/4KuQ/Y7RmuvxHtbW1eF7EOueXeOG2YKJ0Ct6BEH0yHOkdqJy6FbY95vQ2nDoDmyRy0x7Iw1kuquDyA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by DM6PR13MB4431.namprd13.prod.outlook.com (2603:10b6:5:1bb::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.20; Sat, 15 Oct 2022 07:23:14 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::7c26:1a0b:2825:6f4b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::7c26:1a0b:2825:6f4b%4]) with mapi id 15.20.5723.014; Sat, 15 Oct 2022 07:23:14 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com, Chaoyong He Subject: [PATCH v3] net/nfp: add the feature of RSS based on VXLAN inner layer Date: Sat, 15 Oct 2022 15:22:52 +0800 Message-Id: <1665818572-38639-1-git-send-email-chaoyong.he@corigine.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1665738548-20974-1-git-send-email-chaoyong.he@corigine.com> References: <1665738548-20974-1-git-send-email-chaoyong.he@corigine.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: SI2PR01CA0005.apcprd01.prod.exchangelabs.com (2603:1096:4:191::23) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|DM6PR13MB4431:EE_ X-MS-Office365-Filtering-Correlation-Id: becab549-15df-4f58-db34-08daae7e1f4f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JL3r59CTFh1VebCXOvyRJkeEq7p/VOW833knAWfrt9MfxHOL+yzyD+IhcyoRTxhcpvxJL4Q5o8N6wSuIbCbLQ4abwDDlKUrb/x1YSd7DccSgSs/e8Ge6UMy+SJ6UWliDc0Da3l1UPs+S1kiIu7scvOKS6rgR32paCyBu9bvmsJakf/4LqTwMgXrVqrrJNm8hVje4Vwd91UA3Rc1TVQFCC9ZNGtxodlNissTdI9Rx/SUhJLKIkodFk/Hrf3S0OZevegKAEi8gzSXqcSFhlTdN5Zb2OYjo1uahA4UrYUlICN5nLeqRW/Afeja/XoFHDfZCE0EP3/3en0QGiiVNACui67OLV1PJ2QYM6nX994ZKNlR0r4eEaqot27gqtrZ/lXVwwql2ePD5hDY7ZkWsiE5Uj3T4epzF+QjKtaXvB7ejVKtQWU10E1wwv3rNiXkRcJdpVX/fIiS8yPzy1QKYg9Tdm0gkKvs7Cl2LcK9WAuhHfOELgKbYVh/uu3VYpBYGCkrebGmVsaVokovf1UOrG8LVQCVauwFF2KX8Wjn/2N+u8Z56qHxvVeIeps/TPXufrtII/20ZKSQv/dgywTK/XGUdQgq5CkdE4HPeidKMCVx5sEUvt7h1uslCLI9XXiB0Lsuii2F5gz0RksuqA5K0NUZzAkOiYajjbE+cPSImmQSoL6kKSi4mjqJxroR9aaWqKrOus+gCd7mG4UU00xcmhBSa22ar3OZrikh08ZqUC21XNWNjG2nIzRveUkNzZR+OXf2rDN4LsMSeNZPi58iJyerL1Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(396003)(39830400003)(346002)(366004)(376002)(136003)(451199015)(6506007)(5660300002)(83380400001)(66574015)(478600001)(26005)(52116002)(86362001)(44832011)(66476007)(41300700001)(316002)(66556008)(66946007)(6666004)(2906002)(36756003)(4326008)(107886003)(6512007)(8936002)(2616005)(186003)(8676002)(6486002)(38100700002)(6916009)(38350700002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZXNha0hNbkJQcjNzMHlTSEQvdGJaSnBQVHlwOTVQMm1SZ3IwbUJJZ2JoQ3ZK?= =?utf-8?B?czVFVXRDWk5QalBzZ3RIMnlFMnJHUmEvTm81d2NBemVCTVdOSkxnWjJnMEhB?= =?utf-8?B?b09nM01IT1hoTzJMSjhha0RXcjM0bkkrSjNNZDljenNKUlY4RHpMVGRRVlIv?= =?utf-8?B?YTZJYndndnExZEQ2bnVnNXl5N0hpZ2xWK09QVU85bmEwczdIaTFYWVFBd0Vn?= =?utf-8?B?ZjlIalNBSFU4U25qSmdQV1FScGhzdnlEYnl0aHNra2xKM08vOWh3RjZxZFpL?= =?utf-8?B?VDdqS04vK0llOGFBaEw2cFlHbUM0Slg0REdHY1FUVW9rd3dSVVpTNWI1ZWs0?= =?utf-8?B?ODhNTTV3QUQyNmRuMS96N3haSncyZEphN0hnMXlCTU1Nb05PVjJaYStRdE8y?= =?utf-8?B?ZU4yU3hURmJHWDVobER5SXNzUjBCZHdMb2lqbVl2d1FWNVRvWHlENEprdGV2?= =?utf-8?B?OURGc0hTZkU5U1I5YWZ0TDVLL3ZqanFaMnh2cnN0bTY4cTFhNlNZbHhWT3p4?= =?utf-8?B?bTB0Qm5CcmhrcmNwekxpZ24xRUF0TGZWWTBlbVVTUkxqN0NrR3VTR3BFVVgx?= =?utf-8?B?cGVXQzhqdzZBNDdNekVxbVROd1g0TXpVdnBiVk05UGJlRzFUSVI3eGtkRGlZ?= =?utf-8?B?NnhXdWFvSkxxb1FjeDZ1akRyTHU5RXN0aDRkbEF6OGl1WDI2dFVRZWVLeFhJ?= =?utf-8?B?alJVT0ZlUm9yMnRRMTdMbTE4S0hkSjB4S0NHR0VJbzRxK1NzZks4eTNDR2FK?= =?utf-8?B?ZjZtYUZnNEZtbGlyWWI0azZVRUJsU1QyK09Qc3IxemM5YkpzbElwSmN1azZq?= =?utf-8?B?NzhCTWFVbkpyYXk4K0xTWFcvVkdrY3VqV0xZNlhlektnMXhndGtRcm5pZjdl?= =?utf-8?B?TFJXczdvNVRZbldvSWlUTjRKVEFmRG9IVkRRaUQzMlI3b1krVGVacTIzZGFO?= =?utf-8?B?NVA2Y2V1U3Zkcm9xeERVektwZzhvdmg0ZXF1YUxta05pNnE3Q21ZT00vWG5S?= =?utf-8?B?b0crM3VLSjQvem9LL3J0cWRIaDdVeUlLRFBTaHMra2FnWVRVT1d1RS9LQ0lr?= =?utf-8?B?TENPMWtOL2MwTHpmemttZHBQaWR2Syt1U3ZZY3dlWGk2M0wyOXhlY29HOVdq?= =?utf-8?B?bzlyL3JFWjN0MnUzNHlMOThpdHJYUllFSmtqU1BhOGNsOVg4Z3A0dDNyVlV4?= =?utf-8?B?UFJaUG1ZUXNqZkkvYUJpMWhTTmRDRFlZeC9leGlINVBuQll5ck9XZjE3eGNt?= =?utf-8?B?L1FBR0xVdC9HUVgyTEFQYTZxZGlqSWRaanM2ckx6ZVNUVHhqZm1hbDdaYXZR?= =?utf-8?B?a2o3dlRpQkhjTWlyU0xmWTVBUmRqdnRmcVEwTS9wbXUyR2dvSnRRbU0yb1Ni?= =?utf-8?B?MjlzNnBic1lkellZQ2VNV0ZNY3U3ZVNkVHZuNXRCRHRSZ01yQWZVeG5RU1ds?= =?utf-8?B?V3NoL0pCazkxZFNuV0dvWWhqdVcxUUhWWG4wYnRzUlhvcEVMWnBkRDJEUnI5?= =?utf-8?B?aEJkd2U4bnI1aEl5Nmo2d3ZuRy9laXJ2ZTFFZDhKWFpOU1k0TWRiaFRNRUpk?= =?utf-8?B?NnFqQmc5dnF5VHJjdUtFeUE3K1FhcWRMODZWK3pkbUI0b01jbDcrTUFoMXhQ?= =?utf-8?B?b080V0FXaGpHMW5acjQ1RXlqUWxoNFh1MFdYdHhGUGNzZC83emRnOHZQT2VW?= =?utf-8?B?ZkVCUkZtbWhEc29zSjRoTGdMbnlOcmViWHB1ZTV3S3Q4UGl6cXBURnVmUFht?= =?utf-8?B?anVUbGEyb0JlaDRpQWkrZzZCcG04bVprL2JpY2VXZ0VxQ2x3dENENnJTUnBE?= =?utf-8?B?YUpHOTU3dVg0a2FRMnlCbFAxQ0loam5udGZBWDdacXhxSDlMMHBhUC8vdEpi?= =?utf-8?B?dWJ3dHJieG42RjBnYURrNGxRMVIwWVpaL1RUOGNXcHF3Vk5zaEtSa2ZOZjRS?= =?utf-8?B?czJ6eklsdGwzb0hSZ2JrcTlITlNnSXdJWVgrVGtsWmlKTTRTdVVPeU12M3k5?= =?utf-8?B?akpoYWpjamlsR2tKeHdTaDFzTkI5NGNTZ24zUHNCVzdTcjJjQW56SDF2d1BX?= =?utf-8?B?ZEpxSnZXR1RVZHM3WTQ0RVkyYmJjR1pvLzd1V3g3Q0hOVUd5M3I3bVo4K3lj?= =?utf-8?B?bDhGNFJmdWVYUWxBY0RHdmlGaWY3NEo2alcyYklEZnBzd3NJZjI1U0VyN2VB?= =?utf-8?B?Z3c9PQ==?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: becab549-15df-4f58-db34-08daae7e1f4f X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2022 07:23:14.4222 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KxfuaSrjG5Jqtub3DbQHA/XCc6p6atHfiAxN6uvXUZmUJ1QOeqt+XGVaLNIJXSxBdHUqXIbw0rcJRxQ+MydbX/t0rf5fSawDQ8j3JaHM24w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB4431 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add the logics to support the feature of RSS packets based on the inner layer of VXLAN tunnel. Signed-off-by: Chaoyong He Reviewed-by: Niklas Söderlund * Changes since v2 - Still include 'nfp_ctrl.h' in source files that use symbols from it. * Changes since v1 - Don't expose the dependency relation of header files to source files. Depends-on: series-24819 ("preparation for the rte_flow offload of nfp PMD") --- drivers/net/nfp/nfp_common.c | 32 +++++++++++++ drivers/net/nfp/nfp_common.h | 6 +++ drivers/net/nfp/nfp_ethdev.c | 110 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 148 insertions(+) diff --git a/drivers/net/nfp/nfp_common.c b/drivers/net/nfp/nfp_common.c index 07781a1..9be2e21 100644 --- a/drivers/net/nfp/nfp_common.c +++ b/drivers/net/nfp/nfp_common.c @@ -1370,6 +1370,38 @@ } } +int +nfp_net_set_vxlan_port(struct nfp_net_hw *hw, + size_t idx, + uint16_t port) +{ + int ret; + uint32_t i; + + if (idx > NFP_NET_N_VXLAN_PORTS) { + PMD_DRV_LOG(ERR, "The idx value is out of range."); + return -ERANGE; + } + + hw->vxlan_ports[idx] = port; + + for (i = 0; i < NFP_NET_N_VXLAN_PORTS; i += 2) { + nn_cfg_writel(hw, NFP_NET_CFG_VXLAN_PORT + i * sizeof(port), + (hw->vxlan_ports[i + 1] << 16) | hw->vxlan_ports[i]); + } + + rte_spinlock_lock(&hw->reconfig_lock); + + nn_cfg_writel(hw, NFP_NET_CFG_UPDATE, NFP_NET_CFG_UPDATE_VXLAN); + rte_wmb(); + + ret = __nfp_net_reconfig(hw, NFP_NET_CFG_UPDATE_VXLAN); + + rte_spinlock_unlock(&hw->reconfig_lock); + + return ret; +} + RTE_LOG_REGISTER_SUFFIX(nfp_logtype_init, init, NOTICE); RTE_LOG_REGISTER_SUFFIX(nfp_logtype_driver, driver, NOTICE); RTE_LOG_REGISTER_SUFFIX(nfp_logtype_cpp, cpp, NOTICE); diff --git a/drivers/net/nfp/nfp_common.h b/drivers/net/nfp/nfp_common.h index 27ea9c2..d7861a2 100644 --- a/drivers/net/nfp/nfp_common.h +++ b/drivers/net/nfp/nfp_common.h @@ -14,6 +14,8 @@ #ifndef _NFP_COMMON_H_ #define _NFP_COMMON_H_ +#include "nfp_ctrl.h" + #define NFP_NET_PMD_VERSION "0.1" #define PCI_VENDOR_ID_NETRONOME 0x19ee #define PCI_VENDOR_ID_CORIGINE 0x1da8 @@ -209,6 +211,9 @@ struct nfp_net_hw { int stride_rx; int stride_tx; + uint16_t vxlan_ports[NFP_NET_N_VXLAN_PORTS]; + uint8_t vxlan_usecnt[NFP_NET_N_VXLAN_PORTS]; + uint8_t *qcp_cfg; rte_spinlock_t reconfig_lock; @@ -438,6 +443,7 @@ int nfp_net_rss_hash_conf_get(struct rte_eth_dev *dev, void nfp_net_close_rx_queue(struct rte_eth_dev *dev); void nfp_net_stop_tx_queue(struct rte_eth_dev *dev); void nfp_net_close_tx_queue(struct rte_eth_dev *dev); +int nfp_net_set_vxlan_port(struct nfp_net_hw *hw, size_t idx, uint16_t port); #define NFP_NET_DEV_PRIVATE_TO_HW(adapter)\ (&((struct nfp_net_adapter *)adapter)->hw) diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c index e131021..f11a1b6 100644 --- a/drivers/net/nfp/nfp_ethdev.c +++ b/drivers/net/nfp/nfp_ethdev.c @@ -138,6 +138,10 @@ update |= NFP_NET_CFG_UPDATE_GEN | NFP_NET_CFG_UPDATE_RING; + /* Enable vxlan */ + new_ctrl |= NFP_NET_CFG_CTRL_VXLAN; + update |= NFP_NET_CFG_UPDATE_VXLAN; + if (hw->cap & NFP_NET_CFG_CTRL_RINGCFG) new_ctrl |= NFP_NET_CFG_CTRL_RINGCFG; @@ -319,6 +323,110 @@ return 0; } +static int +nfp_net_find_vxlan_idx(struct nfp_net_hw *hw, + uint16_t port, + uint32_t *idx) +{ + uint32_t i; + int free_idx = -1; + + for (i = 0; i < NFP_NET_N_VXLAN_PORTS; i++) { + if (hw->vxlan_ports[i] == port) { + free_idx = i; + break; + } + + if (hw->vxlan_usecnt[i] == 0) { + free_idx = i; + break; + } + } + + if (free_idx == -1) + return -EINVAL; + + *idx = free_idx; + + return 0; +} + +static int +nfp_udp_tunnel_port_add(struct rte_eth_dev *dev, + struct rte_eth_udp_tunnel *tunnel_udp) +{ + int ret; + uint32_t idx; + uint16_t vxlan_port; + struct nfp_net_hw *hw; + enum rte_eth_tunnel_type tnl_type; + + hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private); + vxlan_port = tunnel_udp->udp_port; + tnl_type = tunnel_udp->prot_type; + + if (tnl_type != RTE_ETH_TUNNEL_TYPE_VXLAN) { + PMD_DRV_LOG(ERR, "Not VXLAN tunnel"); + return -ENOTSUP; + } + + ret = nfp_net_find_vxlan_idx(hw, vxlan_port, &idx); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed find valid vxlan idx"); + return -EINVAL; + } + + if (hw->vxlan_usecnt[idx] == 0) { + ret = nfp_net_set_vxlan_port(hw, idx, vxlan_port); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed set vxlan port"); + return -EINVAL; + } + } + + hw->vxlan_usecnt[idx]++; + + return 0; +} + +static int +nfp_udp_tunnel_port_del(struct rte_eth_dev *dev, + struct rte_eth_udp_tunnel *tunnel_udp) +{ + int ret; + uint32_t idx; + uint16_t vxlan_port; + struct nfp_net_hw *hw; + enum rte_eth_tunnel_type tnl_type; + + hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private); + vxlan_port = tunnel_udp->udp_port; + tnl_type = tunnel_udp->prot_type; + + if (tnl_type != RTE_ETH_TUNNEL_TYPE_VXLAN) { + PMD_DRV_LOG(ERR, "Not VXLAN tunnel"); + return -ENOTSUP; + } + + ret = nfp_net_find_vxlan_idx(hw, vxlan_port, &idx); + if (ret != 0 || hw->vxlan_usecnt[idx] == 0) { + PMD_DRV_LOG(ERR, "Failed find valid vxlan idx"); + return -EINVAL; + } + + hw->vxlan_usecnt[idx]--; + + if (hw->vxlan_usecnt[idx] == 0) { + ret = nfp_net_set_vxlan_port(hw, idx, 0); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed set vxlan port"); + return -EINVAL; + } + } + + return 0; +} + /* Initialise and register driver with DPDK Application */ static const struct eth_dev_ops nfp_net_eth_dev_ops = { .dev_configure = nfp_net_configure, @@ -347,6 +455,8 @@ .tx_queue_release = nfp_net_tx_queue_release, .rx_queue_intr_enable = nfp_rx_queue_intr_enable, .rx_queue_intr_disable = nfp_rx_queue_intr_disable, + .udp_tunnel_port_add = nfp_udp_tunnel_port_add, + .udp_tunnel_port_del = nfp_udp_tunnel_port_del, }; static inline int -- 1.8.3.1