From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 27E6CA0C48; Tue, 13 Jul 2021 10:47:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2EF1E4125F; Tue, 13 Jul 2021 10:45:51 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2073.outbound.protection.outlook.com [40.107.237.73]) by mails.dpdk.org (Postfix) with ESMTP id A4FBA4120F for ; Tue, 13 Jul 2021 10:45:49 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ITQKQZsc+cFPI9L5M80ppg9dv9kh+xmuJof10OtUjATV3glqNqN9oqZPOQCpPOEN7Ni5cowMrrhiGudbWRQCfn0xIugbCYf70lg5R7bpzEzqIOlHibCu8gle99bDBo6XRNmsYaEaJg85SpeRf13uN4g+IjetsbRvaVRh/rU/snqQCOtiX9v43cDImCwJzUHcv1tT+tkT8QmEKhVL9uX4v41dm2ph8gPNZmbmDwbNjzogsOPIi8oRs7hs+dhzGojisDOTQAuy5luSe/IPwaXNHmBHFy204rtIGGS2wbI8G3VKzE7KiA2omVrQQLPKrAaSivaD56c3Gg3s3DcXgGiqwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ASOBMvxVFxK3eSbUM1W6ZzumpYV3BpvG2/bL5241gnI=; b=jgxGFZAM9eN56ReIxX3DvqrJkU3SobQ/kl9YxftE/Uh98KaKanO6dHdHHpTuRXj74wQDlMx6jfosQ67v4kTySW9XG659PGUpbEJ3n83lNOTycsmUmirCZeV/FW2KaJLZr8TN0pbc0deVXWrpiINLtEhG/R77a1fePc/NiWcbITduEQVtFP+M8MT9yllHijczPb9XaBIVWQLMYYHXAScgOyLoMrzl2EWe1nP6ereVZ1BLWtBNdshfgMK4uXV52wBNk//NlfuzFUmNc9CZs3t9gvqFVhNF1aBgw6/2hpv+Y8wmeISd0Avf0P9mIVTggBzDVtvMTg6SjkveFNRaxNjFOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ASOBMvxVFxK3eSbUM1W6ZzumpYV3BpvG2/bL5241gnI=; b=ErJHScRmsB5oBP0EHPkqTBZt5KeZFRPo69Zgz9hwHddv8r3npIO88x+8sV8xsUvsf+Ohm+RLUlWdCpAFtiSOyv9r8jn7a01wk065rYc/NBCRtZ2LpumFfTyczKLXN2/W9vOQDgnEDWm9RWGxLMmZkaoy9W4rvvMBDVpbqCvmSNgICrzcRvd/rHSVdTusp8SxuowbISqIP07BEuIRa3AmAGDzFFx2HwHMQcKloTDyciWgdjFgFWm16n0BcmQMeV9ABkJlS5Q8t1iHJdYT8jYgTPWBWRxcKbwExvIcNnl4RX43HSUJteFktMBamQhn6RlX8ImWi7THkL3y6d1c1vYakA== Received: from DM5PR19CA0014.namprd19.prod.outlook.com (2603:10b6:3:151::24) by BL0PR12MB4851.namprd12.prod.outlook.com (2603:10b6:208:1c1::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.19; Tue, 13 Jul 2021 08:45:48 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:3:151:cafe::41) by DM5PR19CA0014.outlook.office365.com (2603:10b6:3:151::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.20 via Frontend Transport; Tue, 13 Jul 2021 08:45:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4308.20 via Frontend Transport; Tue, 13 Jul 2021 08:45:48 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 13 Jul 2021 08:45:46 +0000 From: Suanming Mou To: , CC: , , Date: Tue, 13 Jul 2021 11:44:51 +0300 Message-ID: <20210713084500.19964-18-suanmingm@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20210713084500.19964-1-suanmingm@nvidia.com> References: <20210527093403.1153127-1-suanmingm@nvidia.com> <20210713084500.19964-1-suanmingm@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cfc57423-3b4e-48e5-3817-08d945da9cc4 X-MS-TrafficTypeDiagnostic: BL0PR12MB4851: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:71; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +d7T0YHcWdTBPkOf7HSi+1JwrEmSj0c0Kgj4SKzkG+JuLRpOBkHSJ+EM+tw7Iy6L8TVFBwtCcBb/2i4aSNbYChSFyUJkyGybjk8v7feAYCIACgjIoyWEJbgdumjMyJ/wD/CTwodDtzx6TLt2NVDZZUbHeKnW9SBxS3WE6rXgpD461mK/V1ZivpImjzAR7tGR3uiPofVldFpcH7rf6CfsRT6tdfVNrTmKLrDXx3RiLoYsmliwk5QTnMnELB+H7IpyBTQrtfP0aBQKYXTSj6+xltEBnusZHGFNPEqN/u6ZlwxtcTq7Vn6pGpi1bjRjF2uIXaXpeGzEDvrfiz+sebXnPzg0fORDs74NtZCoHTNogflvDiSKBZCCzrKyaRBR79IZN41aVR4PWBUrusZPWI1iYBku2TDHryRkC8OQ+tjsaXG1d7gBDpd3WM3EbnLtWMK8AANH24jWF83snlgY+3YLcIPO8rln83hrDXE2fX39VewwkVC96A8u75qU/sHtbP98U18sZhphp5lm6khFMKQ7rmhwMrMZhEPFWGKpb4GkOj2IRj3Fs5InDCZ3VYR4u4wQX1c3RcgoBaaUtbuQ5A1NBjXld+/Kqng/WVAMQH4++viUL/+bKO+Kb93681sNwYegW6TUkGrbS814lNtElqP/MsbrSc3p9Mf38fTolFv///bhLmiGojaUkKkMDbq48/a6P863CuIxD8QklR/iufAqyirW5R0luwvyzbnIKdWcKMI= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(376002)(39860400002)(346002)(136003)(396003)(46966006)(36840700001)(83380400001)(7636003)(82740400003)(5660300002)(356005)(2906002)(26005)(36756003)(2616005)(6666004)(8676002)(36906005)(316002)(54906003)(8936002)(16526019)(186003)(6286002)(47076005)(70586007)(86362001)(36860700001)(426003)(70206006)(7696005)(478600001)(55016002)(34020700004)(82310400003)(1076003)(6636002)(336012)(4326008)(110136005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jul 2021 08:45:48.2999 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cfc57423-3b4e-48e5-3817-08d945da9cc4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4851 Subject: [dpdk-dev] [PATCH v6 17/26] common/mlx5: allocate cache list memory individually X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, the list's local cache instance memory is allocated with the list. As the local cache instance array size is RTE_MAX_LCORE, most of the cases the system will only have very limited cores. allocate the instance memory individually per core will be more economic to the memory. This commit changes the instance array to pointer array, allocate the local cache memory only when the core is to be used. Signed-off-by: Suanming Mou Acked-by: Matan Azrad --- drivers/common/mlx5/mlx5_common_utils.c | 62 ++++++++++++++++++------- drivers/common/mlx5/mlx5_common_utils.h | 2 +- 2 files changed, 45 insertions(+), 19 deletions(-) diff --git a/drivers/common/mlx5/mlx5_common_utils.c b/drivers/common/mlx5/mlx5_common_utils.c index ab098b13fa..18d38355fa 100644 --- a/drivers/common/mlx5/mlx5_common_utils.c +++ b/drivers/common/mlx5/mlx5_common_utils.c @@ -15,14 +15,13 @@ static int mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, - bool lcores_share, mlx5_list_create_cb cb_create, + bool lcores_share, struct mlx5_list_cache *gc, + mlx5_list_create_cb cb_create, mlx5_list_match_cb cb_match, mlx5_list_remove_cb cb_remove, mlx5_list_clone_cb cb_clone, mlx5_list_clone_free_cb cb_clone_free) { - int i; - if (!cb_match || !cb_create || !cb_remove || !cb_clone || !cb_clone_free) { rte_errno = EINVAL; @@ -38,9 +37,11 @@ mlx5_list_init(struct mlx5_list *list, const char *name, void *ctx, list->cb_clone = cb_clone; list->cb_clone_free = cb_clone_free; rte_rwlock_init(&list->lock); + if (lcores_share) { + list->cache[RTE_MAX_LCORE] = gc; + LIST_INIT(&list->cache[RTE_MAX_LCORE]->h); + } DRV_LOG(DEBUG, "mlx5 list %s initialized.", list->name); - for (i = 0; i <= RTE_MAX_LCORE; i++) - LIST_INIT(&list->cache[i].h); return 0; } @@ -53,11 +54,16 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_list *list; + struct mlx5_list_cache *gc = NULL; - list = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*list), 0, SOCKET_ID_ANY); + list = mlx5_malloc(MLX5_MEM_ZERO, + sizeof(*list) + (lcores_share ? sizeof(*gc) : 0), + 0, SOCKET_ID_ANY); if (!list) return NULL; - if (mlx5_list_init(list, name, ctx, lcores_share, + if (lcores_share) + gc = (struct mlx5_list_cache *)(list + 1); + if (mlx5_list_init(list, name, ctx, lcores_share, gc, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(list); @@ -69,7 +75,8 @@ mlx5_list_create(const char *name, void *ctx, bool lcores_share, static struct mlx5_list_entry * __list_lookup(struct mlx5_list *list, int lcore_index, void *ctx, bool reuse) { - struct mlx5_list_entry *entry = LIST_FIRST(&list->cache[lcore_index].h); + struct mlx5_list_entry *entry = + LIST_FIRST(&list->cache[lcore_index]->h); uint32_t ret; while (entry != NULL) { @@ -121,14 +128,14 @@ mlx5_list_cache_insert(struct mlx5_list *list, int lcore_index, lentry->ref_cnt = 1u; lentry->gentry = gentry; lentry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, lentry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, lentry, next); return lentry; } static void __list_cache_clean(struct mlx5_list *list, int lcore_index) { - struct mlx5_list_cache *c = &list->cache[lcore_index]; + struct mlx5_list_cache *c = list->cache[lcore_index]; struct mlx5_list_entry *entry = LIST_FIRST(&c->h); uint32_t inv_cnt = __atomic_exchange_n(&c->inv_cnt, 0, __ATOMIC_RELAXED); @@ -161,6 +168,17 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) rte_errno = ENOTSUP; return NULL; } + if (unlikely(!list->cache[lcore_index])) { + list->cache[lcore_index] = mlx5_malloc(0, + sizeof(struct mlx5_list_cache), + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (!list->cache[lcore_index]) { + rte_errno = ENOMEM; + return NULL; + } + list->cache[lcore_index]->inv_cnt = 0; + LIST_INIT(&list->cache[lcore_index]->h); + } /* 0. Free entries that was invalidated by other lcores. */ __list_cache_clean(list, lcore_index); /* 1. Lookup in local cache. */ @@ -186,7 +204,7 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) entry->ref_cnt = 1u; if (!list->lcores_share) { entry->lcore_idx = (uint32_t)lcore_index; - LIST_INSERT_HEAD(&list->cache[lcore_index].h, entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "MLX5 list %s c%d entry %p new: %u.", list->name, lcore_index, (void *)entry, entry->ref_cnt); @@ -217,10 +235,10 @@ mlx5_list_register(struct mlx5_list *list, void *ctx) } } /* 5. Update lists. */ - LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE].h, entry, next); + LIST_INSERT_HEAD(&list->cache[RTE_MAX_LCORE]->h, entry, next); list->gen_cnt++; rte_rwlock_write_unlock(&list->lock); - LIST_INSERT_HEAD(&list->cache[lcore_index].h, local_entry, next); + LIST_INSERT_HEAD(&list->cache[lcore_index]->h, local_entry, next); __atomic_add_fetch(&list->count, 1, __ATOMIC_RELAXED); DRV_LOG(DEBUG, "mlx5 list %s entry %p new: %u.", list->name, (void *)entry, entry->ref_cnt); @@ -245,7 +263,7 @@ mlx5_list_unregister(struct mlx5_list *list, else list->cb_remove(list->ctx, entry); } else if (likely(lcore_idx != -1)) { - __atomic_add_fetch(&list->cache[entry->lcore_idx].inv_cnt, 1, + __atomic_add_fetch(&list->cache[entry->lcore_idx]->inv_cnt, 1, __ATOMIC_RELAXED); } else { return 0; @@ -280,8 +298,10 @@ mlx5_list_uninit(struct mlx5_list *list) MLX5_ASSERT(list); for (i = 0; i <= RTE_MAX_LCORE; i++) { - while (!LIST_EMPTY(&list->cache[i].h)) { - entry = LIST_FIRST(&list->cache[i].h); + if (!list->cache[i]) + continue; + while (!LIST_EMPTY(&list->cache[i]->h)) { + entry = LIST_FIRST(&list->cache[i]->h); LIST_REMOVE(entry, next); if (i == RTE_MAX_LCORE) { list->cb_remove(list->ctx, entry); @@ -292,6 +312,8 @@ mlx5_list_uninit(struct mlx5_list *list) list->cb_clone_free(list->ctx, entry); } } + if (i != RTE_MAX_LCORE) + mlx5_free(list->cache[i]); } } @@ -320,6 +342,7 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, mlx5_list_clone_free_cb cb_clone_free) { struct mlx5_hlist *h; + struct mlx5_list_cache *gc; uint32_t act_size; uint32_t alloc_size; uint32_t i; @@ -333,7 +356,9 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, act_size = size; } alloc_size = sizeof(struct mlx5_hlist) + - sizeof(struct mlx5_hlist_bucket) * act_size; + sizeof(struct mlx5_hlist_bucket) * act_size; + if (lcores_share) + alloc_size += sizeof(struct mlx5_list_cache) * act_size; /* Using zmalloc, then no need to initialize the heads. */ h = mlx5_malloc(MLX5_MEM_ZERO, alloc_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); @@ -345,8 +370,10 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, h->mask = act_size - 1; h->lcores_share = lcores_share; h->direct_key = direct_key; + gc = (struct mlx5_list_cache *)&h->buckets[act_size]; for (i = 0; i < act_size; i++) { if (mlx5_list_init(&h->buckets[i].l, name, ctx, lcores_share, + lcores_share ? &gc[i] : NULL, cb_create, cb_match, cb_remove, cb_clone, cb_clone_free) != 0) { mlx5_free(h); @@ -358,7 +385,6 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key, return h; } - struct mlx5_list_entry * mlx5_hlist_lookup(struct mlx5_hlist *h, uint64_t key, void *ctx) { diff --git a/drivers/common/mlx5/mlx5_common_utils.h b/drivers/common/mlx5/mlx5_common_utils.h index 4bb974fa3e..d49fb64457 100644 --- a/drivers/common/mlx5/mlx5_common_utils.h +++ b/drivers/common/mlx5/mlx5_common_utils.h @@ -104,7 +104,7 @@ struct mlx5_list { mlx5_list_remove_cb cb_remove; /**< entry remove callback. */ mlx5_list_clone_cb cb_clone; /**< entry clone callback. */ mlx5_list_clone_free_cb cb_clone_free; - struct mlx5_list_cache cache[RTE_MAX_LCORE + 1]; + struct mlx5_list_cache *cache[RTE_MAX_LCORE + 1]; /* Lcore cache, last index is the global cache. */ volatile uint32_t gen_cnt; /* List modification may update it. */ volatile uint32_t count; /* number of entries in list. */ -- 2.25.1