From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE45FA0C47; Wed, 18 Aug 2021 11:08:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BE9F411C1; Wed, 18 Aug 2021 11:08:21 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2078.outbound.protection.outlook.com [40.107.237.78]) by mails.dpdk.org (Postfix) with ESMTP id 0FDDA41194 for ; Wed, 18 Aug 2021 11:08:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Oh9Rh2sYXxLvRVHUNWsBvZiNcEsKwSS5Hh1TtD4poXtOeznzavx8XMRJtI/plujHT3BLacBTIMEheGPIJd2FESiMRDv3BqalLgdJ/9YvZGOQsYsl3/P5Ky2korT96uWZJ3PqF71nUW5mbR8ZVrcbjOI4jipkTtW/7QprF3MwVKTsp5MrCxaoQKVLQR3WSuANQlDy1t9dEOMFn1fV/5dOq8eYJMoznywYwhOXpkB7WseWbSMh2nyl97JflN6MdhuHAssi+Zjao5I2X5lqYuWqsCHDV+b/xOdAfJtC3t6VyRi52XoKawtTg+pG8xUobUdYhyo3l296qk3B6N7hfiz+oA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lxDH73fiZolxR1kEoPyesFulrSlFeu9b3KKOO24UJ+w=; b=ZDwKpMn7ekjda2G9zic8B9cQc+M+iUTrNcOlj9z8f89iETI2tfrGds5WU6RAiJks8SJcj9Kz2FHOETzgaTDTtGB9X3SRhv1rPi6iYrv5YKEr5yhXrtHt4I/k3Tu7m4A3kGW49R5fyTNuE4giM90dkdNDpDxbTH4GMulI5huZP9yCaiqhHoCp1d3iHgofvS4kuIRSa/ox1vYUJ72joX1FfilUQDzql2G6pV2umxBqURk9yw7vwqBf6sAEwJhfpPmb+40UnT8DTXHjCiecjd5IYhKg0AksBUkW4DHeJdRH/yl7BNUac25/UPTitTOIcwFenZOYrN25P9CbW1x1pRrldQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lxDH73fiZolxR1kEoPyesFulrSlFeu9b3KKOO24UJ+w=; b=DadFHYP49Fi6cDri9cqQ7RaCg8xCPpj/nYtNFLZNjSoJbtqcK9k8BDuU5N9d/c1rxIH+1IICJ/4nWuW5GlHX0FoCjUl/oTak4HeptFRY3WV+27RKEsEKwF+3xfx4NZAqHGRiZRwyyDbSGtszL5o12nhF2Y7eJloffmlsnRfgmmUtMsKXVtN939rMkU4flY4Z0LXms/9Sp8BrDnmVwbdCB6evha8yx9czYZqFdVfqmrnpZ5QjssxWA4pOsfjFu0xzMboCm+9R5eOIfRtfrPJuXi+pqMT7haNMgQl8z+AjTjo1jm+lG+Soctit6PPa7qSTtE7CYdPGM1288Hn6GrxyHQ== Received: from MWHPR21CA0053.namprd21.prod.outlook.com (2603:10b6:300:db::15) by MN2PR12MB4784.namprd12.prod.outlook.com (2603:10b6:208:39::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Wed, 18 Aug 2021 09:08:18 +0000 Received: from CO1NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:300:db:cafe::a3) by MWHPR21CA0053.outlook.office365.com (2603:10b6:300:db::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.0 via Frontend Transport; Wed, 18 Aug 2021 09:08:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by CO1NAM11FT061.mail.protection.outlook.com (10.13.175.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.16 via Frontend Transport; Wed, 18 Aug 2021 09:08:17 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 18 Aug 2021 09:08:17 +0000 Received: from nvidia.com (172.20.187.5) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 18 Aug 2021 09:08:14 +0000 From: Dmitry Kozlyuk To: CC: Matan Azrad , Olivier Matz , Andrew Rybchenko , Ray Kinsella , Anatoly Burakov Date: Wed, 18 Aug 2021 12:07:52 +0300 Message-ID: <20210818090755.2419483-2-dkozlyuk@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210818090755.2419483-1-dkozlyuk@nvidia.com> References: <20210818090755.2419483-1-dkozlyuk@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5940ae01-d6d3-42e2-034e-08d96227b7e5 X-MS-TrafficTypeDiagnostic: MN2PR12MB4784: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yrETf2vSw9/McHvwfqDvT1IqFpMAnDCTHPkFuQZPXcoBZ1gHENUQnR2KJ6IvHRvzdMRpO7Y3jaDLXuSeRwXD5jgoGs3bxsPMWUB+OQtelNjvsccjWmYjf5bcxEKii/jd6lQl6yQtGViQH1dSB7o5wEnLrWCTXEGbgdUlROvnzcU4pCEHTtPk6copY8KdbT1dtwLS8qCtwOLjr3N2iGaeEp2FrdMd4uIeUL6m7v4Co247u0AkUIz6QsAbSFy7SEmgGzO8R2Nea4+memLEzAo5cEKhzqhKMaFSVb8eDWnl7ZDX+Luyt1PJHys8Olh8duh1eQ0SaKprtJQbsnhd1ub7KKXpM8jKP31RSq78NEU+y0Q3bT0Xe57ZC0l3DmX85x/zQqNz6q8oBpTHh7m+hNtOoAnLtnHPxgOPfttqy/tFGYgvUys6k8TmLQjvXTyC7g/J72ZWWEkUCsuxIM2u3G4EuyCid08pZQrzOEix0/nZ3CJeRmDxLKsq0grFnrY5JNhdTDZkTX2sBiLjSKOB1uM/ipqR4c0OFbTOrbV9dTl7oDrSApbwuznlfRdGPdzc5BfKGScjnja266W7ELe7xvOu1Un/9imssDumLsdTmK2qTXauXzjWUMosLDRNJ4fvev6R0oTO/mYSUv9lwPfIbRyrsw4VeVeGK9dvjUALi5IgZAYyxWpPinKT9BcUIhWdzGLVUlH3Zle1jjpJaqyAxbB5wA== X-Forefront-Antispam-Report: CIP:216.228.112.35; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid02.nvidia.com; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(136003)(39860400002)(46966006)(36840700001)(83380400001)(1076003)(7636003)(316002)(55016002)(478600001)(2906002)(86362001)(6916009)(356005)(54906003)(16526019)(82740400003)(36756003)(6666004)(4326008)(47076005)(8676002)(70206006)(7696005)(70586007)(26005)(186003)(2616005)(6286002)(8936002)(82310400003)(336012)(5660300002)(426003)(36860700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2021 09:08:17.7022 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5940ae01-d6d3-42e2-034e-08d96227b7e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.35]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4784 Subject: [dpdk-dev] [PATCH 1/4] mempool: add event callbacks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Performance of MLX5 PMD of different classes can benefit if PMD knows which memory it will need to handle in advance, before the first mbuf is sent to the PMD. It is impractical, however, to consider all allocated memory for this purpose. Most often mbuf memory comes from mempools that can come and go. PMD can enumerate existing mempools on device start, but it also needs to track mempool creation and destruction after the forwarding starts but before an mbuf from the new mempool is sent to the device. Add an internal API to register callback for mempool lify cycle events, currently RTE_MEMPOOL_EVENT_CREATE and RTE_MEMPOOL_EVENT_DESTROY: * rte_mempool_event_callback_register() * rte_mempool_event_callback_unregister() Signed-off-by: Dmitry Kozlyuk Acked-by: Matan Azrad --- lib/mempool/rte_mempool.c | 153 ++++++++++++++++++++++++++++++++++++-- lib/mempool/rte_mempool.h | 56 ++++++++++++++ lib/mempool/version.map | 8 ++ 3 files changed, 212 insertions(+), 5 deletions(-) diff --git a/lib/mempool/rte_mempool.c b/lib/mempool/rte_mempool.c index 59a588425b..0ec56ad278 100644 --- a/lib/mempool/rte_mempool.c +++ b/lib/mempool/rte_mempool.c @@ -42,6 +42,18 @@ static struct rte_tailq_elem rte_mempool_tailq = { }; EAL_REGISTER_TAILQ(rte_mempool_tailq) +TAILQ_HEAD(mempool_callback_list, rte_tailq_entry); + +static struct rte_tailq_elem callback_tailq = { + .name = "RTE_MEMPOOL_CALLBACK", +}; +EAL_REGISTER_TAILQ(callback_tailq) + +/* Invoke all registered mempool event callbacks. */ +static void +mempool_event_callback_invoke(enum rte_mempool_event event, + struct rte_mempool *mp); + #define CACHE_FLUSHTHRESH_MULTIPLIER 1.5 #define CALC_CACHE_FLUSHTHRESH(c) \ ((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER)) @@ -722,6 +734,7 @@ rte_mempool_free(struct rte_mempool *mp) } rte_mcfg_tailq_write_unlock(); + mempool_event_callback_invoke(RTE_MEMPOOL_EVENT_DESTROY, mp); rte_mempool_trace_free(mp); rte_mempool_free_memchunks(mp); rte_mempool_ops_free(mp); @@ -778,10 +791,10 @@ rte_mempool_cache_free(struct rte_mempool_cache *cache) } /* create an empty mempool */ -struct rte_mempool * -rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size, - unsigned cache_size, unsigned private_data_size, - int socket_id, unsigned flags) +static struct rte_mempool * +mempool_create_empty(const char *name, unsigned int n, + unsigned int elt_size, unsigned int cache_size, + unsigned int private_data_size, int socket_id, unsigned int flags) { char mz_name[RTE_MEMZONE_NAMESIZE]; struct rte_mempool_list *mempool_list; @@ -915,6 +928,19 @@ rte_mempool_create_empty(const char *name, unsigned n, unsigned elt_size, return NULL; } +struct rte_mempool * +rte_mempool_create_empty(const char *name, unsigned int n, + unsigned int elt_size, unsigned int cache_size, + unsigned int private_data_size, int socket_id, unsigned int flags) +{ + struct rte_mempool *mp; + + mp = mempool_create_empty(name, n, elt_size, cache_size, + private_data_size, socket_id, flags); + mempool_event_callback_invoke(RTE_MEMPOOL_EVENT_CREATE, mp); + return mp; +} + /* create the mempool */ struct rte_mempool * rte_mempool_create(const char *name, unsigned n, unsigned elt_size, @@ -926,7 +952,7 @@ rte_mempool_create(const char *name, unsigned n, unsigned elt_size, int ret; struct rte_mempool *mp; - mp = rte_mempool_create_empty(name, n, elt_size, cache_size, + mp = mempool_create_empty(name, n, elt_size, cache_size, private_data_size, socket_id, flags); if (mp == NULL) return NULL; @@ -958,6 +984,8 @@ rte_mempool_create(const char *name, unsigned n, unsigned elt_size, if (obj_init) rte_mempool_obj_iter(mp, obj_init, obj_init_arg); + mempool_event_callback_invoke(RTE_MEMPOOL_EVENT_CREATE, mp); + rte_mempool_trace_create(name, n, elt_size, cache_size, private_data_size, mp_init, mp_init_arg, obj_init, obj_init_arg, flags, mp); @@ -1343,3 +1371,118 @@ void rte_mempool_walk(void (*func)(struct rte_mempool *, void *), rte_mcfg_mempool_read_unlock(); } + +struct mempool_callback { + rte_mempool_event_callback *func; + void *arg; +}; + +static void +mempool_event_callback_invoke(enum rte_mempool_event event, + struct rte_mempool *mp) +{ + struct mempool_callback_list *list; + struct rte_tailq_entry *te; + void *tmp_te; + + rte_mcfg_tailq_read_lock(); + list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); + TAILQ_FOREACH_SAFE(te, list, next, tmp_te) { + struct mempool_callback *cb = te->data; + rte_mcfg_tailq_read_unlock(); + cb->func(event, mp, cb->arg); + rte_mcfg_tailq_read_lock(); + } + rte_mcfg_tailq_read_unlock(); +} + +int +rte_mempool_event_callback_register(rte_mempool_event_callback *func, + void *arg) +{ + struct mempool_callback_list *list; + struct rte_tailq_entry *te = NULL; + struct mempool_callback *cb; + void *tmp_te; + int ret; + + rte_mcfg_mempool_read_lock(); + rte_mcfg_tailq_write_lock(); + + list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); + TAILQ_FOREACH_SAFE(te, list, next, tmp_te) { + struct mempool_callback *cb = + (struct mempool_callback *)te->data; + if (cb->func == func && cb->arg == arg) { + ret = -EEXIST; + goto exit; + } + } + + te = rte_zmalloc("MEMPOOL_TAILQ_ENTRY", sizeof(*te), 0); + if (te == NULL) { + RTE_LOG(ERR, MEMPOOL, + "Cannot allocate event callback tailq entry!\n"); + ret = -ENOMEM; + goto exit; + } + + cb = rte_malloc("MEMPOOL_EVENT_CALLBACK", sizeof(*cb), 0); + if (cb == NULL) { + RTE_LOG(ERR, MEMPOOL, + "Cannot allocate event callback!\n"); + rte_free(te); + ret = -ENOMEM; + goto exit; + } + + cb->func = func; + cb->arg = arg; + te->data = cb; + TAILQ_INSERT_TAIL(list, te, next); + ret = 0; + +exit: + rte_mcfg_tailq_write_unlock(); + rte_mcfg_mempool_read_unlock(); + rte_errno = -ret; + return ret; +} + +int +rte_mempool_event_callback_unregister(rte_mempool_event_callback *func, + void *arg) +{ + struct mempool_callback_list *list; + struct rte_tailq_entry *te = NULL; + struct mempool_callback *cb; + int ret; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + rte_errno = EPERM; + return -1; + } + + rte_mcfg_mempool_read_lock(); + rte_mcfg_tailq_write_lock(); + ret = -ENOENT; + list = RTE_TAILQ_CAST(callback_tailq.head, mempool_callback_list); + TAILQ_FOREACH(te, list, next) { + cb = (struct mempool_callback *)te->data; + if (cb->func == func && cb->arg == arg) + break; + } + if (te != NULL) { + TAILQ_REMOVE(list, te, next); + ret = 0; + } + rte_mcfg_tailq_write_unlock(); + rte_mcfg_mempool_read_unlock(); + + if (ret == 0) { + rte_free(te); + rte_free(cb); + } + rte_errno = -ret; + return ret; +} diff --git a/lib/mempool/rte_mempool.h b/lib/mempool/rte_mempool.h index 4235d6f0bf..1e9b8f0229 100644 --- a/lib/mempool/rte_mempool.h +++ b/lib/mempool/rte_mempool.h @@ -1775,6 +1775,62 @@ void rte_mempool_walk(void (*func)(struct rte_mempool *, void *arg), int rte_mempool_get_page_size(struct rte_mempool *mp, size_t *pg_sz); +/** + * Mempool event type. + * @internal + */ +enum rte_mempool_event { + /** Occurs after a successful mempool creation. */ + RTE_MEMPOOL_EVENT_CREATE = 0, + /** Occurs before destruction of a mempool begins. */ + RTE_MEMPOOL_EVENT_DESTROY = 1, +}; + +/** + * @internal + * Mempool event callback. + */ +typedef void (rte_mempool_event_callback)( + enum rte_mempool_event event, + struct rte_mempool *mp, + void *arg); + +/** + * @internal + * Register a callback invoked on mempool life cycle event. + * Callbacks will be invoked in the process that creates the mempool. + * + * @param cb + * Callback function. + * @param cb_arg + * User data. + * + * @return + * 0 on success, negative on failure and rte_errno is set. + */ +__rte_internal +int +rte_mempool_event_callback_register(rte_mempool_event_callback *cb, + void *cb_arg); + +/** + * @internal + * Unregister a callback added with rte_mempool_event_callback_register(). + * @p cb and @p arg must exactly match registration parameters. + * + * @param cb + * Callback function. + * @param cb_arg + * User data. + * + * @return + * 0 on success, negative on failure and rte_errno is set. + */ +__rte_internal +int +rte_mempool_event_callback_unregister(rte_mempool_event_callback *cb, + void *cb_arg); + #ifdef __cplusplus } #endif diff --git a/lib/mempool/version.map b/lib/mempool/version.map index 9f77da6fff..1b7d7c5456 100644 --- a/lib/mempool/version.map +++ b/lib/mempool/version.map @@ -64,3 +64,11 @@ EXPERIMENTAL { __rte_mempool_trace_ops_free; __rte_mempool_trace_set_ops_byname; }; + +INTERNAL { + global: + + # added in 21.11 + rte_mempool_event_callback_register; + rte_mempool_event_callback_unregister; +}; -- 2.25.1