From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2FD2AA04F1; Sun, 8 Dec 2019 13:32:33 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E61B01BF8E; Sun, 8 Dec 2019 13:31:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id AF5321BFA8 for ; Sun, 8 Dec 2019 13:31:53 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xB8CVHX6021943; Sun, 8 Dec 2019 04:31:52 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=FCycw52LXDR+b0Kwm+QUwqvMFcsVnuqtbIZ93EB9iEs=; b=meRlsv37OqVuD4F56fCKiff6/zbd0o/FtMcfPwLwxgwvCJaasYWxXYnJIa3p7DyxQVG6 yL6pBqOQwef6ZpThNZO9UQ2bk4GuoK57mVpEUJEVN9WRjFJtb0CRmfy1BkFhu/EfkVK5 UaL/fg2LxJVoFoOs5UKcWZQNvgmfSIBiqDeulg5UtkpvV3TOZtM3h1g1+wwVcf+EeMJY +Y7HyaAgx4RgsDlvAAp79AZVrqiiAg9w5/6oxi13abY/YLXg0UKYamV2kVF1WF8TLted 4KKFztcPhnvUr43QlqtiAFkKd7QdqjprzQKrLXTpCigE5xwf4HriF4SLzhIWPHa/M68Q NQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2wrbawjm6q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 08 Dec 2019 04:31:52 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sun, 8 Dec 2019 04:31:51 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sun, 8 Dec 2019 04:31:51 -0800 Received: from ajoseph83.caveonetworks.com.com (unknown [10.29.45.60]) by maili.marvell.com (Postfix) with ESMTP id 6DE553F703F; Sun, 8 Dec 2019 04:31:47 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Radu Nicolau , Thomas Monjalon CC: Lukasz Bartosik , Jerin Jacob , Narayana Prasad , Ankur Dwivedi , Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Vamsi Attunuru , Konstantin Ananyev , Date: Sun, 8 Dec 2019 18:00:42 +0530 Message-ID: <1575808249-31135-8-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1575808249-31135-1-git-send-email-anoobj@marvell.com> References: <1575808249-31135-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-12-08_03:2019-12-05,2019-12-08 signatures=0 Subject: [dpdk-dev] [PATCH 07/14] examples/ipsec-secgw: add routines to launch workers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Lukasz Bartosik With eventmode, workers can be drafted differently according to the capabilities of the underlying event device. The added functions will receive an array of such workers and probe the eventmode properties to choose the worker. Signed-off-by: Anoob Joseph Signed-off-by: Lukasz Bartosik --- examples/ipsec-secgw/event_helper.c | 340 ++++++++++++++++++++++++++++++++++++ examples/ipsec-secgw/event_helper.h | 48 +++++ 2 files changed, 388 insertions(+) diff --git a/examples/ipsec-secgw/event_helper.c b/examples/ipsec-secgw/event_helper.c index f120e43..a67132a 100644 --- a/examples/ipsec-secgw/event_helper.c +++ b/examples/ipsec-secgw/event_helper.c @@ -7,9 +7,12 @@ #include #include #include +#include #include "event_helper.h" +static volatile bool eth_core_running; + static int eh_get_enabled_cores(struct rte_bitmap *eth_core_mask) { @@ -97,6 +100,16 @@ eh_get_eventdev_params(struct eventmode_conf *em_conf, uint8_t eventdev_id) return &(em_conf->eventdev_config[i]); } +static inline bool +eh_dev_has_burst_mode(uint8_t dev_id) +{ + struct rte_event_dev_info dev_info; + + rte_event_dev_info_get(dev_id, &dev_info); + return (dev_info.event_dev_cap & RTE_EVENT_DEV_CAP_BURST_MODE) ? + true : false; +} + static int eh_set_default_conf_eventdev(struct eventmode_conf *em_conf) { @@ -704,6 +717,260 @@ eh_initialize_rx_adapter(struct eventmode_conf *em_conf) return 0; } +static int32_t +eh_start_worker_eth_core(struct eventmode_conf *conf, uint32_t lcore_id) +{ + uint32_t service_id[EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE]; + struct rx_adapter_conf *rx_adapter; + struct tx_adapter_conf *tx_adapter; + int service_count = 0; + int adapter_id; + int32_t ret; + int i; + + EH_LOG_INFO("Entering eth_core processing on lcore %u", lcore_id); + + /* + * Parse adapter config to check which of all Rx adapters need + * to be handled by this core. + */ + for (i = 0; i < conf->nb_rx_adapter; i++) { + /* Check if we have exceeded the max allowed */ + if (service_count > EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE) { + EH_LOG_ERR( + "Exceeded the max allowed adapters per rx core"); + break; + } + + rx_adapter = &(conf->rx_adapter[i]); + if (rx_adapter->rx_core_id != lcore_id) + continue; + + /* Adapter is handled by this core */ + adapter_id = rx_adapter->adapter_id; + + /* Get the service ID for the adapters */ + ret = rte_event_eth_rx_adapter_service_id_get(adapter_id, + &(service_id[service_count])); + + if (ret != -ESRCH && ret < 0) { + EH_LOG_ERR( + "Failed to get service id used by rx adapter"); + return ret; + } + + /* Update service count */ + service_count++; + } + + /* + * Parse adapter config to see which all Tx adapters need + * to be handled this core. + */ + for (i = 0; i < conf->nb_tx_adapter; i++) { + /* Check if we have exceeded the max allowed */ + if (service_count > EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE) { + EH_LOG_ERR( + "Exceeded the max allowed adapters per tx core"); + break; + } + + tx_adapter = &conf->tx_adapter[i]; + if (tx_adapter->tx_core_id != lcore_id) + continue; + + /* Adapter is handled by this core */ + adapter_id = tx_adapter->adapter_id; + + /* Get the service ID for the adapters */ + ret = rte_event_eth_tx_adapter_service_id_get(adapter_id, + &(service_id[service_count])); + + if (ret != -ESRCH && ret < 0) { + EH_LOG_ERR( + "Failed to get service id used by tx adapter"); + return ret; + } + + /* Update service count */ + service_count++; + } + + eth_core_running = true; + + while (eth_core_running) { + for (i = 0; i < service_count; i++) { + /* Initiate adapter service */ + rte_service_run_iter_on_app_lcore(service_id[i], 0); + } + } + + return 0; +} + +static int32_t +eh_stop_worker_eth_core(void) +{ + if (eth_core_running) { + EH_LOG_INFO("Stopping eth cores"); + eth_core_running = false; + } + return 0; +} + +static struct eh_app_worker_params * +eh_find_worker(uint32_t lcore_id, struct eh_conf *conf, + struct eh_app_worker_params *app_wrkrs, uint8_t nb_wrkr_param) +{ + struct eh_app_worker_params curr_conf = { + {{0} }, NULL}; + struct eh_event_link_info *link = NULL; + struct eh_app_worker_params *tmp_wrkr; + struct eventmode_conf *em_conf; + uint8_t eventdev_id; + int i; + + /* Get eventmode config */ + em_conf = (struct eventmode_conf *)(conf->mode_params); + + /* + * Use event device from the first lcore-event link. + * + * Assumption: All lcore-event links tied to a core are using the + * same event device. In other words, one core would be polling on + * queues of a single event device only. + */ + + /* Get a link for this lcore */ + for (i = 0; i < em_conf->nb_link; i++) { + link = &(em_conf->link[i]); + if (link->lcore_id == lcore_id) + break; + } + + if (link == NULL) { + EH_LOG_ERR( + "No valid link found for lcore %d", lcore_id); + return NULL; + } + + /* Get event dev ID */ + eventdev_id = link->eventdev_id; + + /* Populate the curr_conf with the capabilities */ + + /* Check for burst mode */ + if (eh_dev_has_burst_mode(eventdev_id)) + curr_conf.cap.burst = EH_RX_TYPE_BURST; + else + curr_conf.cap.burst = EH_RX_TYPE_NON_BURST; + + /* Parse the passed list and see if we have matching capabilities */ + + /* Initialize the pointer used to traverse the list */ + tmp_wrkr = app_wrkrs; + + for (i = 0; i < nb_wrkr_param; i++, tmp_wrkr++) { + + /* Skip this if capabilities are not matching */ + if (tmp_wrkr->cap.u64 != curr_conf.cap.u64) + continue; + + /* If the checks pass, we have a match */ + return tmp_wrkr; + } + + return NULL; +} + +static int +eh_verify_match_worker(struct eh_app_worker_params *match_wrkr) +{ + /* Verify registered worker */ + if (match_wrkr->worker_thread == NULL) { + EH_LOG_ERR("No worker registered"); + return 0; + } + + /* Success */ + return 1; +} + +static uint8_t +eh_get_event_lcore_links(uint32_t lcore_id, struct eh_conf *conf, + struct eh_event_link_info **links) +{ + struct eh_event_link_info *link_cache; + struct eventmode_conf *em_conf = NULL; + struct eh_event_link_info *link; + uint8_t lcore_nb_link = 0; + size_t single_link_size; + size_t cache_size; + int index = 0; + int i; + + if (conf == NULL || links == NULL) { + EH_LOG_ERR("Invalid args"); + return -EINVAL; + } + + /* Get eventmode conf */ + em_conf = (struct eventmode_conf *)(conf->mode_params); + + if (em_conf == NULL) { + EH_LOG_ERR("Invalid event mode parameters"); + return -EINVAL; + } + + /* Get the number of links registered */ + for (i = 0; i < em_conf->nb_link; i++) { + + /* Get link */ + link = &(em_conf->link[i]); + + /* Check if we have link intended for this lcore */ + if (link->lcore_id == lcore_id) { + + /* Update the number of links for this core */ + lcore_nb_link++; + + } + } + + /* Compute size of one entry to be copied */ + single_link_size = sizeof(struct eh_event_link_info); + + /* Compute size of the buffer required */ + cache_size = lcore_nb_link * + sizeof(struct eh_event_link_info); + + /* Compute size of the buffer required */ + link_cache = calloc(1, cache_size); + + /* Get the number of links registered */ + for (i = 0; i < em_conf->nb_link; i++) { + + /* Get link */ + link = &(em_conf->link[i]); + + /* Check if we have link intended for this lcore */ + if (link->lcore_id == lcore_id) { + + /* Cache the link */ + memcpy(&link_cache[index], link, single_link_size); + + /* Update index */ + index++; + } + } + + /* Update the links for application to use the cached links */ + *links = link_cache; + + /* Return the number of cached links */ + return lcore_nb_link; +} + static int eh_tx_adapter_configure(struct eventmode_conf *em_conf, struct tx_adapter_conf *adapter) @@ -1227,6 +1494,79 @@ eh_devs_uninit(struct eh_conf *conf) return 0; } +void +eh_launch_worker(struct eh_conf *conf, struct eh_app_worker_params *app_wrkr, + uint8_t nb_wrkr_param) +{ + struct eh_app_worker_params *match_wrkr; + struct eh_event_link_info *links = NULL; + struct eventmode_conf *em_conf; + uint32_t lcore_id; + uint8_t nb_links; + + if (conf == NULL) { + EH_LOG_ERR("Invalid event helper configuration"); + return; + } + + if (conf->mode_params == NULL) { + EH_LOG_ERR("Invalid event mode parameters"); + return; + } + + /* Get eventmode conf */ + em_conf = (struct eventmode_conf *)(conf->mode_params); + + /* Get core ID */ + lcore_id = rte_lcore_id(); + + /* Check if this is eth core */ + if (rte_bitmap_get(em_conf->eth_core_mask, lcore_id)) { + eh_start_worker_eth_core(em_conf, lcore_id); + return; + } + + if (app_wrkr == NULL || nb_wrkr_param == 0) { + EH_LOG_ERR("Invalid args"); + return; + } + + /* + * This is a regular worker thread. The application registers + * multiple workers with various capabilities. Run worker + * based on the selected capabilities of the event + * device configured. + */ + + /* Get the first matching worker for the event device */ + match_wrkr = eh_find_worker(lcore_id, conf, app_wrkr, nb_wrkr_param); + if (match_wrkr == NULL) { + EH_LOG_ERR("Failed to match worker registered for lcore %d", + lcore_id); + goto clean_and_exit; + } + + /* Verify sanity of the matched worker */ + if (eh_verify_match_worker(match_wrkr) != 1) { + EH_LOG_ERR("Failed to validate the matched worker"); + goto clean_and_exit; + } + + /* Get worker links */ + nb_links = eh_get_event_lcore_links(lcore_id, conf, &links); + + /* Launch the worker thread */ + match_wrkr->worker_thread(links, nb_links); + + /* Free links info memory */ + free(links); + +clean_and_exit: + + /* Flag eth_cores to stop, if started */ + eh_stop_worker_eth_core(); +} + uint8_t eh_get_tx_queue(struct eh_conf *conf, uint8_t eventdev_id) { diff --git a/examples/ipsec-secgw/event_helper.h b/examples/ipsec-secgw/event_helper.h index 3e2627f..aad87f7 100644 --- a/examples/ipsec-secgw/event_helper.h +++ b/examples/ipsec-secgw/event_helper.h @@ -44,6 +44,9 @@ extern "C" { #define EVENT_MODE_MAX_LCORE_LINKS \ (EVENT_MODE_MAX_EVENT_DEVS * EVENT_MODE_MAX_EVENT_QUEUES_PER_DEV) +/* Max adapters that one Rx core can handle */ +#define EVENT_MODE_MAX_ADAPTERS_PER_RX_CORE EVENT_MODE_MAX_RX_ADAPTERS + /* Max adapters that one Tx core can handle */ #define EVENT_MODE_MAX_ADAPTERS_PER_TX_CORE EVENT_MODE_MAX_TX_ADAPTERS @@ -55,6 +58,14 @@ enum eh_pkt_transfer_mode { EH_PKT_TRANSFER_MODE_EVENT, }; +/** + * Event mode packet rx types + */ +enum eh_rx_types { + EH_RX_TYPE_NON_BURST = 0, + EH_RX_TYPE_BURST +}; + /* Event dev params */ struct eventdev_params { uint8_t eventdev_id; @@ -165,6 +176,22 @@ struct eh_conf { /**< Mode specific parameters */ }; +/* Workers registered by the application */ +struct eh_app_worker_params { + union { + RTE_STD_C11 + struct { + uint64_t burst : 1; + /**< Specify status of rx type burst */ + }; + uint64_t u64; + } cap; + /**< Capabilities of this worker */ + void (*worker_thread)(struct eh_event_link_info *links, + uint8_t nb_links); + /**< Worker thread */ +}; + /** * Initialize event mode devices * @@ -232,6 +259,27 @@ eh_get_tx_queue(struct eh_conf *conf, uint8_t eventdev_id); void eh_display_conf(struct eh_conf *conf); + +/** + * Launch eventmode worker + * + * The application can request the eventmode helper subsystem to launch the + * worker based on the capabilities of event device and the options selected + * while initializing the eventmode. + * + * @param conf + * Event helper configuration + * @param app_wrkr + * List of all the workers registered by application, along with its + * capabilities + * @param nb_wrkr_param + * Number of workers passed by the application + * + */ +void +eh_launch_worker(struct eh_conf *conf, struct eh_app_worker_params *app_wrkr, + uint8_t nb_wrkr_param); + #ifdef __cplusplus } #endif -- 2.7.4