From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BE52A034C; Sat, 9 Apr 2022 04:14:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F27AD40041; Sat, 9 Apr 2022 04:14:05 +0200 (CEST) Received: from smtp.tom.com (smtprz14.163.net [106.3.154.247]) by mails.dpdk.org (Postfix) with ESMTP id 32EA54003F for ; Sat, 9 Apr 2022 04:14:03 +0200 (CEST) Received: from my-app01.tom.com (my-app01.tom.com [127.0.0.1]) by freemail01.tom.com (Postfix) with ESMTP id B903B1EA0071 for ; Sat, 9 Apr 2022 10:14:02 +0800 (CST) Received: from my-app01.tom.com (HELO smtp.tom.com) ([127.0.0.1]) by my-app01 (TOM SMTP Server) with SMTP ID -369355248 for ; Sat, 09 Apr 2022 10:14:02 +0800 (CST) Received: from antispam3.tom.com (unknown [172.25.16.54]) by freemail01.tom.com (Postfix) with ESMTP id AEE681EA006E for ; Sat, 9 Apr 2022 10:14:01 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tom.com; s=201807; t=1649470442; bh=ugi0ZvLOFotNAHw2Fx8MxnkXhonGIzUp/1+q7HNI7Xs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uCncsLQPN6trYwhKvD5zfnNmzNXlBBLv0WoBf3SfhokjJkoqumgICh2Jy7OsnT+Jv Q2pFA2do2q1L1aMQvRaEu+FQICaQNnVVljpjdJ/rWklocLwJtb8Nbz3qRESKnsQbFn 6hepNOv+DzjtUgIB758A30UDiRybxYhBTiGjWGpM= Received: from antispam3.tom.com (antispam3.tom.com [127.0.0.1]) by antispam3.tom.com (Postfix) with ESMTP id DCF129C1CB7 for ; Sat, 9 Apr 2022 10:14:01 +0800 (CST) X-Virus-Scanned: Debian amavisd-new at antispam3.tom.com Received: from antispam3.tom.com ([127.0.0.1]) by antispam3.tom.com (antispam3.tom.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fgTqVnuiHcPS for ; Sat, 9 Apr 2022 10:14:00 +0800 (CST) Received: from localhost.localdomain (unknown [113.81.210.244]) by antispam3.tom.com (Postfix) with ESMTPA id 6820D9C083D; Sat, 9 Apr 2022 10:14:00 +0800 (CST) From: Tianli Lai To: dev@dpdk.org Cc: stephen@networkplumber.org Subject: [PATCH v4] examples/kni: add interrupt mode to receive packets Date: Sat, 9 Apr 2022 18:13:46 +0800 Message-Id: <20220409101346.3621-1-laitianli@tom.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220407161205.8633-1-laitianli@tom.com> References: <20220407161205.8633-1-laitianli@tom.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org kni application have two main-loop threads that they CPU utilization are up to 100 percent, this two theads are writing thread and reading thread. I thank set interrupt mode at reading thread would reduce this thread CPU utilization. Signed-off-by: Tianli Lai --- examples/kni/main.c | 91 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 84 insertions(+), 7 deletions(-) diff --git a/examples/kni/main.c b/examples/kni/main.c index e99ef5c38a..d4d7a3daa9 100644 --- a/examples/kni/main.c +++ b/examples/kni/main.c @@ -73,6 +73,7 @@ #define KNI_US_PER_SECOND 1000000 #define KNI_SECOND_PER_DAY 86400 +#define MIN_ZERO_POLL_COUNT 100 #define KNI_MAX_KTHREAD 32 /* @@ -107,6 +108,8 @@ static uint32_t ports_mask = 0; static int promiscuous_on = 0; /* Monitor link status continually. off by default. */ static int monitor_links; +/* rx set in interrupt mode off by default. */ +static int intr_rx_en; /* Structure type for recording kni interface specific stats */ struct kni_interface_stats { @@ -206,7 +209,7 @@ kni_burst_free_mbufs(struct rte_mbuf **pkts, unsigned num) /** * Interface to burst rx and enqueue mbufs into rx_q */ -static void +static int kni_ingress(struct kni_port_params *p) { uint8_t i; @@ -214,9 +217,9 @@ kni_ingress(struct kni_port_params *p) unsigned nb_rx, num; uint32_t nb_kni; struct rte_mbuf *pkts_burst[PKT_BURST_SZ]; - + int ret = 0; if (p == NULL) - return; + return -1; nb_kni = p->nb_kni; port_id = p->port_id; @@ -225,8 +228,10 @@ kni_ingress(struct kni_port_params *p) nb_rx = rte_eth_rx_burst(port_id, 0, pkts_burst, PKT_BURST_SZ); if (unlikely(nb_rx > PKT_BURST_SZ)) { RTE_LOG(ERR, APP, "Error receiving from eth\n"); - return; + return -1; } + if (nb_rx == 0) + ret = 1; /* Burst tx to kni */ num = rte_kni_tx_burst(p->kni[i], pkts_burst, nb_rx); if (num) @@ -239,6 +244,7 @@ kni_ingress(struct kni_port_params *p) kni_stats[port_id].rx_dropped += nb_rx - num; } } + return ret; } /** @@ -277,12 +283,56 @@ kni_egress(struct kni_port_params *p) } } +static int +sleep_until_rx_interrupt(int num, int lcore) +{ + static struct { + bool wakeup; + } __rte_cache_aligned status[RTE_MAX_LCORE]; + struct rte_epoll_event event[num]; + int n; + + if (status[lcore].wakeup) { + RTE_LOG(DEBUG, APP, + "lcore %u sleeps until interrupt triggers\n", + rte_lcore_id()); + } + n = rte_epoll_wait(RTE_EPOLL_PER_THREAD, event, num, 10); + status[lcore].wakeup = n != 0; + + return 0; +} + +static void +turn_on_off_intr(uint16_t port_id, uint16_t queue_id, bool on) +{ + if (on) + rte_eth_dev_rx_intr_enable(port_id, queue_id); + else + rte_eth_dev_rx_intr_disable(port_id, queue_id); +} + +static int event_register(void) +{ + int ret; + + ret = rte_eth_dev_rx_intr_ctl_q(0, 0, + RTE_EPOLL_PER_THREAD, + RTE_INTR_EVENT_ADD, NULL); + if (ret) + return ret; + + return 0; +} + static int main_loop(__rte_unused void *arg) { uint16_t i; int32_t f_stop; int32_t f_pause; + int ret = 0; + uint32_t zero_rx_packet_count = 0; const unsigned lcore_id = rte_lcore_id(); enum lcore_rxtx { LCORE_NONE, @@ -291,12 +341,17 @@ main_loop(__rte_unused void *arg) LCORE_MAX }; enum lcore_rxtx flag = LCORE_NONE; + int intr_en = 0; RTE_ETH_FOREACH_DEV(i) { if (!kni_port_params_array[i]) continue; if (kni_port_params_array[i]->lcore_rx == (uint8_t)lcore_id) { flag = LCORE_RX; + if (intr_rx_en && !event_register()) + intr_en = 1; + else + RTE_LOG(DEBUG, APP, "RX interrupt won't enable.\n"); break; } else if (kni_port_params_array[i]->lcore_tx == (uint8_t)lcore_id) { @@ -316,7 +371,23 @@ main_loop(__rte_unused void *arg) break; if (f_pause) continue; - kni_ingress(kni_port_params_array[i]); + ret = kni_ingress(kni_port_params_array[i]); + if (ret == 1) { + zero_rx_packet_count++; + if (zero_rx_packet_count <= + MIN_ZERO_POLL_COUNT) + continue; + } else + zero_rx_packet_count = 0; + + if (zero_rx_packet_count > 0) { + zero_rx_packet_count = 0; + if (unlikely(intr_en)) { + turn_on_off_intr(i, 0, 1); + sleep_until_rx_interrupt(1, lcore_id); + turn_on_off_intr(i, 0, 0); + } + } } } else if (flag == LCORE_TX) { RTE_LOG(INFO, APP, "Lcore %u is writing to port %d\n", @@ -341,12 +412,13 @@ main_loop(__rte_unused void *arg) static void print_usage(const char *prgname) { - RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m " + RTE_LOG(INFO, APP, "\nUsage: %s [EAL options] -- -p PORTMASK -P -m -I " "[--config (port,lcore_rx,lcore_tx,lcore_kthread...)" "[,(port,lcore_rx,lcore_tx,lcore_kthread...)]]\n" " -p PORTMASK: hex bitmask of ports to use\n" " -P : enable promiscuous mode\n" " -m : enable monitoring of port carrier state\n" + " -I : enable rx interrupt mode\n" " --config (port,lcore_rx,lcore_tx,lcore_kthread...): " "port and lcore configurations\n", prgname); @@ -527,7 +599,7 @@ parse_args(int argc, char **argv) opterr = 0; /* Parse command line */ - while ((opt = getopt_long(argc, argv, "p:Pm", longopts, + while ((opt = getopt_long(argc, argv, "p:PmI", longopts, &longindex)) != EOF) { switch (opt) { case 'p': @@ -539,6 +611,9 @@ parse_args(int argc, char **argv) case 'm': monitor_links = 1; break; + case 'I': + intr_rx_en = 1; + break; case 0: if (!strncmp(longopts[longindex].name, CMDLINE_OPT_CONFIG, @@ -610,6 +685,8 @@ init_port(uint16_t port) if (dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) local_port_conf.txmode.offloads |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; + if (intr_rx_en) + local_port_conf.intr_conf.rxq = 1; ret = rte_eth_dev_configure(port, 1, 1, &local_port_conf); if (ret < 0) rte_exit(EXIT_FAILURE, "Could not configure port%u (%d)\n", -- 2.27.0