Hi Jaeeun,

See my responses inline below.

-Jay

From: Jaeeun Ham <jaeeun.ham@ericsson.com>
Sent: Monday, June 13, 2022 5:51 AM
To: dev@dpdk.org
Cc: Jerin Jacob <jerinj@marvell.com>; Jayatheerthan, Jay <jay.jayatheerthan@intel.com>
Subject: ask for TXA_FLUSH_THRESHOLD change

Hi,

There were latency delay when I increase dpdk(20.11.1) core. (one worker core was okay.)

When I decrease the TXA_FLUSH_THRESHOLD value(1024 to 32), it was okay

It’s TXA_FLUSH_THRESHOLD at lib/librte_eventdev/rte_event_eth_tx_adapter.c. // https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth_tx_adapter.c?h=20.11#n15

When TXA_FLUSH_THRESHOLD value was changed from 1024 to 32, the latency test result was fine on 10 cores for low traffic(DL:20Mbps/UL:17kbps).

I think that this can make call for rte_eth_tx_buffer_flush() more frequently.

But, I’m not sure whether this approach can cause worse performance or not.

Do you have any opinion about this?

[Jay] Yes, it will cause rte_eth_tx_buffer_flush() to be called more often. It can lead to lesser batching benefit. Typical performance vs. latency trade-off decision apply here.

Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is patched on DUSG3 from 1024 to smaller value since DPDK 18.11.2:

I’m not aware of any side-effect, I think it is needed to have low enough latency even at low traffic rates.  For more details see Intel FP 22288.

[Jay] Currently, TXA_FLUSH_THRESHOLD is not a configurable attr.

TXA_MAX_NB_TX(128) looks the same as CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE(16384), then is it also should be tuned?

[Jay] They both are different attributes. TXA_MAX_NB_TX refers to max number of queues in Tx adapter. CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE refers to event buffer size in Rx BD.

--- dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base.orig  2020-01-29 15:05:10.000000000 +0100

+++ dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base   2020-01-29 15:11:10.000000000 +0100

@@ -566,9 +566,9 @@

CONFIG_RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=100

CONFIG_RTE_MAX_BRIDGE_ETH_INSTANCE=4

CONFIG_RTE_BRIDGE_ETH_INTR_RING_SIZE=32

-CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=128

+CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=16384

CONFIG_RTE_LIBRTE_BRIDGE_ETH_DEBUG=n

-CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=1024

+CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=32

--- dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h   2021-08-05 23:46:52.051051000 +0200

+++ dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h   2021-08-06 00:50:07.310766255 +0200

@@ -175,8 +175,8 @@

#define RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100

#define RTE_MAX_BRIDGE_ETH_INSTANCE 4

#define RTE_BRIDGE_ETH_INTR_RING_SIZE 32

-#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128

-#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024

+#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384

+#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 10

#undef RTE_BRIDGE_ETH_TX_MULTI_PKT_EVENT

BR/Jaeeun