From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70737A0559; Sun, 26 Jun 2022 14:12:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1578441144; Sun, 26 Jun 2022 14:12:57 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60070.outbound.protection.outlook.com [40.107.6.70]) by mails.dpdk.org (Postfix) with ESMTP id C284640223 for ; Sat, 25 Jun 2022 05:19:12 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=enV78KYCCDzJ4IIPW+AVTH9VlVNbtWmfkLsTxC+9b2LlEsLui28CicXgHQ5AL+0Ij0PWgeT7oFmQxSbcEbS2zusp/EkiaOu4rlhof4hfx9oWnyyfdcIqBnXsInYaV+/F1IfSCeRiZMQCqdz3jtyDl9VWFv4OGhd5WSAGtCvyUOhvsVUP9z84ABC/bFDOL/Dv7J742MzP5RNH4VDE/xSqURQJCra/PidXTKp5el24Ozal2XULdrVgI2ivLlP7/mlljy1c67MVXkKrF0koU6GrBYQmniT9uDUgjRDaiKSZ8aSBNqs9i+K82cYW43WMPo1IyY73djK2r2yWNV84sfQrUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zZ8pKueLFmFi6lAL8RbCU1txN/97Hb/IM4xUyfA7uH0=; b=CHxVjaCDpZgh9eHv/dtH+b+3dOWXh3gbMgD1FwbLgpWl2GwCNw3gV+Er6IWlU9baurVl8xJvDXl1vm35e/TPgmLCdaTvtBum7KoxgF9/tbvHcmbZLsIZV3BoJ39hLTbEca2InL5/4p7nF5bjN/x1bxp7aGXrj7YaLz6SsxlBtupNcnASIUjQZsgv8ORHxEC/0fmz6euwm9AZnBZ8W9z1bU4bav4Hx+AC7SzCzzGyJ/w5UCYxIaUeqgxfxAKeXyjzLJpaKi8oZic7kXanJVG7Z2bvxE1Z8n/eJXX8ohLYiBzmlXbktZB0KaKW0jchUEYi79l6Xs9r2mtQfyK7BJok8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zZ8pKueLFmFi6lAL8RbCU1txN/97Hb/IM4xUyfA7uH0=; b=cQH+4ywCZkus7eR8ibYXMZI8UzYI+sqtTPvUhUqrCvbI4g0O36azeFxrt3ErPu0+fGL/PLB/yoxIPSU02WPOZ5L/prabNnbYBJFdXNJT4VGpUjANhkni7r23Tzw1kV3HN2nOtViSiKPB9i3zLI9GmKgLu+A+2uSHcmLVQZ7aFWM= Received: from DB7PR07MB4489.eurprd07.prod.outlook.com (2603:10a6:5:3a::19) by DBBPR07MB7530.eurprd07.prod.outlook.com (2603:10a6:10:1ee::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.9; Sat, 25 Jun 2022 03:19:11 +0000 Received: from DB7PR07MB4489.eurprd07.prod.outlook.com ([fe80::6900:77a9:8c73:a5f]) by DB7PR07MB4489.eurprd07.prod.outlook.com ([fe80::6900:77a9:8c73:a5f%7]) with mapi id 15.20.5373.015; Sat, 25 Jun 2022 03:19:11 +0000 From: Jaeeun Ham To: "Jayatheerthan, Jay" , "dev@dpdk.org" CC: Jerin Jacob Subject: RE: ask for TXA_FLUSH_THRESHOLD change Thread-Topic: ask for TXA_FLUSH_THRESHOLD change Thread-Index: Adh+ueQKpScROpOOTUS3XZ7egyFC0wCkcH4QASRrMHAAmKQh4A== Date: Sat, 25 Jun 2022 03:19:10 +0000 Message-ID: References: In-Reply-To: Accept-Language: ko-KR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ericsson.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6d5ea36a-b866-43a3-bc80-08da56597919 x-ms-traffictypediagnostic: DBBPR07MB7530:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4Vm7N20J7Sn4RMTt9C2CT3sQfvOg3AohfNb2k2FU5Zt6Q0XpTyAO6TD+PTauuaNTJaich1VyqZiD+W1RC9sCNcDLPQdZThDenJmIP9Qd4MnrZnYaowcwIB/WPV66XvVi3iw+3h6pJ91os1Nc0q0nVKDM+Csu0qa1JAz7IME5okkfvJsl13qLpLx9EYS4DbXXysg3FsulHWDpB0cHMSdZ5fP9joKXgJBvcjdHiduHDWqQRKt4gu558DpjO/CWjtGg8OQP6K+WahzWdXReo52gyYhUMGzVmKnwfaVAJHbZTzofsF/FMuiXTJ92xOXAHsktQRQKqgY98SZO5ixE4eXYfzX2j6YQP1HM4UxByLgjHgOf7YO0ujfrJH6jUIKYKg8oqjGEUp07ulF/3q0kigb4HEOoICFfSyQeCa77kUEbMm8N0DqcymCeo8UPVyJKHJzVJRGcPGf87OpWbv0jt8jILBmb/DnjkCfQes6iFtLb14w/yPQW9+P9/FRlVU4DHuFz4Ew0IK36e3CFanhHGhKl0VHjOxLryXOTepnhrAWmTgYGhFFkPJGTl6IfuKXhKkXM8BuNLHHW/w2WZvpXkOr5dV1OFAgUA3YFbzw5SSsUXGeSBegOIU8oWaVeHKCxsRS0qu5n6sT7eJDEr9+/jvnivGbCtbLmhiQRFJ8lsYEDEsbN0siJ8hkW2fO2do3spt+3cEitvlWylhHakTuwyUrpfm7nse405FaIvkic6qmHLIXafzsjuGZvAEa3sgG/LpWBsmowpTqv5YSgk4QhIOnHpb4LpTYEkEcsoHAdcoUdBEWpKqRszsmy8GbVipigkhk+MX5snlr0ld7/bAcEZz4NnYwJ7DmGMVp4Yu1LwcDPOIheyw34D32EYwyOosLh5KEQH+6/FsQnu846Nnj7kut+rRt5kFfy7mkhcVC9Kgm1CJ8= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB7PR07MB4489.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(366004)(136003)(396003)(376002)(39860400002)(2906002)(6506007)(7696005)(82960400001)(53546011)(166002)(76116006)(110136005)(83380400001)(9686003)(316002)(86362001)(478600001)(966005)(52536014)(5660300002)(8936002)(122000001)(26005)(186003)(38100700002)(71200400001)(55016003)(41300700001)(38070700005)(8676002)(66446008)(66946007)(66556008)(66476007)(33656002)(64756008)(4326008)(44832011); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?G14wtd6D7MP7VliigTf6jOG6WDbBM8wSCMPnxpxb2VXL3m9bdGL4OqIELKK1?= =?us-ascii?Q?WidhiQ23SMYMXv/3p59sLrvha4usOECS6vPS/HknQqCPqBAECnqY8E0r+7Af?= =?us-ascii?Q?oyAhkCEre/DSaAGin2wTBaMXiHbQo47BQG9NjeQclSD8uAcgEXs2BI4Sg/lG?= =?us-ascii?Q?XPavc5bGNWMxiEddgq4ttQHbdGHjqj2ytnZHbCMQaoCqxf3zNusoMoF87OSa?= =?us-ascii?Q?Fq2Nluh9ejxNPF10An4TB91Mj71Cd7vfw51bPm5nMgKADRzzUYh8mvgfqjxq?= =?us-ascii?Q?sd9ttDbQ9ipNFdGQeg+UzAgiD9A2P/O7LJ2Q3iZYj4xW0U1Y01V4k6hVShj3?= =?us-ascii?Q?u7K6+zj2Ump+99T/z1P1BdyDktafOFB2OaxKIkA04EmPbyLxECKq7x/0LEEt?= =?us-ascii?Q?GE7xdy4FmdEZ2BHYuAkJ813+Fpkq2QAWwn+gWY203YPLaRbTsnHhJ6DPNcUS?= =?us-ascii?Q?iNYCLu3L0zmCoKQnqstyQgb+Mu1oW9W0QdL7IxniU7t/bU2A5h8mSd4eaMcV?= =?us-ascii?Q?WdZbTq9HztHZqcTBGPjyYVdhgxZxu71LGBsz42WHc17/0y4lddCcbWApUw7e?= =?us-ascii?Q?5J0nog0VC6tnrl0Jt6I7E8PvcVlXzxthJppQ1P7CWVT7v5OOuJ8n5CD2+KyV?= =?us-ascii?Q?rufyItCeu0rPqS/oC1wvTT3f1wxtevl9TE3eLqLx0T+fiCubEB1dpWm2YuHL?= =?us-ascii?Q?sAL6gtRESFzPaZDXrbTv+tzXWB25rk5QaUx0zimaOYwXyDvaCStSLVcyyTYs?= =?us-ascii?Q?MzktkxRfLLvPtLojNE1e4fgoSbuKK9MlGorl2xQVIuy0xEdJ5dh2/J1pBgVy?= =?us-ascii?Q?UG2X6Rm1lYhNckqTlcpMHlhIlfXicvfu1TR2K2xmOi/x31iaqi4NUWzFQQrU?= =?us-ascii?Q?baWjdf560NiK7RaiaGUzcs26C/BrYEZneAzLJK/2C0BV8yGHUszHNcWkBIqp?= =?us-ascii?Q?SujkEiaCgzrVjnhAIiHkAKubdzGE7kJrny8dSGrAD7ylGeP4/npow4mRyGTH?= =?us-ascii?Q?HP8NaxZ8DOK31TvOgH9xAH6v8JQLqkS6B6AxtQPk0jB/TodFTXXVOhivXTaP?= =?us-ascii?Q?LpoL+kpZig5G9JgYFUXsWYBHfaL8WB/+Dmz1FKlzN9R+8zVfIzx2VTI3UL27?= =?us-ascii?Q?3jEX7qsQLSTiVNEvzEhZk8AY+9ePyivq+sze15wubOCj3y8dqEjqh1TCNyGC?= =?us-ascii?Q?532Hd2ky6+Arq0FmGOtRCE0RqJ6InJMmmYmZ1KQ8MemJuLWI7tl9/HW12Ndf?= =?us-ascii?Q?WuR+7MqGhz7ICOJXs99AE55VQp58iOuO3qPY+Vt5tudkDlDuPRofMomRkg2z?= =?us-ascii?Q?qlW7WZVylZOj8xP34o6TY0HMb3Gpr2iFgUkzZq3748AKxNEbGJWc4cPUrwAJ?= =?us-ascii?Q?F5KHchJkJbhgm2BWQjDfgzx2Cnk0o8IOlnAr4qTKZHA8ZdW24yfsWrBwVcuW?= =?us-ascii?Q?Pd61HaUvk59Kl4RMM8WHzFiP9ywaUnQFExMueQXMKAk9aZpuWJkNARpD06JX?= =?us-ascii?Q?cnbx2XPFT2ouPnU3rDAqssAgF8LMwl6iJakOKb9l1s6FOmLMciWG9RIk+led?= =?us-ascii?Q?EvSwFtT+KikrmMmPCKRhmzEeKJ+V+y9wDNaXSSBC?= Content-Type: multipart/alternative; boundary="_000_DB7PR07MB4489846FDDF7BA9FF0D7F8F9F3B79DB7PR07MB4489eurp_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DB7PR07MB4489.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6d5ea36a-b866-43a3-bc80-08da56597919 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jun 2022 03:19:10.9529 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4xHDgxP/mL1WpBIyLn9Pj0hS2SMjJ5HsGmGe3qVeiX+tFvj8Q/yP3aJiJQMJS3VvAV6rLTdVg0Y/pHb8ee8U9A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR07MB7530 X-Mailman-Approved-At: Sun, 26 Jun 2022 14:12:56 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_DB7PR07MB4489846FDDF7BA9FF0D7F8F9F3B79DB7PR07MB4489eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, Packets seem to be silently discarded in the RX NIC while waiting in the bu= ffer, and it does not happen when I use only one worker core. I think multiple worker cores packet processing would have this problem to = handle heavy traffic in parallel. I want to increase buffer size to solve this as below, then is it right to = change only modify the number from 4 to 8? I am trying to gather DPDK and PMD driver log such as "--log-level=3D"7" --= log-level=3D"pmd.net,7". Could you recommend how to set the log-level strin= g to confirm silent packet drop? dpdk-stable-20.11.1/lib/librte_eventdev/rte_event_eth_rx_adapter.c #define BATCH_SIZE 32 #define BLOCK_CNT_THRESHOLD 10 #define ETH_EVENT_BUFFER_SIZE (4*BATCH_SIZE) // e.g. (8*BATCH_SIZE) #define ETH_RX_ADAPTER_SERVICE_NAME_LEN 32 #define ETH_RX_ADAPTER_MEM_NAME_LEN 32 #define RSS_KEY_SIZE 40 /* value written to intr thread pipe to signal thread exit */ #define ETH_BRIDGE_INTR_THREAD_EXIT 1 /* Sentinel value to detect initialized file handle */ #define INIT_FD -1 BR/Jaeeun From: Jaeeun Ham Sent: Wednesday, June 22, 2022 11:14 AM To: Jayatheerthan, Jay ; dev@dpdk.org Cc: Jerin Jacob Subject: RE: ask for TXA_FLUSH_THRESHOLD change Hi, Could you guide me on how to eliminate or reduce tx drop/retry? TXA_FLUSH_THRESHOLD helped somewhat but it was not cleared packet tx drop. =3D=3D=3D=3D=3D=3D=3D=3D[ TX adapter stats ]=3D=3D=3D=3D=3D=3D=3D=3D tx_retry: 17499893 tx_packets: 7501716 tx_dropped: 5132458 BR/Jaeeun From: Jayatheerthan, Jay > Sent: Thursday, June 16, 2022 3:40 PM To: Jaeeun Ham >; d= ev@dpdk.org Cc: Jerin Jacob > Subject: RE: ask for TXA_FLUSH_THRESHOLD change Hi Jaeeun, See my responses inline below. -Jay From: Jaeeun Ham > Sent: Monday, June 13, 2022 5:51 AM To: dev@dpdk.org Cc: Jerin Jacob >; Jayatheert= han, Jay > Subject: ask for TXA_FLUSH_THRESHOLD change Hi, There were latency delay when I increase dpdk(20.11.1) core. (one worker co= re was okay.) When I decrease the TXA_FLUSH_THRESHOLD value(1024 to 32), it was okay It's TXA_FLUSH_THRESHOLD at lib/librte_eventdev/rte_event_eth_tx_adapter.c.= // https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth= _tx_adapter.c?h=3D20.11#n15 When TXA_FLUSH_THRESHOLD value was changed from 1024 to 32, the latency tes= t result was fine on 10 cores for low traffic(DL:20Mbps/UL:17kbps). I think that this can make call for rte_eth_tx_buffer_flush() more frequent= ly. But, I'm not sure whether this approach can cause worse performance or not. Do you have any opinion about this? [Jay] Yes, it will cause rte_eth_tx_buffer_flush() to be called more often.= It can lead to lesser batching benefit. Typical performance vs. latency tr= ade-off decision apply here. Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is patched on DUSG3 from 1024 to = smaller value since DPDK 18.11.2: I'm not aware of any side-effect, I think it is needed to have low enough l= atency even at low traffic rates. For more details see Intel FP 22288. [Jay] Currently, TXA_FLUSH_THRESHOLD is not a configurable attr. TXA_MAX_NB_TX(128) looks the same as CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZ= E(16384), then is it also should be tuned? [Jay] They both are different attributes. TXA_MAX_NB_TX refers to max numbe= r of queues in Tx adapter. CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE refers t= o event buffer size in Rx BD. --- dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base.orig 2020-01= -29 15:05:10.000000000 +0100 +++ dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base 2020-01-29 = 15:11:10.000000000 +0100 @@ -566,9 +566,9 @@ CONFIG_RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=3D100 CONFIG_RTE_MAX_BRIDGE_ETH_INSTANCE=3D4 CONFIG_RTE_BRIDGE_ETH_INTR_RING_SIZE=3D32 -CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D128 +CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D16384 CONFIG_RTE_LIBRTE_BRIDGE_ETH_DEBUG=3Dn -CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D1024 +CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D32 --- dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 20= 21-08-05 23:46:52.051051000 +0200 +++ dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 20= 21-08-06 00:50:07.310766255 +0200 @@ -175,8 +175,8 @@ #define RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100 #define RTE_MAX_BRIDGE_ETH_INSTANCE 4 #define RTE_BRIDGE_ETH_INTR_RING_SIZE 32 -#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128 -#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024 +#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384 +#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 10 #undef RTE_BRIDGE_ETH_TX_MULTI_PKT_EVENT BR/Jaeeun --_000_DB7PR07MB4489846FDDF7BA9FF0D7F8F9F3B79DB7PR07MB4489eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

 

Packets seem to be silently discarded in the RX NIC whi= le waiting in the buffer, and it does not happen when I use only one worker= core.

I think multiple worker cores packet processing would h= ave this problem to handle heavy traffic in parallel.

 

I want to increase buffer size to solve this as below, = then is it right to change only modify the number from 4 to 8?

I am trying to gather DPDK and PMD driver log such as &= quot;--log-level=3D"7" --log-level=3D"pmd.net,7". Could= you recommend how to set the log-level string to confirm silent packet dro= p?

 

dpdk-stable-20.11.1/lib/librte_eventdev/rte_event_eth_rx_adapter.c=

 

#define BATCH_SIZE      32

#define BLOCK_CNT_THRESHOLD 10

#define ETH_EVENT_BUFFER_SIZE   (4*BATCH_SIZE)  // e.g. (8*BAT= CH_SIZE)

 

#define ETH_RX_ADAPTER_SERVICE_NAME_LEN 32

#define ETH_RX_ADAPTER_MEM_NAME_LEN 32

 

#define RSS_KEY_SIZE    40

/* value written to intr thread pipe to signal thread exit */

#define ETH_BRIDGE_INTR_THREAD_EXIT 1

/* Sentinel value to detect initialized file handle */

#define INIT_FD     -1

 

BR/Jaeeun

 

From: Jaeeun Ham
Sent: Wednesday, June 22, 2022 11:14 AM
To: Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; dev@dpdk= .org
Cc: Jerin Jacob <jerinj@marvell.com>
Subject: RE: ask for TXA_FLUSH_THRESHOLD change

 

Hi,

 

Could you guide me on how to eliminate or reduce tx dro= p/retry?

TXA_FLUSH_THRESHOLD helped somewhat but it was not clea= red packet tx drop.

 

=3D=3D=3D=3D=3D=3D=3D=3D[ TX adapter stats ]=3D=3D=3D=3D=3D=3D=3D=3D

tx_retry: 17499893

tx_packets: 7501716

tx_dropped: 5132458

 

BR/Jaeeun

 

From: Jayatheerthan, Jay <= jay.jayatheerthan@intel.com<= /a>>
Sent: Thursday, June 16, 2022 3:40 PM
To: Jaeeun Ham <jaeeun= .ham@ericsson.com>; dev@dpdk.org
Cc: Jerin Jacob <jerinj@mar= vell.com>
Subject: RE: ask for TXA_FLUSH_THRESHOLD change

 

Hi Jae= eun,

See my= responses inline below.

&= nbsp;

-Jay

&= nbsp;

&= nbsp;

From: Jaeeun Ham <jaeeun.ham@ericsson.com>
Sent: Monday, June 13, 2022 5:51 AM
To: dev@dpdk.org
Cc: Jerin Jacob <jerinj@mar= vell.com>; Jayatheerthan, Jay <jay.jayatheerthan@intel.com>
Subject: ask for TXA_FLUSH_THRESHOLD change

 

Hi,

 

There were latency delay when I increase dpdk(20.11.1) = core. (one worker core was okay.)

When I decrease the TXA_FLUSH_THRESHOLD value(1024 to 3= 2), it was okay

 

It’s TXA_FLUSH_THRESHOLD at lib/librte_eventdev/r= te_event_eth_tx_adapter.c. // https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth_tx_= adapter.c?h=3D20.11#n15

When TXA_FLUSH_THRESHOLD value was changed from 1024 to 32, the latency test result was fine on 10 cores for = low traffic(DL:20Mbps/UL:17kbps).

I think that this can make call for rte_eth_tx_buffer_f= lush() more frequently.

But, I’m not sure whether this approach can ca= use worse performance or not.

Do you have any opinion about this?

 

[Jay] Yes, it will cause rte_eth_tx_= buffer_flush() to be called more often. It can lead to lesser batching bene= fit. Typical performance vs. latency trade-off decision apply here.

 

 

Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is patched on= DUSG3 from 1024 to smaller value since DPDK 18.11.2:

I’m not aware of any side-effect, I think it is n= eeded to have low enough latency even at low traffic rates.  For more = details see Intel FP= 22288.

 

[Jay] Currently, TXA_FLUSH_THRESHOLD is not a configurable attr.<= o:p>

 

TXA_MAX_NB_TX(128) looks the same as CONFIG_RTE_= BRIDGE_ETH_EVENT_BUFFER_SIZE(16384), then is it also should be tuned?

 

[Jay] They both are different attrib= utes. TXA_MAX_NB_TX refers to max number of queues in Tx adapter. CONFIG_RTE_= BRIDGE_ETH_EVENT_BUFFER_SIZE refers to event buffer size in Rx BD.

 

 

--- dpdk-3pp-swu= -18.11/dpdk-stable-18.11.2/config/common_base.orig  2020-01-29 15:05:1= 0.000000000 +0100

+++ dpdk-3pp-swu= -18.11/dpdk-stable-18.11.2/config/common_base   2020-01-29 15:11:= 10.000000000 +0100

@@ -566,9 +566,9= @@

CONFIG_RTE_LIBRT= E_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=3D100

CONFIG_RTE_MAX_B= RIDGE_ETH_INSTANCE=3D4

CONFIG_RTE_BRIDG= E_ETH_INTR_RING_SIZE=3D32

-CONFI= G_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D128

+C= ONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D16384

CONFIG_RTE_LIBRT= E_BRIDGE_ETH_DEBUG=3Dn

-CONFI= G_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D1024

+C= ONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D32

 

--- dpdk-3pp-swu= -dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h   2021-08-= 05 23:46:52.051051000 +0200

+++ dpdk-3pp-swu= -dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h   2021-08-= 06 00:50:07.310766255 +0200

@@ -175,8 +175,8= @@

#define RTE_LIBR= TE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100

#define RTE_MAX_= BRIDGE_ETH_INSTANCE 4

#define RTE_BRID= GE_ETH_INTR_RING_SIZE 32

-#defi= ne RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128

-#defi= ne RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024

+#= define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384

+#= define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 10

#undef RTE_BRIDG= E_ETH_TX_MULTI_PKT_EVENT

 

 

BR/Jaeeun

 

--_000_DB7PR07MB4489846FDDF7BA9FF0D7F8F9F3B79DB7PR07MB4489eurp_--