From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A2378A055B; Mon, 27 Jun 2022 08:23:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4461F410EF; Mon, 27 Jun 2022 08:23:54 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 7D42B40E78 for ; Mon, 27 Jun 2022 08:23:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656311032; x=1687847032; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=6h0eCCKozM4RKRD4Ek68cpq9tp7BRHHG+8zTCR5QJ58=; b=VIPIEy4RAdNi+12alDtVDPbkrWlLyMQX7DmDHSl37HPbpSjM3Hw05lGA AovzmV7A/1O9WDinGgEG2DvgWNNTyCODJ0f1mgUE6qxyEh7xXGWyewkI8 5aZ4+GMUJE/LuqWE2uP+hCJQzfqRXvXu7sPQw2hBTu1Yd/VSDl6v6z762 yRjJh5WDNg42xKaQVlDsTCWYPEJajkbIEgE7EYglxpztx6gqKd1s3M9eF H5kjCt8vZO9QcxJ1EaJ0YAEOk6X7GSAxnDDVrq1n/Hp+DA5Xa3pp+i/kE B6kHrmUuC3e4PLdz57kM71eZxiDcEWUVIMdlEIPdYelgSaxD+ahVU4KO2 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10390"; a="270113308" X-IronPort-AV: E=Sophos;i="5.92,225,1650956400"; d="scan'208,217";a="270113308" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jun 2022 23:23:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,225,1650956400"; d="scan'208,217";a="587321097" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga007.jf.intel.com with ESMTP; 26 Jun 2022 23:23:50 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sun, 26 Jun 2022 23:23:50 -0700 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sun, 26 Jun 2022 23:23:49 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Sun, 26 Jun 2022 23:23:49 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Sun, 26 Jun 2022 23:23:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KHnV1kbKGnFKXAj4zLScU6VL43l7xZlHl7ZhmJqYTiw0isAUKLyl3ZW7OKrPnqDs4I/p6u4rgUm5YVcIynCzPsWn4gEHMGuQ9k8VB+dBu1BooQu4tlpNzXzcx5oLrFS7dP3Qoop91Q6xQxxWBLtamE5afp+kHnnZSKzLtzTMLMRTJBl0cMfiqJcHN3iFqRozJvQX4TteF7k87Rs6yWhsWqOnG4v2lVRJmVDS6uOJCcgeGaIBxAohG3kiJOeGvpzJwgBHZ/6Wt4oinXTHzV5wyI/Uj6CjwI+JdDU0qg2NNb5qUZAeptBEHt4GM9nT7d0Elp+iW5MsZxdak1MRgm1bfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=viRB1Kw+qpdc+nQxasBJu5M1A2EdxVaoS+R0guXhxZY=; b=h7zEHIaP7NWjPElD+gpMJYyWtfbUmDDqF4DVUhbZVtlNUECtazmmch+1CsTlIfSwuSXUntTgmxu/YykEXSHSaNO7RPgNhcPbcOJKWvvXqKJChksRvQR2K+tGl+CfOQv1jNwx5gTrb6CSt2T0OjfP8tg6NBWZLKcgUtJmwyrYtcaSpCA7lI2e1N5EKjeoG+8TGnUHr6XxtgEE++91U/haGBGXjw15hwL1Jonft+CzpzNyz1nSUdbtAXzPwyNAOV1vHHWU3Vt6xsBWCFL5HAScsznvx/lcObHYRS1Qg97zV0yq+XZXJXjpelc5/p6EZZq+ZJ/jlgCcXGswcPNfu9WWEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB4348.namprd11.prod.outlook.com (2603:10b6:5:1db::18) by DM4PR11MB6094.namprd11.prod.outlook.com (2603:10b6:8:ab::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15; Mon, 27 Jun 2022 06:23:47 +0000 Received: from DM6PR11MB4348.namprd11.prod.outlook.com ([fe80::d57f:4e1a:5ec7:118]) by DM6PR11MB4348.namprd11.prod.outlook.com ([fe80::d57f:4e1a:5ec7:118%7]) with mapi id 15.20.5373.018; Mon, 27 Jun 2022 06:23:47 +0000 From: "Jayatheerthan, Jay" To: Jaeeun Ham , "dev@dpdk.org" CC: Jerin Jacob Subject: RE: ask for TXA_FLUSH_THRESHOLD change Thread-Topic: ask for TXA_FLUSH_THRESHOLD change Thread-Index: Adh+ueQKpScROpOOTUS3XZ7egyFC0wCkcH4QASRrMHAAmKQh4ABrKqmQ Date: Mon, 27 Jun 2022 06:23:47 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e5708c76-9a1f-421b-11c2-08da580597f2 x-ms-traffictypediagnostic: DM4PR11MB6094:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ho3dOM/S7y4QzopPsabF/WeNmqAsVTobrZqyySlQZAj0bzIdSET2p8x0KNiVGGswqueI3ShY+K0rWe0kqHUkOzBqbHm/L5tugiIvPAYeIOOeo9LYKyJurNX/vsu9CWQfSNxv6V6mXkBJ6KIjZdIUM1q1xW84J1WC4Gqf0gzAnTUePc3AvaR+IGE3e/xW2hacx4Teay3WenuTtPKWs34F8zxEF4CoZQs9vDxnJKnV7fHCma6L5D9IYRZ5ey8D429bUZPQfTNqrsjParshhlBnpW2TGW3oUFJakQcvCihVve29P2WY5REVBVHsMeWbuwvtRKH6MDKMijuV7TZHf88u5P7/FacCoEpGm9fD7RWgNBQrmU6ZmSIBKhNmAMBIAemUZaMgQmNHe7aNBDf3tI7OzjLe4e443+diYGs//6FyqnEaE2itwqnLO95RfhcEKfcsxU3xJ0lDj+EkKYz4PWIc5GBwgW4JyWLJwWXAsV6qvMaqfJIBCPy/PlMrfjZOLoa/7XP9yEtBid1lTrYjNB8RhjJXOeS5CP+DV8lfBGXf+bzod1sPwwRqC20M8onPlIET95KlV8hLkhSOHLb2Ta5ocq9vRc/o/6gvdT9Uw4cvTQXMUpjdgCp0+lS9jnQye2ioPUiYBfTS5Wu3f6t+38deZL1DJBiyUrewXYFI8yJWifBF0azswHJGh1lXCEEgD8NWht9r1W7L0PECQqDRX7Uze4kuFLgBOI2wgiZ/N6Cr3N0vkFyChlzGuSJED6qUQc2OH5tIdl+JgLjF30wu6EoytQXuUq9ulh5p5VV/GlT1s+q+zfBfccs1Q0BWh00MaQsZXbJAhsVv96QiO6z12DUE+7exGYn+lLPN1ZdHTrEEU4T3jNRjz6MZHoec4HXZRmET x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4348.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(376002)(136003)(39860400002)(396003)(346002)(366004)(5660300002)(966005)(66476007)(9686003)(52536014)(38070700005)(8676002)(66556008)(76116006)(9326002)(21615005)(26005)(8936002)(166002)(478600001)(122000001)(2906002)(83380400001)(41300700001)(55016003)(7696005)(71200400001)(38100700002)(53546011)(86362001)(186003)(33656002)(82960400001)(66946007)(64756008)(4326008)(66446008)(110136005)(316002)(6506007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?97Bs2T6bXwJCxTk7lAeRWQSfFwJJIc01V1j3fykjZw0h3ruNBh9T854rrpHm?= =?us-ascii?Q?1DW+D2d5ExqwB9qQqFyoSCXs/M46qTt4JOEPVXpOrY1Ng7VqXJVD81bHp5ki?= =?us-ascii?Q?ZLSSMn3WhDDFFxcI8pt5IMUQFYMkWEy+p0zWd/WCNiasw6sOlSoo0wNuCooA?= =?us-ascii?Q?ycxDynur2rhaauQe/RYYEBPv3Z6z7s3SXOmPI/3Br7Y/lYXa2QyjQ46IJtht?= =?us-ascii?Q?DUlerGuT/Dn2vTtow25spBoawA/nSP1EGL+A6KS76CZAW3xckKOId5ifx69Z?= =?us-ascii?Q?4wLPzvNSm6NYL5N4kMMqyUuVRWu0x5eLJK2MiJSprUHH3g8LGPiLN0QWJxtX?= =?us-ascii?Q?jbNMvGKyUIqFnn9POPbVo6wPAgIRe4p4hOytjjQeKy4NZC3EvxFVzbZSD3Ld?= =?us-ascii?Q?sYl8QoQmq7gQUxYcOoCDYbVQQ2Jnh08wdYWYJ64frXXj/qVbsaVLIBkgcJ5g?= =?us-ascii?Q?0SAbUCaZZPEqassdLc/ltdDJoODTQobGLyIVI5tXNLAxGtzDxTwQWv2ar4Ve?= =?us-ascii?Q?NtCxOf0Wd7JSAH8RuEHpIIS+ulpSYsiHu+rLISdoDAQ1n4V2t2IWUfNEc992?= =?us-ascii?Q?hUvvvMshe9/AR77oixyX0WvHC+4nJcnTo1l02PTsGB4+a13q9FIIPYjMzY/f?= =?us-ascii?Q?WXWS1cZCrdNRKsCZnTO95GA+TKj4dqor/0gZykxEBHIejJCRVuc8RbPKDJJX?= =?us-ascii?Q?s5CMJeyWEJ9z0huIp3gZZHDC8l4Pnp3sUoxYc3oRbgoNuW62xEUxD8QvkHIr?= =?us-ascii?Q?tc4OuaV/5a/QcUvcjiByaXmXF1+dfEGJZ6LRBCSUDvZnmHIEamzjuShBjvKq?= =?us-ascii?Q?Mzvpv+qVvs3CRUDbGznAvxsYRd9MjytNhF5NvYhgrMX9XiUiZhv2n9va1Fmi?= =?us-ascii?Q?/SC6nN/OpfhBSjdRGzk1pxO9tcaNE1DlPV05Nydm4MWLtGLR/LGWyvGT10kP?= =?us-ascii?Q?JG2VRcnCFloCNZKfEZWTKqhJfAkSEfZtMVSM/VfhpuPUmQYpfwqCJbCFi27h?= =?us-ascii?Q?X2AzDHNX2bhaN6flUz84dNzBvrAIrxsGoh559n6MAp5VNJQ78Iiap2HMBAtE?= =?us-ascii?Q?AKsgMfs/ANg1EIqrTC8XwY55Z7Md6dEk9+o8bEI7ZSOUbG5m8D0AAL44K2I7?= =?us-ascii?Q?t1AvIy42Fv+GWHW6BOVDaAnicNiLUDUw/XqNYpGH9IZHSa7AEtfEmsU48TbW?= =?us-ascii?Q?AAetmCA46mFGgdd68+ERmFM4OgVvHr0vKpe/0cDEb72fJE62c5ZyruGJMixb?= =?us-ascii?Q?IwL5e1T9DBWejlyx5z6NusnnKEw6ImceYEFJ1Ec3nmyRcgvT+Szrqw1kem5A?= =?us-ascii?Q?1Fz9+aZO1thjUCoJrxsrvcmDNDqKO6A+ZONCYnQ8PsZioGBgvaNZOtDunShn?= =?us-ascii?Q?UU/xP4olixXW9DqBSJx9u9+AnvTR6BQsWJjCwv79qgrX0Sw88oHA5c3r8/6a?= =?us-ascii?Q?hZ95VRA7wXDNQYHJs4U+q5cqHd3rEkAhKzSPgp7YBhDwqp3L5lSZdkIVCCmx?= =?us-ascii?Q?u4QUsdDPTf0GQknbrzReVl8qs4rJAxF3yw/261UIfW0BpmuzovRZaXGWk8Fu?= =?us-ascii?Q?qFTcXTGtIrorLeT6OHGNlMLPk4bJ1c2QwBdu/1K/Efk8536fH3/jNBB78KPy?= =?us-ascii?Q?FQ=3D=3D?= Content-Type: multipart/alternative; boundary="_000_DM6PR11MB4348F53414D1F3846494157CFDB99DM6PR11MB4348namp_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4348.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e5708c76-9a1f-421b-11c2-08da580597f2 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Jun 2022 06:23:47.3026 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: RVpEwzQnfko/D19+GAMh9tmvj+p/nVlmUMxgn+panFioJKvNNtWBI7VkoOCR5e1XetpG8bga4q++s5ixbusyrtzAzRyMnyK1fIv0KciSe/s= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6094 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_DM6PR11MB4348F53414D1F3846494157CFDB99DM6PR11MB4348namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, >From DPDK 21.11, the Rx adapter event buffer size is configurable through A= PI during creation time. Please refer to rte_event_eth_rx_adapter_create_wi= th_params() API in the below link: https://doc.dpdk.org/api/rte__event__eth__rx__adapter_8h.html You could refer to EAL docs on how to use log-level option based on the PMD= you are using. -Jay From: Jaeeun Ham Sent: Saturday, June 25, 2022 8:49 AM To: Jayatheerthan, Jay ; dev@dpdk.org Cc: Jerin Jacob Subject: RE: ask for TXA_FLUSH_THRESHOLD change Hi, Packets seem to be silently discarded in the RX NIC while waiting in the bu= ffer, and it does not happen when I use only one worker core. I think multiple worker cores packet processing would have this problem to = handle heavy traffic in parallel. I want to increase buffer size to solve this as below, then is it right to = change only modify the number from 4 to 8? I am trying to gather DPDK and PMD driver log such as "--log-level=3D"7" --= log-level=3D"pmd.net,7". Could you recommend how to set the log-level strin= g to confirm silent packet drop? dpdk-stable-20.11.1/lib/librte_eventdev/rte_event_eth_rx_adapter.c #define BATCH_SIZE 32 #define BLOCK_CNT_THRESHOLD 10 #define ETH_EVENT_BUFFER_SIZE (4*BATCH_SIZE) // e.g. (8*BATCH_SIZE) #define ETH_RX_ADAPTER_SERVICE_NAME_LEN 32 #define ETH_RX_ADAPTER_MEM_NAME_LEN 32 #define RSS_KEY_SIZE 40 /* value written to intr thread pipe to signal thread exit */ #define ETH_BRIDGE_INTR_THREAD_EXIT 1 /* Sentinel value to detect initialized file handle */ #define INIT_FD -1 BR/Jaeeun From: Jaeeun Ham Sent: Wednesday, June 22, 2022 11:14 AM To: Jayatheerthan, Jay >; dev@dpdk.org Cc: Jerin Jacob > Subject: RE: ask for TXA_FLUSH_THRESHOLD change Hi, Could you guide me on how to eliminate or reduce tx drop/retry? TXA_FLUSH_THRESHOLD helped somewhat but it was not cleared packet tx drop. =3D=3D=3D=3D=3D=3D=3D=3D[ TX adapter stats ]=3D=3D=3D=3D=3D=3D=3D=3D tx_retry: 17499893 tx_packets: 7501716 tx_dropped: 5132458 BR/Jaeeun From: Jayatheerthan, Jay > Sent: Thursday, June 16, 2022 3:40 PM To: Jaeeun Ham >; d= ev@dpdk.org Cc: Jerin Jacob > Subject: RE: ask for TXA_FLUSH_THRESHOLD change Hi Jaeeun, See my responses inline below. -Jay From: Jaeeun Ham > Sent: Monday, June 13, 2022 5:51 AM To: dev@dpdk.org Cc: Jerin Jacob >; Jayatheert= han, Jay > Subject: ask for TXA_FLUSH_THRESHOLD change Hi, There were latency delay when I increase dpdk(20.11.1) core. (one worker co= re was okay.) When I decrease the TXA_FLUSH_THRESHOLD value(1024 to 32), it was okay It's TXA_FLUSH_THRESHOLD at lib/librte_eventdev/rte_event_eth_tx_adapter.c.= // https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth= _tx_adapter.c?h=3D20.11#n15 When TXA_FLUSH_THRESHOLD value was changed from 1024 to 32, the latency tes= t result was fine on 10 cores for low traffic(DL:20Mbps/UL:17kbps). I think that this can make call for rte_eth_tx_buffer_flush() more frequent= ly. But, I'm not sure whether this approach can cause worse performance or not. Do you have any opinion about this? [Jay] Yes, it will cause rte_eth_tx_buffer_flush() to be called more often.= It can lead to lesser batching benefit. Typical performance vs. latency tr= ade-off decision apply here. Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is patched on DUSG3 from 1024 to = smaller value since DPDK 18.11.2: I'm not aware of any side-effect, I think it is needed to have low enough l= atency even at low traffic rates. For more details see Intel FP 22288. [Jay] Currently, TXA_FLUSH_THRESHOLD is not a configurable attr. TXA_MAX_NB_TX(128) looks the same as CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZ= E(16384), then is it also should be tuned? [Jay] They both are different attributes. TXA_MAX_NB_TX refers to max numbe= r of queues in Tx adapter. CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE refers t= o event buffer size in Rx BD. --- dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base.orig 2020-01= -29 15:05:10.000000000 +0100 +++ dpdk-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base 2020-01-29 = 15:11:10.000000000 +0100 @@ -566,9 +566,9 @@ CONFIG_RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=3D100 CONFIG_RTE_MAX_BRIDGE_ETH_INSTANCE=3D4 CONFIG_RTE_BRIDGE_ETH_INTR_RING_SIZE=3D32 -CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D128 +CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D16384 CONFIG_RTE_LIBRTE_BRIDGE_ETH_DEBUG=3Dn -CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D1024 +CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D32 --- dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 20= 21-08-05 23:46:52.051051000 +0200 +++ dpdk-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h 20= 21-08-06 00:50:07.310766255 +0200 @@ -175,8 +175,8 @@ #define RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100 #define RTE_MAX_BRIDGE_ETH_INSTANCE 4 #define RTE_BRIDGE_ETH_INTR_RING_SIZE 32 -#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128 -#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024 +#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384 +#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 10 #undef RTE_BRIDGE_ETH_TX_MULTI_PKT_EVENT BR/Jaeeun --_000_DM6PR11MB4348F53414D1F3846494157CFDB99DM6PR11MB4348namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

From DPDK 21.11, the Rx adapter event buffer size is configurable t= hrough API during creation time. Please refer to rte_event_eth_rx_adapter_c= reate_with_params() API in the below link:

https://doc.dpdk.org/api= /rte__event__eth__rx__adapter_8h.html

 

You could refer to EAL = docs on how to use log-level option based on the PMD you are using.

 

-Jay

 

 

 

From: Jaeeun Ham <jaeeun.ham@ericsson.com>
Sent: Saturday, June 25, 2022 8:49 AM
To: Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; dev@dpdk= .org
Cc: Jerin Jacob <jerinj@marvell.com>
Subject: RE: ask for TXA_FLUSH_THRESHOLD change

 =

Hi,

 

Packets seem to be silently discarded in the R= X NIC while waiting in the buffer, and it does not happen when I use only o= ne worker core.

I think multiple worker cores packet processin= g would have this problem to handle heavy traffic in parallel.

 

I want to increase buffer size to solve this a= s below, then is it right to change only modify the number from 4 to 8?

I am trying to gather DPDK and PMD driver log = such as "--log-level=3D"7" --log-level=3D"pmd.net,7&quo= t;. Could you recommend how to set the log-level string to confirm silent p= acket drop?

 

dpdk-stable-20.11.1/lib/librte_eventdev/rte_event_eth_rx_adapt= er.c

 

#define BATCH_SIZE      32=

#define BLOCK_CNT_THRESHOLD 10

#define ETH_EVENT_BUFFER_SIZE   (4*BATCH_SIZE)  // e.g. (8*BATCH_SIZE)

 

#define ETH_RX_ADAPTER_SERVICE_NAME_LEN 32

#define ETH_RX_ADAPTER_MEM_NAME_LEN 32

 

#define RSS_KEY_SIZE    40

/* value written to intr thread pipe to signal thread exit */<= o:p>

#define ETH_BRIDGE_INTR_THREAD_EXIT 1

/* Sentinel value to detect initialized file handle */

#define INIT_FD     -1

 

BR/Jaeeun

 

From: Jaeeu= n Ham
Sent: Wednesday, June 22, 2022 11:14 AM
To: Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; dev@dpdk.org
Cc: Jerin Jacob <jerinj@mar= vell.com>
Subject: RE: ask for TXA_FLUSH_THRESHOLD change

 

Hi,

 

Could you guide me on how to eliminate or redu= ce tx drop/retry?

TXA_FLUSH_THRESHOLD helped somewhat but it was= not cleared packet tx drop.

 

=3D=3D=3D=3D=3D=3D=3D=3D[ TX adapter stats ]=3D=3D=3D=3D=3D=3D= =3D=3D

tx_retry: 17499893

tx_packets: 7501716

tx_dropped: 5132458

 

BR/Jaeeun

 

From: Jayat= heerthan, Jay <jay.jayath= eerthan@intel.com>
Sent: Thursday, June 16, 2022 3:40 PM
To: Jaeeun Ham <jaeeun= .ham@ericsson.com>; dev@dpdk.org
Cc: Jerin Jacob <jerinj@mar= vell.com>
Subject: RE: ask for TXA_FLUSH_THRESHOLD change

 

Hi Jaeeun,

See my responses inline below.

 

-Jay

 

 

From: Jaeeu= n Ham <jaeeun.ham@ericsson.co= m>
Sent: Monday, June 13, 2022 5:51 AM
To: dev@dpdk.org
Cc: Jerin Jacob <jerinj@mar= vell.com>; Jayatheerthan, Jay <jay.jayatheerthan@intel.com>
Subject: ask for TXA_FLUSH_THRESHOLD change

 

Hi,

 

There were latency delay when I increase dpdk(= 20.11.1) core. (one worker core was okay.)

When I decrease the TXA_FLUSH_THRESHOLD value(= 1024 to 32), it was okay

 

It’s TXA_FLUSH_THRESHOLD at lib/librte_e= ventdev/rte_event_eth_tx_adapter.c. // https://git.dpdk.org/dpdk-stable/tree/lib/librte_eventdev/rte_event_eth_tx_= adapter.c?h=3D20.11#n15

When TXA_FLUSH_THRESHOLD value was changed fro= m 1024 to 32, the latency test result was fine on 10 cores for = low traffic(DL:20Mbps/UL:17kbps).

I think that this can make call for rte_eth_tx= _buffer_flush() more frequently.

But, I’m not sure whether this approa= ch can cause worse performance or not.

Do you have any opinion about this?=

 

[Jay] Yes, it will cause rt= e_eth_tx_buffer_flush() to be called more often. It can lead to lesser batc= hing benefit. Typical performance vs. latency trade-off decision apply here.

 

 

Similar RDK RTE_BRIDGE_ETH_TX_FLUSH_THOLD is p= atched on DUSG3 from 1024 to smaller value since DPDK 18.11.2:

I’m not aware of any side-effect, I thin= k it is needed to have low enough latency even at low traffic rates.  = For more details see Intel FP= 22288.

 

[Jay] Currently, TXA_FLUSH_THRESHOLD is not a c= onfigurable attr.

 

TXA_MAX_NB_TX(128) looks the same as CO= NFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE(16384), then is it also should be tuned?

 

[Jay] They both are differe= nt attributes. TXA_MAX_NB_TX refers to max number of queues in Tx adapter. CO= NFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE refers to event buffer size in Rx BD.=

 

 

--- dpd= k-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base.orig  2020-01-2= 9 15:05:10.000000000 +0100

+++ dpd= k-3pp-swu-18.11/dpdk-stable-18.11.2/config/common_base   2020-01-= 29 15:11:10.000000000 +0100

@@ -566= ,9 +566,9 @@

CONFIG_= RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES=3D100

CONFIG_= RTE_MAX_BRIDGE_ETH_INSTANCE=3D4

CONFIG_= RTE_BRIDGE_ETH_INTR_RING_SIZE=3D32

-CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D128

+CONFIG_RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE=3D16384

CONFIG_= RTE_LIBRTE_BRIDGE_ETH_DEBUG=3Dn

-CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D1024

+CONFIG_RTE_BRIDGE_ETH_TX_FLUSH_THOLD=3D32<= /i>

&n= bsp;

--- dpd= k-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h  = 2021-08-05 23:46:52.051051000 +0200

+++ dpd= k-3pp-swu-dusg3-20.11.3/dpdk-stable-20.11.3/config/rte_config.h  = 2021-08-06 00:50:07.310766255 +0200

@@ -175= ,8 +175,8 @@

#define= RTE_LIBRTE_BRIDGE_ETH_MAX_CP_ENQ_RETRIES 100

#define= RTE_MAX_BRIDGE_ETH_INSTANCE 4

#define= RTE_BRIDGE_ETH_INTR_RING_SIZE 32

-#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 128

-#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 1024

+#define RTE_BRIDGE_ETH_EVENT_BUFFER_SIZE 16384

+#define RTE_BRIDGE_ETH_TX_FLUSH_THOLD 10

#undef = RTE_BRIDGE_ETH_TX_MULTI_PKT_EVENT

 

 

BR/Jaeeun

 <= /o:p>

--_000_DM6PR11MB4348F53414D1F3846494157CFDB99DM6PR11MB4348namp_--