From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E9DA2A052A; Tue, 22 Dec 2020 21:58:25 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 86C60CA67; Tue, 22 Dec 2020 21:57:17 +0100 (CET) Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by dpdk.org (Postfix) with ESMTP id BDACACA60 for ; Tue, 22 Dec 2020 10:46:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1608630417; x=1640166417; h=from:to:cc:date:message-id:references:in-reply-to: mime-version:subject; bh=J/lZxlwQTRtaaT+Q/xRiAHqmFEzeurpqUONLASYkVR8=; b=ezjXNlSosbdL5QP0O+fIbMswwlx3whpRhtP+rB0GW+tiPmjRDvLFHEtH f8GR58hdJYVd2oeSUbY1CvwiivjgVSw9aw4m23HJQamoDz4vabTs+xMCj sVK5O89TcTBvWSotNC/saux/qM3VHyjV7Sb6j2FX1Jl0pV48rERuPuuz4 Y=; X-Amazon-filename: smime.p7s X-IronPort-AV: E=Sophos;i="5.78,438,1599523200"; d="p7s'?scan'208,217";a="70682823" Thread-Topic: net/ena: traffic lock Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-76e0922c.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 22 Dec 2020 09:46:46 +0000 Received: from EX13D08EUC002.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-76e0922c.us-west-2.amazon.com (Postfix) with ESMTPS id A8EE3A8581; Tue, 22 Dec 2020 09:46:44 +0000 (UTC) Received: from EX13D12EUA003.ant.amazon.com (10.43.165.147) by EX13D08EUC002.ant.amazon.com (10.43.164.124) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 22 Dec 2020 09:46:43 +0000 Received: from EX13D12EUA003.ant.amazon.com ([10.43.165.147]) by EX13D12EUA003.ant.amazon.com ([10.43.165.147]) with mapi id 15.00.1497.010; Tue, 22 Dec 2020 09:46:43 +0000 From: "Chauskin, Igor" To: RajeshKumar Kalidass , "mk@semihalf.com" , "Tzalik, Guy" , "ar@semihalf.com" , "dev@dpdk.org" CC: Tanmay Kishore , Rakesh Jagota Thread-Index: AdbYN4nnzupBjlHlR2uhatDo67EkcwAAiFNg Date: Tue, 22 Dec 2020 09:46:43 +0000 Message-ID: <8dfef9bf31d74aac81eacd4dbee9a475@EX13D12EUA003.ant.amazon.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.43.165.145] MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 22 Dec 2020 21:57:09 +0100 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] net/ena: traffic lock X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Rajesh, For the tx queue getting into unrecoverable state - can you please share the details of your usecase? (instance types, traffic type, number of queues in use, etc.). Also, please share your instances ids, if possible. Regarding prepare_ctx_err counter - under certain circumstances this counter indeed can be incorrectly incremented and we're working on a fix to this. However, this shouldn't have effect beyond statistics (unless your application explicitly relies on this counter in its business logic). Thanks, Igor From: RajeshKumar Kalidass Sent: Tuesday, December 22, 2020 10:03 To: mk@semihalf.com; Chauskin, Igor ; Tzalik, Guy ; ar@semihalf.com; dev@dpdk.org Cc: Tanmay Kishore ; Rakesh Jagota Subject: [EXTERNAL] net/ena: traffic lock CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Dpdk: 19.11 Driver: ena During longevity(after 24+ hrs) testing at 10Gbps, one of tx-queue is getting into unrecoverable state ( its not able to find enough tx-descriptor nor its freeing up). So for every tx-burst, eth_ena_xmit_pkts() neither finds free tx-descriptor nor able to free txd (ena_com_tx_comp_req_id_get() is always returning ENA_COM_TRY_AGAIN). We see eth_ena_xmit_pkts() has been refactored in latest LTS version, is there any related issue got fixed ? Can you help (gdb) p *(struct ena_ring *) rte_eth_devices[2].data->tx_queues[5] $14 = { next_to_use = 4979, next_to_clean = 3958, type = ENA_RING_TYPE_TX, tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_DEV, { empty_tx_reqs = 0x11e406b00, empty_rx_reqs = 0x11e406b00 }, { tx_buffer_info = 0x11d2dfc80, rx_buffer_info = 0x11d2dfc80 }, rx_refill_buffer = 0x0, ring_size = 1024, ena_com_io_cq = 0x11e40e640, ena_com_io_sq = 0x11e4168c0, ena_bufs = {{ len = 0, req_id = 0 } }, mb_pool = 0x0, port_id = 2, id = 5, tx_max_header_size = 96 '`', configured = 1, push_buf_intermediate_buf = 0x11e406a00 "", adapter = 0x11e40e040, offloads = 2, sgl_size = 17, { rx_stats = { cnt = 4979, bytes = 417580, refill_partial = 35426, bad_csum = 0, mbuf_alloc_fail = 0, bad_desc_num = 38603, ---Type to continue, or q to quit--- bad_req_id = 3178 }, tx_stats = { cnt = 4979, bytes = 417580, prepare_ctx_err = 35426, <-- Errors linearize = 0, linearize_failed = 0, tx_poll = 38603, doorbells = 3178, bad_req_id = 0, available_desc = 2 } }, numa_socket_id = 0 } Thanks, -Rajesh This message may contain confidential and privileged information. If it has been sent to you in error, please reply to advise the sender of the error and then immediately delete it. If you are not the intended recipient, do not read, copy, disclose or otherwise use this message. The sender disclaims any liability for such unauthorized use. NOTE that all incoming emails sent to Gigamon email accounts will be archived and may be scanned by us and/or by external service providers to detect and prevent threats to our systems, investigate illegal or inappropriate behavior, and/or eliminate unsolicited promotional emails ("spam").