From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 68774A0524 for ; Sun, 11 Apr 2021 03:31:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E42F94068A; Sun, 11 Apr 2021 03:31:26 +0200 (CEST) Received: from mx0a-00000d04.pphosted.com (mx0a-00000d04.pphosted.com [148.163.149.245]) by mails.dpdk.org (Postfix) with ESMTP id 4C8264014E for ; Sun, 11 Apr 2021 03:31:24 +0200 (CEST) Received: from pps.filterd (m0102887.ppops.net [127.0.0.1]) by mx0a-00000d04.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 13B1TY3C020377 for ; Sat, 10 Apr 2021 18:31:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stanford.edu; h=mime-version : references : in-reply-to : from : date : message-id : subject : to : content-type; s=pps05272020; bh=bHguX47dBOgs58d8GyfjnfA9QatThSCbvjc3zwpQQFg=; b=DX+87sZyFHdoj6QwvEDtfwdVCD4V/ugd8tus9HY+NCkrh4HOBjTP1nbF9NWvZIhb9pDi /tsr3X+m+G3naCOvuwUgQ0UJZkzVYeyn9EdXSZcrgxlbK9mda3VR5hr3qxHlzDwXAkWx GO2xofYb4t7IsgyrClP9WqLv2lSci6af8bXEQap136Qw9Bkexsk4F6HcPncPHxFHY+Wl hrH+h2Sep348GAhY1ybce6+D/DjqwSdNNSAi+Ww97FTXDK8CBlrV+57EC1+ssVVkXHW5 yNVgIrUhx2pU7jpMopsj1GCgL2pLrMCPd3tYL+weJ6tu9+t6X0I8vxb38/UClLDRnoQx bA== Received: from mx0b-00000d03.pphosted.com (mx0b-00000d03.pphosted.com [148.163.153.234]) by mx0a-00000d04.pphosted.com with ESMTP id 37u8nqsw8h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 10 Apr 2021 18:31:24 -0700 Received: from pps.filterd (m0214089.ppops.net [127.0.0.1]) by mx0a-00000d03.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 13B1RqTS002082 for ; Sat, 10 Apr 2021 18:31:23 -0700 Received: from mx0a-00000d06.pphosted.com (mx0a-00000d06.pphosted.com [148.163.135.119]) by mx0a-00000d03.pphosted.com with ESMTP id 37ub76hyyc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 10 Apr 2021 18:31:23 -0700 Received: from pps.filterd (m0167934.ppops.net [127.0.0.1]) by mx0a-00000d06.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 13B1VMdA000948 for ; Sat, 10 Apr 2021 18:31:22 -0700 Received: from smtp.stanford.edu (smtp6.stanford.edu [171.67.219.73]) by mx0a-00000d06.pphosted.com with ESMTP id 37u9e9rh63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 10 Apr 2021 18:31:22 -0700 Received: from mail-ot1-f53.google.com (mail-ot1-f53.google.com [209.85.210.53]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: gerryw) by smtp.stanford.edu (Postfix) with ESMTPSA id C71B880F6A for ; Sat, 10 Apr 2021 18:31:21 -0700 (PDT) Received: by mail-ot1-f53.google.com with SMTP id 101-20020a9d0d6e0000b02902816815ff62so3559396oti.9 for ; Sat, 10 Apr 2021 18:31:21 -0700 (PDT) X-Gm-Message-State: AOAM531AgZQVAdZd9XCXQF95iUSbvJb1LA09eTLSr+uDXbVWLXs0Htzd ClSe/ehoOMbhqbTYbaLtRNJwkFFWf2zqRIt+fA0= X-Google-Smtp-Source: ABdhPJwo8X5sVxxlq1rl9Dc9bXplup/mNlv2YLzgZTv36N8IpMEmhz4/M98AU4fPdiB10rMB4BIdW4qYPXM01cmRais= X-Received: by 2002:a9d:1c87:: with SMTP id l7mr17824461ota.285.1618104681125; Sat, 10 Apr 2021 18:31:21 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Gerry Wan Date: Sat, 10 Apr 2021 18:31:10 -0700 X-Gmail-Original-Message-ID: Message-ID: To: users@dpdk.org x-proofpoint-stanford-dir: outbound X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-10_13:2021-04-09, 2021-04-10 signatures=0 X-Proofpoint-ORIG-GUID: AkxyTn3zjhPQrcTUs1Hm4NNE_YdFQKD0 X-Proofpoint-GUID: AkxyTn3zjhPQrcTUs1Hm4NNE_YdFQKD0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-10_13:2021-04-09, 2021-04-10 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 impostorscore=0 adultscore=100 mlxlogscore=318 suspectscore=0 lowpriorityscore=0 clxscore=1015 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104110007 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-users] mlx5: packets lost between good+discard and phy counters X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" After further investigation, I think this may be a bug introduced in DPDK v20.11, where these "lost" packets should be counted as "rx_out_of_buffer" and "rx_missed_errors". On v20.08 both of these counters increment, but on v20.11 and v21.02 these counters always remain 0. Any workarounds for this? This is an important statistic for my use case. On Fri, Apr 2, 2021 at 5:03 PM Gerry Wan wrote: > I have a simple forwarding experiment using a mlx5 NIC directly connected > to a generator. I am noticing that at high enough throughput, > rx_good_packets + rx_phy_discard_packets may not equal rx_phy_packets. > Where are these packets being dropped? > > Below is an example xstats where I receive at almost the limit of what my > application can handle with no loss. It shows rx_phy_discard_packets is 0 > but the number actually received by the CPU is less than rx_phy_packets. > rx_out_of_buffer and other errors are also 0. > > I have disabled Ethernet flow control via rte_eth_dev_flow_ctrl_set with > mode = RTE_FC_NONE, if that matters. > > { > "rx_good_packets": 319992439, > "tx_good_packets": 0, > "rx_good_bytes": 19199546340, > "tx_good_bytes": 0, > "rx_missed_errors": 0, > "rx_errors": 0, > "tx_errors": 0, > "rx_mbuf_allocation_errors": 0, > "rx_q0_packets": 319992439, > "rx_q0_bytes": 19199546340, > "rx_q0_errors": 0, > "rx_wqe_errors": 0, > "rx_unicast_packets": 319999892, > "rx_unicast_bytes": 19199993520, > "tx_unicast_packets": 0, > "tx_unicast_bytes": 0, > "rx_multicast_packets": 0, > "rx_multicast_bytes": 0, > "tx_multicast_packets": 0, > "tx_multicast_bytes": 0, > "rx_broadcast_packets": 0, > "rx_broadcast_bytes": 0, > "tx_broadcast_packets": 0, > "tx_broadcast_bytes": 0, > "tx_phy_packets": 0, > "rx_phy_packets": 319999892, > "rx_phy_crc_errors": 0, > "tx_phy_bytes": 0, > "rx_phy_bytes": 20479993088, > "rx_phy_in_range_len_errors": 0, > "rx_phy_symbol_errors": 0, > "rx_phy_discard_packets": 0, > "tx_phy_discard_packets": 0, > "tx_phy_errors": 0, > "rx_out_of_buffer": 0, > "tx_pp_missed_interrupt_errors": 0, > "tx_pp_rearm_queue_errors": 0, > "tx_pp_clock_queue_errors": 0, > "tx_pp_timestamp_past_errors": 0, > "tx_pp_timestamp_future_errors": 0, > "tx_pp_jitter": 0, > "tx_pp_wander": 0, > "tx_pp_sync_lost": 0, > } > >