From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 334F642BCF for ; Mon, 5 Jun 2023 12:29:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E945740DF6; Mon, 5 Jun 2023 12:29:34 +0200 (CEST) Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by mails.dpdk.org (Postfix) with ESMTP id 1E14F4003C for ; Mon, 5 Jun 2023 12:29:33 +0200 (CEST) Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-51669dd574aso51276a12.0 for ; Mon, 05 Jun 2023 03:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685960973; x=1688552973; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=DDEwDyZ595REiqpVPJOKTpvAH/u2DB8ZT0JnzLk5ajw=; b=jFwDWBMc397ATZdSLADA/muN8uwjmkeyMmSH7+qHZQP+w7LVdzBrCzbPAFew+9tcSI 4y1UAPyxD2jz+lLQffkOlE+edeOZj5H8tXpXapd2NJ3Y4MoECShP2cJ2NCPH1XG9uAn5 MdflZAsYUp62P7XxgjgowIOXQNv7HKCDGnwwux53GBxdq0vtdybftH6/qV766RTazKux /Dup5na4vCgqzt4GIno+74zbuzEcSoEGoMWkyuiASLjpXZeyfg16CTmAB8Lh/YgTWjfK nGxXyrQB10jIWC3RxvmBKVgm9IuKO+fJDT+HV1MgENPDvPBpCo1OC2QzRh0xohSR6qmg KLSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685960973; x=1688552973; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DDEwDyZ595REiqpVPJOKTpvAH/u2DB8ZT0JnzLk5ajw=; b=L8asETuMO214TeVRtwib8CCKxg7fvxahM/GiimSk+HVyezxL8eDYI4QKYFEP/o1Qy4 ALS57wfYX6ddw77oVn1ft3bzPShZ7JeDPOgR6G3T1jlG+mOVfsmcIaGcRU0UAxWIMPqT mOZuK7h0N1ony+zjo2bwAp5r40IhbsP1kfNOhrPYOu8wsScEuOa1AkCAv+GXwXEI38tI V3iJutKOT43ulbhsrk/cx1fizsNd6llI30i27bBE8/GyUoaMeJawHYFcHr8jy0cWOY4/ quz/Do08nsGKoHLrVp4uV4BeRfuc8Fv8lD8nEWuijgK1JNxml01SeLRiH9g9u2sO15XS /mpQ== X-Gm-Message-State: AC+VfDxPU1v+FNdmuTA+CFGmXLvUsrFWsT7m60lU/KGdH+TlHWa+8S+W gsFWbiV9IMMoW9CmtMmAjhwaGuCtDXpGak4fHWY= X-Google-Smtp-Source: ACHHUZ482B+QvUn+xUqE8Fv7KJ/ojipRLrLchho1838/pFl3Vgm4zA6kLUq1iJNLc8Mzfm4wC9lVrTferPdNacCLCUc= X-Received: by 2002:a05:6402:5203:b0:502:1f7b:f069 with SMTP id s3-20020a056402520300b005021f7bf069mr14032191edd.2.1685960972503; Mon, 05 Jun 2023 03:29:32 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Erez Ferber Date: Mon, 5 Jun 2023 13:29:21 +0300 Message-ID: Subject: Re: mlx5: imissed / out_of_buffer counter always 0 To: Slava Ovsiienko Cc: =?UTF-8?Q?Daniel_=C3=96stman?= , "users@dpdk.org" , Matan Azrad , "maxime.coquelin@redhat.com" , "david.marchand@redhat.com" Content-Type: multipart/alternative; boundary="000000000000df0ff505fd5f6098" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000df0ff505fd5f6098 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Daniel, is the container running in shared or non-shared mode ? For shared mode, I assume the kernel sysfs counters which DPDK relies on for imissed/out_of_buffer are not exposed. Best regards, Erez On Fri, 2 Jun 2023 at 18:07, Slava Ovsiienko wrote= : > Hi, Daniel > > > > I would recommend to take the following action: > > - update the firmware, 16.33.xxxx looks to be outdated a little bit. > Please, try 16.35.1012 or later. > mlx5_glue->devx_obj_create might succeed with the newer FW. > > - try to specify dv_flow_en=3D0 devarg, it forces mlx5 PMD to use rdma_co= re > library for queue management > and kernel driver will be aware about Rx queues being created and attac= h > them to the kernel counter set > > > > With best regards, > Slava > > > > *From:* Daniel =C3=96stman > *Sent:* Friday, June 2, 2023 3:59 PM > *To:* users@dpdk.org > *Cc:* Matan Azrad ; Slava Ovsiienko < > viacheslavo@nvidia.com>; maxime.coquelin@redhat.com; > david.marchand@redhat.com > *Subject:* mlx5: imissed / out_of_buffer counter always 0 > > > > Hi, > > > > I=E2=80=99m deploying a containerized DPDK application in an OpenShift Ku= bernetes > environment using DPDK 21.11.3. > > The application uses a Mellanox ConnectX-5 100G NIC through VFs. > > > > The problem I have is that the ETH stats counter imissed (which seems to > be mapped to =E2=80=9Cout_of_buffer=E2=80=9D internally in mlx5 PMD drive= r) is 0 when I > don=E2=80=99t expect it to be, i.e. when the application doesn=E2=80=99t = read the packets > fast enough. > > > > Using GDB I can see that it tries to access the counter through > /sys/class/infiniband/mlx5_99/ports/1/hw_counters/out_of_buffer but the > hw_counters directory is missing so it will just return a zero value. I > don=E2=80=99t know why it is missing. > > When looking at mlx5_os_read_dev_stat() I can see that there is an > alternative way of reading the counter, through > mlx5_devx_cmd_queue_counter_query() but under the condition that > priv->q_counters are set. > > It doesn=E2=80=99t get set in my case because mlx5_glue->devx_obj_create(= ) fails > (errno 22) in mlx5_devx_cmd_queue_counter_alloc(). > > > > Have I missed something? > > > > NIC info: > > Mellanox Technologies MT27800 Family [ConnectX-5] - 100Gb 2-port QSFP28 > MCX516A-CCHT > driver: mlx5_core > version: 5.0-0 > firmware-version: 16.33.1048 (MT_0000000417) > > > > Please let me know if I need to provide more information. > > > > Best regards, > > Daniel > > > --000000000000df0ff505fd5f6098 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Daniel,

i= s the container running in shared or non-shared mode ?=C2=A0
For = shared mode, I assume the kernel sysfs counters which DPDK relies on for im= issed/out_of_buffer are not exposed.

Best regards,=
Erez

On Fri, 2 Jun 2023 at 18:07, Slava Ovsiienko <= ;viacheslavo@nvidia.com> w= rote:

Hi, Daniel

=C2=A0

I would recommend to take the following action:

- update the firmware, 16.33.xxxx looks to be outdat= ed a little bit. Please, try 16.35.1012 or later.
=C2=A0=C2=A0mlx5_glue->devx_obj_create might succeed with the newer FW.<= u>

- try to specify dv_flow_en=3D0 devarg, it forces ml= x5 PMD to use rdma_core library for queue management
=C2=A0and kernel driver will =C2=A0be aware about Rx queues being created a= nd attach them to the kernel counter set

=C2=A0

With best regards,
Slava

=C2=A0

From: Daniel =C3=96stman <daniel.ostman@ericsson.com<= /a>>
Sent: Friday, June 2, 2023 3:59 PM
To: users@dpdk.o= rg
Cc: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; maxime.coquelin@r= edhat.com; david.marchand@redhat.com
Subject: mlx5: imissed / out_of_buffer counter always 0

=C2=A0

Hi,

=C2=A0

I=E2=80=99m deploying a containerized DPDK application in an OpenShift Kubernetes environment using= DPDK 21.11.3.

The application uses a Mellanox ConnectX-5=C2=A0100G= NIC through VFs.

=C2=A0

The problem I have is that the ETH stats counter imi= ssed (which seems to be mapped to =E2=80=9Cout_of_buffer=E2=80=9D internall= y in mlx5 PMD driver) is 0 when I don=E2=80=99t expect it to be, i.e. when = the application doesn=E2=80=99t read the packets fast enough.=

=C2=A0

Using GDB I can see that it tries to access the coun= ter through /sys/class/infiniband/mlx5_99/ports/1/hw_counters/out_of_buffer= but the hw_counters directory is missing so it will just return a zero val= ue. I don=E2=80=99t know why it is missing.

When looking at mlx5_os_read_dev_stat() I can see th= at there is an alternative way of reading the counter, through mlx5_devx_cm= d_queue_counter_query() but under the condition that priv->q_counters ar= e set.

It doesn=E2=80=99t get set in my case because mlx5_g= lue->devx_obj_create() fails (errno 22) in mlx5_devx_cmd_queue_counter_a= lloc().

=C2=A0

Have I missed something?

=C2=A0

NIC info:

Mellanox Technologies MT27800 Family [ConnectX-5] - = 100Gb 2-port QSFP28 MCX516A-CCHT
driver: mlx5_core
version: 5.0-0
firmware-version: 16.33.1048 (MT_0000000417)

=C2=A0

Please let me know if I need to provide more informa= tion.

=C2=A0

Best regards,

Daniel

=C2=A0

--000000000000df0ff505fd5f6098--