From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by dpdk.org (Postfix) with ESMTP id DC9201B8CA for ; Fri, 6 Jul 2018 05:36:52 +0200 (CEST) Received: by mail-oi0-f46.google.com with SMTP id n84-v6so20889660oib.9 for ; Thu, 05 Jul 2018 20:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qXQ1+4vhOZAyu3IFRWiHeYX7ojlaGOJ2PTMwILt7UfI=; b=V6kYCEH39Rnaq5q+MyPhXbxPA5xorT/mOwpIqPd2t5OFL6R+LykAVfZZ+GnklP4qzP sJCjOl4/KWTvqufhRWBsbpol1ZtdooKzjTr6He1+T7EJWGwrzX4tZxVS7aMmYlOltV9Y j+OLvQahikruZu9Ph39DiaQrQCbJtyMvNndPN1uhm/WRCLClnwCQ2SLCQ1MYV2Eb1WMz /z64CKqM4VpEuNo+MuBbZAokjYzwpkWp/Sj5F10lpFCL722hFM+uGOV1QbA3DYHXGs6l DAc1UFazVaZxrBDtYXweSLfJDM9//LFbmtatGi6rnrCE+gFlkyckZF0fxBnQ9ED03yLb O+2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qXQ1+4vhOZAyu3IFRWiHeYX7ojlaGOJ2PTMwILt7UfI=; b=R99yExVyTjKDZBRubKkWTswrocvBnkUOP/B9LkgmfEsqe1lXA6SpnGd5QUfOtLMwMl P5RkOfhU/0yRhlfsLljPnyJEHPAQ/0xZ6410VPgNWplBLCyfnGVpEf/ZvJN+b/CGuGKv orIVaC7NTBu68XGRCK7UuY3sWBGmkus9p0MdVU3z1/Bg8lSVEgkHEdzDMl7GeTOKRc49 bAQSslvoQCMgY9b9YMhyZm41kfrmugIXxpWAPSBmz5qF67IAQutaEtRaIeb/INE5yCeJ n6dzaZskYTt9O5SsNHwQjy4vjrasp7NdcnV24pyclT23y+Q/0jQLVnLnXZ2+nEYtyLRv sbUw== X-Gm-Message-State: APt69E02805w30//nGc2gLvxAZ5Pxjt11xGpGimKT8ZLJ4Jqfxe33f4T RnEILp4HWkZjzDErtnR+o1qWIkLpbfxCZu2RUTo= X-Google-Smtp-Source: AAOMgpfbuUmybjMO7fzeqPKFR5MGeGbtCnfSjRcZ+Yy+ALbk/9DoNf3x5AbX1vnlKEhzskCx5WEA0ziKJ6gIlLMe+FY= X-Received: by 2002:aca:b287:: with SMTP id b129-v6mr9138612oif.148.1530848212178; Thu, 05 Jul 2018 20:36:52 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac9:7047:0:0:0:0:0 with HTTP; Thu, 5 Jul 2018 20:36:51 -0700 (PDT) In-Reply-To: References: From: Hui Liu Date: Thu, 5 Jul 2018 20:36:51 -0700 Message-ID: To: Amarnath Nallapothula Cc: "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] occasionally traffic stalls due to rx and tx descriptor not available X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 03:36:53 -0000 Hi Amar, I'm a DPDK newbie and I saw a similar problem recently on one 82599 port. My app is doing a job like this: 1. TX thread calls rte_pktmbuf_alloc() to allocate buffers from mbuf_pool and fills it as ICMP packet and sends out, with speed of around 400,000 packets/sec, 1.6Gbps; 2. RX thread receives ICMP responses and worker threads work with the responses. This app was running fine for some time, typically from 8 hours to 5 days randomly, then it goes into a bad state, that TX thread could not send packets out any more via rte_eth_tx_buffer() or rte_eth_tx_buffer_flush() while rte_eth_tx_buffer_count_callback() is called for all packets flush. I'm highly suspecting the problem with descriptor exhausted but not get it clear yet.. In my app, I set max pkt burst as 256, rx descriptor as 2048, tx descriptor as 4096 with single rx/tx queue for one port to get good performance, not sure if they are the best combination. Just FYI. For descriptor problem, I'm still investigating on what kind of behavior/condition takes descriptors and never release it, just as your Query 2. If applicable, would you please let me know if there is a way to get the number of available tx/rx descriptor of ports and I could see when descriptors are really taken without being released time by time? Due to my system environment limit, I'm not able to directly attach gdb to debug... While I'm investigating this problem, would you please update me when you have any clue on your issue and I might get some inspiration from you? Thank you very much! Regards, Hui On Thu, Jul 5, 2018 at 4:34 AM, Amarnath Nallapothula < Amarnath.Nallapothula@riverbed.com> wrote: > Hi Experts, > > I am testing performance of my dpdk based application which forwards > packets from port 1 to port 2 of 40G NIC card and via versa.Occasionally = we > see that packets rx and tx stops on one of the port. I looked through the > dpdk=E2=80=99s fm10k driver=E2=80=99s code and found out that this could = happen if rx/tx > descriptors are not available. > > To improve performance, I am using RSS functionality and created five rx > and tx queue. Dedicated lcores are assigned to forward packets from port1 > queue 0 to port2 queue 0 and via versa. > > During port initialization rx_queue is initialized with 128 Rx ring > descriptor size and tx_queue is initialized 512 Tx ring descriptor. > Threshold values are left default. > > I have few queries here: > > 1. Is above initialization value for rx and tx descriptor is good for > each queue for given port. > 2. Under what conditions rx and tx descriptor gets exhausted? > 3. Any suggestion or information you can provide to debug this issue? > > Regards, > Amar >