From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87A8E45900 for ; Mon, 9 Sep 2024 15:41:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E60C4028A; Mon, 9 Sep 2024 15:41:56 +0200 (CEST) Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by mails.dpdk.org (Postfix) with ESMTP id C468F40299 for ; Mon, 9 Sep 2024 15:25:28 +0200 (CEST) Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2da4ea973bdso3125082a91.1 for ; Mon, 09 Sep 2024 06:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725888328; x=1726493128; darn=dpdk.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=JDlfP1VVnHPTIl16rMkLVUOZK61fZllhK7sfX9541ac=; b=Lv8ZgMYt4kGO6CyxKJbAaTMphrFmj+U7JvfnZGAoCrQ1g62uBxmvT9KcuerXy6Xc8W 66Lt5wMlNUHrxV4hPhj3XyY70A4SzrRkEvuD9ObYLJ4epQXGy2GzV4G6D1tUHywXLUFz 8xZQ/OkMHkM7hdyh4lhCjKqViSCrzpbIn+h51PQvK5bHJrlSpHUg/a0dsayU4hpG/Sn8 YZPWf/eBb1Jh7DYW37GgaPY5UdWVcT1BbxlzBIk9Bop0pONkKnMA9Kkvlzu7D17dzEp+ 70xPnOT2vm2+aeLWSEjAUcY7csuhIuf/nZ20WDjFrTO8nbIxNCFPzg86J93DtC+gt4V3 ydxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725888328; x=1726493128; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=JDlfP1VVnHPTIl16rMkLVUOZK61fZllhK7sfX9541ac=; b=s9EkJi2IVzSwqR5m2rHQ4naAGmOaPIVD3XQ0oVM7dB4MVW8O7BkvtBlS+NebBk/HNm iVbC9aHtMVqjzbBa4tPQjeAd+ue2HEyzgTYm46leQkF+2uO7ihl7uf7lcmTScCgQpLuP JG0iAjeD5xqZDj8YdyiTNvwUECT4cuuwKvPLs//oXaDnhrWqSJLmP4U6gSA6Q+D5Hzho Riwzj8U2OSGUzxCdW3dMlxj/uRpFPsaVviWUFpYYs/neJKQG+nnAgy1f29ihSMrRZsJY 9Mljtc/Fsg+VycyW9IsQTinwIh12/K9B490rdCa8zjO8rrT2uUnofaZcE+4gRMheRqbW 88pA== X-Gm-Message-State: AOJu0YyZtTU50FBO2huOtop/GopfnXOK0MvgT/gY0fub2HrkKScZCheD om+gF4Q9+0JQsWTkX0u0S/AiOJCgW8Wle0X/mSMz28e/keadUziozu11xYzC4Peqsj6oX+IocgG yXtRNsIVLLPNyXylvtPDSn82wd5kdva+b X-Google-Smtp-Source: AGHT+IEDx3SsA+OXMJ84SryB4ePNobGkK1ygExsK0nzQa1DirRdFUDTpcBedNa24CjzvC9HL9Bj60Asdm/IsHjDvWvo= X-Received: by 2002:a17:90b:234f:b0:2d8:5b9f:75ab with SMTP id 98e67ed59e1d1-2dafd09dfbbmr7467151a91.29.1725888327563; Mon, 09 Sep 2024 06:25:27 -0700 (PDT) MIME-Version: 1.0 From: Alan Arondel Date: Mon, 9 Sep 2024 15:25:23 +0200 Message-ID: Subject: rte_eth_tx_burst return 0 after some time To: users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Mon, 09 Sep 2024 15:41:54 +0200 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hello Everyone, I'm trying to add an export module to my application. I use DPDK version 20.11, a 82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb and the vfio-pci driver. port configuration is the following: port_conf->txmode.mq_mode = ETH_MQ_TX_NONE; port_conf->txmode.offloads = DEV_TX_OFFLOAD_MULTI_SEGS; I use only 1 queue on the card and the configuration is the following : nb_desc : 4096 rs_threshold : 32 free_threshold : 32 I have an issue where at some point during the lifetime of the application, the rte_eth_tx_burst return only 0 and nothing can be done once it is stuck after several seconds of the beginning of transmission. I have checked couple of things : - i still get burst of packets to send - i still have lot of mbuf in my mempool - i use t-rex to generate traffic to forward and one of them have lot of small packet and i can support up to 2,8Mpps (roughly 7,8Gbps) sadly with another traffic i can't go up more than 200Kpps at best (roughly 2Gbps if not less) once the application is stuck, I tried to stop the traffic and restart it with a very low bandwidth like 40Mbps but it changes nothing here is a code snippet of what i'm trying to do : nb_rx = rte_ring_sc_dequeue_burst(od_h->rx_rings[i], (void **) pkts_burst, OUTPUT_DPDK_BURST_SIZE, NULL); for (j = 0; j < nb_rx; j++) { if (!rte_pktmbuf_is_contiguous(pkts_burst[j])) { stats->jumbo_frame ++; /* i got some jumbo frame in both traffic which i want to track, mbuf size is 1512 + RTE_PKTMBUF_HEADROOM */ } ret = rte_eth_tx_burst(port_id, 0, &pkts_burst[j], 1); /* i know this is weird but i have another issue with the order of the packets when sending more than 1 packet even if the mbuf inside pkts_burst are ordered correctly */ if (unlikely(! ret)) { /* packet not sent, just drop it */ rte_pktmbuf_free(pkts_burst[j]); stats->freed_mbuf ++; int flushed = rte_eth_tx_done_cleanup(port_id, 0, 0); /* this is my last test since i saw we could flush the mbuf stuck in the cache of the driver */ if (flushed >= 0) { stats->mbuf_flushed += flushed; /* i can see some of them but not moving once the application do not send anymore packets */ } else { stats->mbuf_flushed = flushed; } } else { stats->nb_sent ++; sent = 1; } } i ++; } As far as I understand, the logic of the code seems to be ok since it works on another type of traffic. However i have no clue why it is stuck at some point. If you got some insight to dig further on this i'm all ears. Also, I would like to understand the free_threshold. As far as i understand, mbuf will "stay" in the driver into there is not enough descriptor available regarding the threshold hence the mbuf return to the mempool. Is that right ? Thank you for any help i will have on this. Regards, Alan.