From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 53124A0567; Tue, 9 Mar 2021 11:22:45 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D936322A493; Tue, 9 Mar 2021 11:22:44 +0100 (CET) Received: from inbox.dpdk.org (inbox.dpdk.org [95.142.172.178]) by mails.dpdk.org (Postfix) with ESMTP id 4B9E622A443 for ; Tue, 9 Mar 2021 11:22:44 +0100 (CET) Received: by inbox.dpdk.org (Postfix, from userid 33) id 27A6CA056A; Tue, 9 Mar 2021 11:22:44 +0100 (CET) From: bugzilla@dpdk.org To: dev@dpdk.org Date: Tue, 09 Mar 2021 10:22:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: other X-Bugzilla-Version: 20.05 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: huangying-c@360.cn X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 Subject: [dpdk-dev] =?utf-8?q?=5BBug_650=5D_af=5Fxdp=EF=BC=9Athe_packets_?= =?utf-8?q?in_complete=5Fqueue_can=27t_match_the_tx=5Fqueue?= X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" https://bugs.dpdk.org/show_bug.cgi?id=3D650 Bug ID: 650 Summary: af_xdp=EF=BC=9Athe packets in complete_queue can't matc= h the tx_queue Product: DPDK Version: 20.05 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: normal Priority: Normal Component: other Assignee: dev@dpdk.org Reporter: huangying-c@360.cn Target Milestone: --- kernel:5.9.1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D NIC: driver: ixgbe version: 5.9.1 firmware-version: 0x800006c5, 15.0.27 expansion-rom-version:=20 bus-info: 0000:03:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes ethtool -l eth2 Channel parameters for eth2: Pre-set maximums: RX: 0 TX: 0 Other: 1 Combined: 63 Current hardware settings: RX: 0 TX: 0 Other: 1 Combined: 1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D appplication args: EAL =3D --syslog local1 --log-level=3Dpmd.net.af_xdp:8 --vdev=3Dnet_af_xdp,iface=3Deth2,start_queue=3D0,queue_count=3D1 -l0-1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D I wrote a DNS server using AF_XDP Drivre, and when I stress tested it, I fo= und that the number of packets that have been sent has been consistently lower = (in complete_queue) than the number that have been put into tx_queue, and this = has always been the case, resulting in poor performance during stress tests. Then, I added logging to the code. I'm so sorry, my English is not good, so the Chinese description is attached =E8=8B=B1=E8=AF=AD=E4=B8=8D=E5=A5=BD=EF=BC=8C=E6=89=80=E4=BB=A5=E9=99=84=E4= =B8=8A=E4=B8=AD=E6=96=87=E6=8F=8F=E8=BF=B0 =E6=88=91=E7=94=A8af_xdp drivre=E5=86=99=E4=BA=86=E4=B8=AAdns=E7=9A=84=E6=9C=8D=E5=8A=A1=EF=BC=8C=E5= =BD=93=E6=88=91=E5=8E=8B=E5=8A=9B=E6=B5=8B=E8=AF=95=E7=9A=84=E6=97=B6=E5=80= =99=E5=8F=91=E7=8E=B0=E5=8F=91=E9=80=81=E5=AE=8C=E6=88=90=E7=9A=84=E6=95=B0= =E6=8D=AE=E5=8C=85=E7=9A=84=E6=95=B0=E9=87=8F=E4=B8=80=E7=9B=B4=E6=AF=94=E6= =94=BE=E8=BF=9Btx_queue=E7=9A=84=E5=8C=85=E7=9A=84=E6=95=B0=E9=87=8F=E8=A6= =81=E5=B0=91=EF=BC=8C=E8=80=8C=E4=B8=94=E4=B8=80=E7=9B=B4=E6=98=AF=E8=BF=99= =E6=A0=B7=EF=BC=8C=E5=A6=82=E6=AD=A4=E4=B8=80=E6=9D=A5=E5=9C=A8=E5=8E=8B=E5= =8A=9B=E6=B5=8B=E8=AF=95=E7=9A=84=E6=97=B6=E5=80=99=EF=BC=8C=E6=80=A7=E8=83= =BD=E5=BE=88=E4=BD=8E=E3=80=82 log is following : af_xdp_tx_zc(): want to send nb_pkts 8 /*here I want to send 8 packets to t= he af_xdp_tx_zc */ pull_umem_cq(): send complete n 3 packets=20 af_xdp_tx_zc(): submit to tx_ring count 8 packets /*here put the 8 packets = into tx_ring*/ af_xdp_rx_zc(): rcvd 5 packets reserve_fill_queue_zc(): reserve size 5 for fill queue af_xdp_tx_zc(): want to send nb_pkts 5 pull_umem_cq(): send complete n 5 packets //here the send completely packet= s is 5 and not 8 the added log in rte_eth_af_xdp.c:af_xdp_tx_zc static uint16_t af_xdp_tx_zc(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { struct pkt_tx_queue *txq =3D queue; struct xsk_umem_info *umem =3D txq->umem; struct rte_mbuf *mbuf; unsigned long tx_bytes =3D 0; int i; uint32_t idx_tx; uint16_t count =3D 0; struct xdp_desc *desc; uint64_t addr, offset; AF_XDP_LOG(DEBUG, "want to send nb_pkts %u\n", nb_pkts); //the log is a= dded by me pull_umem_cq(umem, nb_pkts); // here, check last tx sended packets for (i =3D 0; i < nb_pkts; i++) { mbuf =3D bufs[i]; if (mbuf->pool =3D=3D umem->mb_pool) { if (!xsk_ring_prod__reserve(&txq->tx, 1, &idx_tx)) { AF_XDP_LOG(DEBUG, "=3D=3D=3D=3D=3D=3D=3Dnb_pkts %u, count %= u\n", nb_pkts, count); kick_tx(txq);=20 goto out; } desc =3D xsk_ring_prod__tx_desc(&txq->tx, idx_tx); desc->len =3D mbuf->pkt_len; addr =3D (uint64_t)mbuf - (uint64_t)umem->buffer - umem->mb_pool->header_size; offset =3D rte_pktmbuf_mtod(mbuf, uint64_t) - (uint64_t)mbuf + umem->mb_pool->header_size; offset =3D offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT; desc->addr =3D addr | offset; count++; } else { struct rte_mbuf *local_mbuf =3D rte_pktmbuf_alloc(umem->mb_pool); void *pkt; if (local_mbuf =3D=3D NULL) goto out; if (!xsk_ring_prod__reserve(&txq->tx, 1, &idx_tx)) { rte_pktmbuf_free(local_mbuf); kick_tx(txq); goto out; } desc =3D xsk_ring_prod__tx_desc(&txq->tx, idx_tx); desc->len =3D mbuf->pkt_len; addr =3D (uint64_t)local_mbuf - (uint64_t)umem->buffer - umem->mb_pool->header_size; offset =3D rte_pktmbuf_mtod(local_mbuf, uint64_t) - (uint64_t)local_mbuf + umem->mb_pool->header_size; pkt =3D xsk_umem__get_data(umem->buffer, addr + offset); offset =3D offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT; desc->addr =3D addr | offset; rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), desc->len); rte_pktmbuf_free(mbuf); count++; } tx_bytes +=3D mbuf->pkt_len; } kick_tx(txq); AF_XDP_LOG(DEBUG, "submit to tx_ring count %u packets\n", count);//the = log is added by me out: xsk_ring_prod__submit(&txq->tx, count); txq->stats.tx_pkts +=3D count; txq->stats.tx_bytes +=3D tx_bytes; txq->stats.tx_dropped +=3D nb_pkts - count; return count; } the added log in rte_eth_af_xdp.c:af_xdp_tx_zc static void pull_umem_cq(struct xsk_umem_info *umem, int size) { struct xsk_ring_cons *cq =3D &umem->cq; size_t i, n; uint32_t idx_cq =3D 0; n =3D xsk_ring_cons__peek(cq, size, &idx_cq); for (i =3D 0; i < n; i++) { uint64_t addr; addr =3D *xsk_ring_cons__comp_addr(cq, idx_cq++); #if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG) addr =3D xsk_umem__extract_addr(addr); rte_pktmbuf_free((struct rte_mbuf *) xsk_umem__get_data(umem->buffer, addr + umem->mb_pool->header_size)); #else rte_ring_enqueue(umem->buf_ring, (void *)addr); #endif } AF_XDP_LOG(DEBUG, "send complete n %lu packets\n", n);//here the log xsk_ring_cons__release(cq, n); } --=20 You are receiving this mail because: You are the assignee for the bug.=