From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38055A04BA; Thu, 1 Oct 2020 18:24:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7F7541D449; Thu, 1 Oct 2020 18:24:41 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id DB1211D440 for ; Thu, 1 Oct 2020 18:24:39 +0200 (CEST) IronPort-SDR: cZhua41cUp87SA4+XOPm1784S9DmMSSCUv/eYHeohPVDgYbqZ+ReSfCgZ3e729WSJKR+8sHqd6 +cRs8Tqjq1lw== X-IronPort-AV: E=McAfee;i="6000,8403,9761"; a="226891379" X-IronPort-AV: E=Sophos;i="5.77,323,1596524400"; d="scan'208";a="226891379" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Oct 2020 09:24:20 -0700 IronPort-SDR: bvVjFOu11xpvmiUYB5Zhwy/bIVm2udk1jQbShlmpqgKTU571BysczH6Sopa7ZvZs4DPz7hBENQ 20+BvkdGe8nQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,323,1596524400"; d="scan'208";a="325469114" Received: from irsmsx602.ger.corp.intel.com ([163.33.146.8]) by orsmga002.jf.intel.com with ESMTP; 01 Oct 2020 09:24:19 -0700 Received: from irsmsx604.ger.corp.intel.com (163.33.146.137) by irsmsx602.ger.corp.intel.com (163.33.146.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 1 Oct 2020 17:24:18 +0100 Received: from irsmsx604.ger.corp.intel.com ([163.33.146.137]) by IRSMSX604.ger.corp.intel.com ([163.33.146.137]) with mapi id 15.01.1713.004; Thu, 1 Oct 2020 17:24:18 +0100 From: "Loftus, Ciara" To: Li RongQing CC: "dev@dpdk.org" Thread-Topic: [PATCH][v2] net/af_xdp: avoid to unnecessary allocation and free mbuf in rx path Thread-Index: AQHWkweR+JinpWhObkCY3I0GBwa/l6mC8bew Date: Thu, 1 Oct 2020 16:24:18 +0000 Message-ID: <0bf5839c183544559e33846c7b0bc053@intel.com> References: <1601016336-12233-1-git-send-email-lirongqing@baidu.com> In-Reply-To: <1601016336-12233-1-git-send-email-lirongqing@baidu.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 x-originating-ip: [163.33.253.164] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH][v2] net/af_xdp: avoid to unnecessary allocation and free mbuf in rx path X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > when receive packets, the max bunch number of mbuf are allocated > if hardware does not receive the max bunch number packets, it > will free redundancy mbuf, that is low-performance >=20 > so optimize rx performance, by allocating number of mbuf based on > result of xsk_ring_cons__peek, to avoid to redundancy allocation, > and free mbuf when receive packets Hi, Thanks for the patch and fixing the issue I raised. With my testing so far I haven't measured an improvement in performance wit= h the patch. Do you have data to share which shows the benefit of your patch? I agree the potential excess allocation of mbufs for the fill ring is not t= he most optimal, but if doing it does not significantly impact the performa= nce I would be in favour of keeping that approach versus touching the cache= d_cons outside of libbpf which is unconventional. If a benefit can be shown and we proceed with the approach, I would suggest= creating a new function for the cached consumer rollback eg. xsk_ring_cons= _cancel() or similar, and add a comment describing what it does. Thanks, Ciara >=20 > V2: rollback rx cached_cons if mbuf failed to be allocated >=20 > Signed-off-by: Li RongQing > Signed-off-by: Dongsheng Rong > --- > drivers/net/af_xdp/rte_eth_af_xdp.c | 67 ++++++++++++++++--------------- > ------ > 1 file changed, 29 insertions(+), 38 deletions(-) >=20 > diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c > b/drivers/net/af_xdp/rte_eth_af_xdp.c > index 01f462b46..e04fa43f6 100644 > --- a/drivers/net/af_xdp/rte_eth_af_xdp.c > +++ b/drivers/net/af_xdp/rte_eth_af_xdp.c > @@ -251,28 +251,29 @@ af_xdp_rx_zc(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > struct xsk_umem_info *umem =3D rxq->umem; > uint32_t idx_rx =3D 0; > unsigned long rx_bytes =3D 0; > - int rcvd, i; > + int i; > struct rte_mbuf *fq_bufs[ETH_AF_XDP_RX_BATCH_SIZE]; >=20 > - /* allocate bufs for fill queue replenishment after rx */ > - if (rte_pktmbuf_alloc_bulk(umem->mb_pool, fq_bufs, nb_pkts)) { > - AF_XDP_LOG(DEBUG, > - "Failed to get enough buffers for fq.\n"); > - return 0; > - } > + nb_pkts =3D xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); >=20 > - rcvd =3D xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); > - > - if (rcvd =3D=3D 0) { > + if (nb_pkts =3D=3D 0) { > #if defined(XDP_USE_NEED_WAKEUP) > if (xsk_ring_prod__needs_wakeup(fq)) > (void)poll(rxq->fds, 1, 1000); > #endif >=20 > - goto out; > + return 0; > + } > + > + /* allocate bufs for fill queue replenishment after rx */ > + if (rte_pktmbuf_alloc_bulk(umem->mb_pool, fq_bufs, nb_pkts)) { > + AF_XDP_LOG(DEBUG, > + "Failed to get enough buffers for fq.\n"); > + rx->cached_cons -=3D nb_pkts; > + return 0; > } >=20 > - for (i =3D 0; i < rcvd; i++) { > + for (i =3D 0; i < nb_pkts; i++) { > const struct xdp_desc *desc; > uint64_t addr; > uint32_t len; > @@ -297,20 +298,14 @@ af_xdp_rx_zc(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > rx_bytes +=3D len; > } >=20 > - xsk_ring_cons__release(rx, rcvd); > - > - (void)reserve_fill_queue(umem, rcvd, fq_bufs, fq); > + xsk_ring_cons__release(rx, nb_pkts); > + (void)reserve_fill_queue(umem, nb_pkts, fq_bufs, fq); >=20 > /* statistics */ > - rxq->stats.rx_pkts +=3D rcvd; > + rxq->stats.rx_pkts +=3D nb_pkts; > rxq->stats.rx_bytes +=3D rx_bytes; >=20 > -out: > - if (rcvd !=3D nb_pkts) > - rte_mempool_put_bulk(umem->mb_pool, (void > **)&fq_bufs[rcvd], > - nb_pkts - rcvd); > - > - return rcvd; > + return nb_pkts; > } > #else > static uint16_t > @@ -322,7 +317,7 @@ af_xdp_rx_cp(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) > struct xsk_ring_prod *fq =3D &rxq->fq; > uint32_t idx_rx =3D 0; > unsigned long rx_bytes =3D 0; > - int rcvd, i; > + int i; > uint32_t free_thresh =3D fq->size >> 1; > struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE]; >=20 > @@ -330,20 +325,21 @@ af_xdp_rx_cp(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > (void)reserve_fill_queue(umem, > ETH_AF_XDP_RX_BATCH_SIZE, > NULL, fq); >=20 > - if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, nb_pkts) > !=3D 0)) > - return 0; > - > - rcvd =3D xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); > - if (rcvd =3D=3D 0) { > + nb_pkts =3D xsk_ring_cons__peek(rx, nb_pkts, &idx_rx); > + if (nb_pkts =3D=3D 0) { > #if defined(XDP_USE_NEED_WAKEUP) > if (xsk_ring_prod__needs_wakeup(fq)) > (void)poll(rxq->fds, 1, 1000); > #endif > + return 0; > + } >=20 > - goto out; > + if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, > nb_pkts))) { > + rx->cached_cons -=3D nb_pkts; > + return 0; > } >=20 > - for (i =3D 0; i < rcvd; i++) { > + for (i =3D 0; i < nb_pkts; i++) { > const struct xdp_desc *desc; > uint64_t addr; > uint32_t len; > @@ -362,18 +358,13 @@ af_xdp_rx_cp(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > bufs[i] =3D mbufs[i]; > } >=20 > - xsk_ring_cons__release(rx, rcvd); > + xsk_ring_cons__release(rx, nb_pkts); >=20 > /* statistics */ > - rxq->stats.rx_pkts +=3D rcvd; > + rxq->stats.rx_pkts +=3D nb_pkts; > rxq->stats.rx_bytes +=3D rx_bytes; >=20 > -out: > - if (rcvd !=3D nb_pkts) > - rte_mempool_put_bulk(rxq->mb_pool, (void > **)&mbufs[rcvd], > - nb_pkts - rcvd); > - > - return rcvd; > + return nb_pkts; > } > #endif >=20 > -- > 2.16.2