From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by dpdk.org (Postfix) with ESMTP id 604891B2A1 for ; Mon, 6 Nov 2017 02:42:20 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 147D520AE7; Sun, 5 Nov 2017 20:42:20 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Sun, 05 Nov 2017 20:42:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=Jlwm5H1thX8wDh sYHauH8v84xQDXhsUNkajx4z73PfM=; b=SIKCcmTJpJwZZdozzWi/Utz00slpMC pa2Hpsx3AkdwdOGv8YOKGEe290+d82189TWZVg0Aaee9vrDcNeT7M9wegrHjCiWN CZViEPNYUFWqWnWXDbFUkNuKylnKrZ8FUC0q7Fqxf47FkVlhZOYzxkObKYx6AX+M aPxXPGMUygHlA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=Jlwm5H1thX8wDhsYHauH8v84xQDXhsUNkajx4z73PfM=; b=Nm6XGKYg jZUxfC8R5o51vYCov9uYkypw2EpSIPqHg8RRzKVLPW8YoU0OM6vAGd/ERROJcW9n +tfr3I6VxBcyHlKgzc05dqmhyWTTLso1YhDpvGmHSorJ7aT5xigKCTdAzizN9ciL IxcwN6gGfCj5cWcsFFsZwA9joNHdi3yvopiGG8ZSr4VDHGN7gkBiJtpQCixxplr3 cHgi/KdR0D3FLjbqpzZSHKN7oZ9BXkhZgnWo+xpmR9TAgDQjCmgoHYxBJvdhS5ZS e3LQi5gAfIg3E5CV3l08lRtcIVGP33u1ZtZDQkNx4r6Dn4eWs4nZQYgXRnH3tDGX EuO0ao+r1kJ4bw== X-ME-Sender: Received: from xps.monjalon.net (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 256732469F; Sun, 5 Nov 2017 20:42:19 -0500 (EST) From: Thomas Monjalon To: Santosh Shukla Cc: olivier.matz@6wind.com, sergio.gonzalez.monroy@intel.com, anatoly.burakov@intel.com, dev@dpdk.org Date: Mon, 6 Nov 2017 02:41:37 +0100 Message-Id: <20171106014141.13266-12-thomas@monjalon.net> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171106014141.13266-1-thomas@monjalon.net> References: <20170814151537.29454-1-santosh.shukla@caviumnetworks.com> <20171106014141.13266-1-thomas@monjalon.net> Subject: [dpdk-dev] [PATCH v4 11/15] mbuf: rename physical address to IOVA X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Nov 2017 01:42:21 -0000 From: Santosh Shukla Rename buf_physaddr to buf_iova. Keep the deprecated name in an anonymous union to avoid breaking the API. Signed-off-by: Santosh Shukla Reviewed-by: Anatoly Burakov Signed-off-by: Thomas Monjalon --- app/test-crypto-perf/cperf_test_common.c | 4 ++-- doc/guides/prog_guide/img/mbuf1.svg | 2 +- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 2 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 4 ++-- drivers/net/ark/ark_ethdev_rx.c | 8 ++++---- drivers/net/bnx2x/bnx2x_rxtx.c | 4 ++-- drivers/net/bnxt/bnxt_ring.h | 2 +- drivers/net/cxgbe/sge.c | 4 ++-- drivers/net/dpaa/dpaa_rxtx.c | 6 +++--- drivers/net/ena/ena_ethdev.c | 6 +++--- drivers/net/enic/enic_main.c | 2 +- drivers/net/enic/enic_rxtx.c | 6 +++--- drivers/net/fm10k/fm10k.h | 4 ++-- drivers/net/fm10k/fm10k_rxtx_vec.c | 4 ++-- drivers/net/i40e/i40e_rxtx_vec_altivec.c | 4 ++-- drivers/net/i40e/i40e_rxtx_vec_neon.c | 6 +++--- drivers/net/i40e/i40e_rxtx_vec_sse.c | 6 +++--- drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c | 6 +++--- drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c | 6 +++--- drivers/net/nfp/nfp_net.c | 2 +- drivers/net/virtio/virtio_ethdev.c | 2 +- drivers/net/virtio/virtqueue.h | 2 +- lib/librte_kni/rte_kni.c | 2 +- lib/librte_mbuf/rte_mbuf.c | 6 +++--- lib/librte_mbuf/rte_mbuf.h | 18 +++++++++++------- lib/librte_vhost/virtio_net.c | 2 +- test/test/test_mbuf.c | 2 +- 27 files changed, 63 insertions(+), 59 deletions(-) diff --git a/app/test-crypto-perf/cperf_test_common.c b/app/test-crypto-perf/cperf_test_common.c index 08b313f8b..84c5a3d76 100644 --- a/app/test-crypto-perf/cperf_test_common.c +++ b/app/test-crypto-perf/cperf_test_common.c @@ -50,7 +50,7 @@ fill_single_seg_mbuf(struct rte_mbuf *m, struct rte_mempool *mp, /* start of buffer is after mbuf structure and priv data */ m->priv_size = 0; m->buf_addr = (char *)m + mbuf_hdr_size; - m->buf_physaddr = rte_mempool_virt2iova(obj) + + m->buf_iova = rte_mempool_virt2iova(obj) + mbuf_offset + mbuf_hdr_size; m->buf_len = segment_sz; m->data_len = segment_sz; @@ -81,7 +81,7 @@ fill_multi_seg_mbuf(struct rte_mbuf *m, struct rte_mempool *mp, /* start of buffer is after mbuf structure and priv data */ m->priv_size = 0; m->buf_addr = (char *)m + mbuf_hdr_size; - m->buf_physaddr = next_seg_phys_addr; + m->buf_iova = next_seg_phys_addr; next_seg_phys_addr += mbuf_hdr_size + segment_sz; m->buf_len = segment_sz; m->data_len = segment_sz; diff --git a/doc/guides/prog_guide/img/mbuf1.svg b/doc/guides/prog_guide/img/mbuf1.svg index 5bd84d1bf..8750424e3 100644 --- a/doc/guides/prog_guide/img/mbuf1.svg +++ b/doc/guides/prog_guide/img/mbuf1.svg @@ -482,7 +482,7 @@ sodipodi:role="line" x="187.85715" y="347.7193" - id="tspan5240">(m->buf_physaddr is the(m->buf_iova is thebuf_physaddr) +#define DPAA2_MBUF_VADDR_TO_IOVA(mbuf) ((mbuf)->buf_iova) #define DPAA2_OP_VADDR_TO_IOVA(op) (op->phys_addr) /** diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.c b/drivers/crypto/dpaa_sec/dpaa_sec.c index 30a2a3807..234c84f22 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec.c @@ -798,9 +798,9 @@ build_cipher_auth(struct rte_crypto_op *op, dpaa_sec_session *ses) uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, ses->iv.offset); - src_start_addr = sym->m_src->buf_physaddr + sym->m_src->data_off; + src_start_addr = sym->m_src->buf_iova + sym->m_src->data_off; if (sym->m_dst) - dst_start_addr = sym->m_dst->buf_physaddr + sym->m_dst->data_off; + dst_start_addr = sym->m_dst->buf_iova + sym->m_dst->data_off; else dst_start_addr = src_start_addr; diff --git a/drivers/net/ark/ark_ethdev_rx.c b/drivers/net/ark/ark_ethdev_rx.c index 4e39d9557..2ffaeac3e 100644 --- a/drivers/net/ark/ark_ethdev_rx.c +++ b/drivers/net/ark/ark_ethdev_rx.c @@ -499,22 +499,22 @@ eth_ark_rx_seed_mbufs(struct ark_rx_queue *queue) case 0: while (count != nb) { queue->paddress_q[seed_m++] = - (*mbufs++)->buf_physaddr; + (*mbufs++)->buf_iova; count++; /* FALLTHROUGH */ case 3: queue->paddress_q[seed_m++] = - (*mbufs++)->buf_physaddr; + (*mbufs++)->buf_iova; count++; /* FALLTHROUGH */ case 2: queue->paddress_q[seed_m++] = - (*mbufs++)->buf_physaddr; + (*mbufs++)->buf_iova; count++; /* FALLTHROUGH */ case 1: queue->paddress_q[seed_m++] = - (*mbufs++)->buf_physaddr; + (*mbufs++)->buf_iova; count++; /* FALLTHROUGH */ diff --git a/drivers/net/bnx2x/bnx2x_rxtx.c b/drivers/net/bnx2x/bnx2x_rxtx.c index 99d7d285c..7c3324a6d 100644 --- a/drivers/net/bnx2x/bnx2x_rxtx.c +++ b/drivers/net/bnx2x/bnx2x_rxtx.c @@ -140,7 +140,7 @@ bnx2x_dev_rx_queue_setup(struct rte_eth_dev *dev, return -ENOMEM; } rxq->sw_ring[idx] = mbuf; - rxq->rx_ring[idx] = mbuf->buf_physaddr; + rxq->rx_ring[idx] = mbuf->buf_iova; } rxq->pkt_first_seg = NULL; rxq->pkt_last_seg = NULL; @@ -400,7 +400,7 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rx_mb = rxq->sw_ring[bd_cons]; rxq->sw_ring[bd_cons] = new_mb; - rxq->rx_ring[bd_prod] = new_mb->buf_physaddr; + rxq->rx_ring[bd_prod] = new_mb->buf_iova; rx_pref = NEXT_RX_BD(bd_cons) & MAX_RX_BD(rxq); rte_prefetch0(rxq->sw_ring[rx_pref]); diff --git a/drivers/net/bnxt/bnxt_ring.h b/drivers/net/bnxt/bnxt_ring.h index 6d1eb5888..ac1ce2a57 100644 --- a/drivers/net/bnxt/bnxt_ring.h +++ b/drivers/net/bnxt/bnxt_ring.h @@ -41,7 +41,7 @@ #define RING_NEXT(ring, idx) (((idx) + 1) & (ring)->ring_mask) #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - ((uint64_t)((mb)->buf_physaddr + (mb)->data_off)) + ((uint64_t)((mb)->buf_iova + (mb)->data_off)) #define DB_IDX_MASK 0xffffff #define DB_IDX_VALID (0x1 << 26) diff --git a/drivers/net/cxgbe/sge.c b/drivers/net/cxgbe/sge.c index 9f915158a..fc10d9585 100644 --- a/drivers/net/cxgbe/sge.c +++ b/drivers/net/cxgbe/sge.c @@ -149,7 +149,7 @@ static int map_mbuf(struct rte_mbuf *mbuf, dma_addr_t *addr) struct rte_mbuf *m = mbuf; for (; m; m = m->next, addr++) { - *addr = m->buf_physaddr + rte_pktmbuf_headroom(m); + *addr = m->buf_iova + rte_pktmbuf_headroom(m); if (*addr == 0) goto out_err; } @@ -423,7 +423,7 @@ static unsigned int refill_fl_usembufs(struct adapter *adap, struct sge_fl *q, mbuf->nb_segs = 1; mbuf->port = rxq->rspq.port_id; - mapping = (dma_addr_t)RTE_ALIGN(mbuf->buf_physaddr + + mapping = (dma_addr_t)RTE_ALIGN(mbuf->buf_iova + mbuf->data_off, adap->sge.fl_align); mapping |= buf_size_idx; diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c index a5a0db714..41e57f2e2 100644 --- a/drivers/net/dpaa/dpaa_rxtx.c +++ b/drivers/net/dpaa/dpaa_rxtx.c @@ -77,7 +77,7 @@ (_fd)->opaque = QM_FD_CONTIG << DPAA_FD_FORMAT_SHIFT; \ (_fd)->opaque |= ((_mbuf)->data_off) << DPAA_FD_OFFSET_SHIFT; \ (_fd)->opaque |= (_mbuf)->pkt_len; \ - (_fd)->addr = (_mbuf)->buf_physaddr; \ + (_fd)->addr = (_mbuf)->buf_iova; \ (_fd)->bpid = _bpid; \ } while (0) @@ -514,7 +514,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf, sgt = temp->buf_addr + temp->data_off; fd->format = QM_FD_SG; - fd->addr = temp->buf_physaddr; + fd->addr = temp->buf_iova; fd->offset = temp->data_off; fd->bpid = bpid; fd->length20 = mbuf->pkt_len; @@ -523,7 +523,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf, sg_temp = &sgt[i++]; sg_temp->opaque = 0; sg_temp->val = 0; - sg_temp->addr = cur_seg->buf_physaddr; + sg_temp->addr = cur_seg->buf_iova; sg_temp->offset = cur_seg->data_off; sg_temp->length = cur_seg->data_len; if (RTE_MBUF_DIRECT(cur_seg)) { diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index a62c39875..22db8951f 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -1167,7 +1167,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) rte_prefetch0(mbufs[((next_to_use + 4) & ring_mask)]); /* prepare physical address for DMA transaction */ - ebuf.paddr = mbuf->buf_physaddr + RTE_PKTMBUF_HEADROOM; + ebuf.paddr = mbuf->buf_iova + RTE_PKTMBUF_HEADROOM; ebuf.len = mbuf->buf_len - RTE_PKTMBUF_HEADROOM; /* pass resource to device */ rc = ena_com_add_single_rx_desc(rxq->ena_com_io_sq, @@ -1726,7 +1726,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, * consideration pushed header */ if (mbuf->data_len > ena_tx_ctx.header_len) { - ebuf->paddr = mbuf->buf_physaddr + + ebuf->paddr = mbuf->buf_iova + mbuf->data_off + ena_tx_ctx.header_len; ebuf->len = mbuf->data_len - ena_tx_ctx.header_len; @@ -1735,7 +1735,7 @@ static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, } while ((mbuf = mbuf->next) != NULL) { - ebuf->paddr = mbuf->buf_physaddr + mbuf->data_off; + ebuf->paddr = mbuf->buf_iova + mbuf->data_off; ebuf->len = mbuf->data_len; ebuf++; tx_info->num_of_bufs++; diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index 3a2a8d182..8af0ccd3c 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -316,7 +316,7 @@ enic_alloc_rx_queue_mbufs(struct enic *enic, struct vnic_rq *rq) } mb->data_off = RTE_PKTMBUF_HEADROOM; - dma_addr = (dma_addr_t)(mb->buf_physaddr + dma_addr = (dma_addr_t)(mb->buf_iova + RTE_PKTMBUF_HEADROOM); rq_enet_desc_enc(rqd, dma_addr, (rq->is_sop ? RQ_ENET_TYPE_ONLY_SOP diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c index 1d43bde9a..74450ea71 100644 --- a/drivers/net/enic/enic_rxtx.c +++ b/drivers/net/enic/enic_rxtx.c @@ -386,7 +386,7 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, /* Push descriptor for newly allocated mbuf */ nmb->data_off = RTE_PKTMBUF_HEADROOM; - dma_addr = (dma_addr_t)(nmb->buf_physaddr + + dma_addr = (dma_addr_t)(nmb->buf_iova + RTE_PKTMBUF_HEADROOM); rq_enet_desc_enc(rqd_ptr, dma_addr, (rq->is_sop ? RQ_ENET_TYPE_ONLY_SOP @@ -578,7 +578,7 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, vlan_id = 0; vlan_tag_insert = 0; bus_addr = (dma_addr_t) - (tx_pkt->buf_physaddr + tx_pkt->data_off); + (tx_pkt->buf_iova + tx_pkt->data_off); descs = (struct wq_enet_desc *)wq->ring.descs; desc_p = descs + head_idx; @@ -630,7 +630,7 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, if (tx_pkt->next == NULL) eop = 1; desc_p = descs + head_idx; - bus_addr = (dma_addr_t)(tx_pkt->buf_physaddr + bus_addr = (dma_addr_t)(tx_pkt->buf_iova + tx_pkt->data_off); wq_enet_desc_enc((struct wq_enet_desc *) &desc_tmp, bus_addr, data_len, diff --git a/drivers/net/fm10k/fm10k.h b/drivers/net/fm10k/fm10k.h index 594dca4f9..1273aa865 100644 --- a/drivers/net/fm10k/fm10k.h +++ b/drivers/net/fm10k/fm10k.h @@ -253,11 +253,11 @@ struct fm10k_txq_ops { }; #define MBUF_DMA_ADDR(mb) \ - ((uint64_t) ((mb)->buf_physaddr + (mb)->data_off)) + ((uint64_t) ((mb)->buf_iova + (mb)->data_off)) /* enforce 512B alignment on default Rx DMA addresses */ #define MBUF_DMA_ADDR_DEFAULT(mb) \ - ((uint64_t) RTE_ALIGN(((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM),\ + ((uint64_t) RTE_ALIGN(((mb)->buf_iova + RTE_PKTMBUF_HEADROOM),\ FM10K_RX_DATABUF_ALIGN)) static inline void fifo_reset(struct fifo *fifo, uint32_t len) diff --git a/drivers/net/fm10k/fm10k_rxtx_vec.c b/drivers/net/fm10k/fm10k_rxtx_vec.c index edc4ae16e..ce042d3d2 100644 --- a/drivers/net/fm10k/fm10k_rxtx_vec.c +++ b/drivers/net/fm10k/fm10k_rxtx_vec.c @@ -330,8 +330,8 @@ fm10k_rxq_rearm(struct fm10k_rx_queue *rxq) p1 = (uintptr_t)&mb1->rearm_data; *(uint64_t *)p1 = rxq->mbuf_initializer; - /* load buf_addr(lo 64bit) and buf_physaddr(hi 64bit) */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_physaddr) != + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) != offsetof(struct rte_mbuf, buf_addr) + 8); vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr); vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr); diff --git a/drivers/net/i40e/i40e_rxtx_vec_altivec.c b/drivers/net/i40e/i40e_rxtx_vec_altivec.c index 5ab9445ed..5e4e472a3 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_altivec.c +++ b/drivers/net/i40e/i40e_rxtx_vec_altivec.c @@ -100,7 +100,7 @@ i40e_rxq_rearm(struct i40e_rx_queue *rxq) p1 = (uintptr_t)&mb1->rearm_data; *(uint64_t *)p1 = rxq->mbuf_initializer; - /* load buf_addr(lo 64bit) and buf_physaddr(hi 64bit) */ + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ vaddr0 = vec_ld(0, (vector unsigned long *)&mb0->buf_addr); vaddr1 = vec_ld(0, (vector unsigned long *)&mb1->buf_addr); @@ -538,7 +538,7 @@ vtx1(volatile struct i40e_tx_desc *txdp, ((uint64_t)pkt->data_len << I40E_TXD_QW1_TX_BUF_SZ_SHIFT)); vector unsigned long descriptor = (vector unsigned long){ - pkt->buf_physaddr + pkt->data_off, high_qw}; + pkt->buf_iova + pkt->data_off, high_qw}; *(vector unsigned long *)txdp = descriptor; } diff --git a/drivers/net/i40e/i40e_rxtx_vec_neon.c b/drivers/net/i40e/i40e_rxtx_vec_neon.c index d0e604068..12219274e 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_neon.c +++ b/drivers/net/i40e/i40e_rxtx_vec_neon.c @@ -81,13 +81,13 @@ i40e_rxq_rearm(struct i40e_rx_queue *rxq) mb0 = rxep[0].mbuf; mb1 = rxep[1].mbuf; - paddr = mb0->buf_physaddr + RTE_PKTMBUF_HEADROOM; + paddr = mb0->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr0 = vdupq_n_u64(paddr); /* flush desc with pa dma_addr */ vst1q_u64((uint64_t *)&rxdp++->read, dma_addr0); - paddr = mb1->buf_physaddr + RTE_PKTMBUF_HEADROOM; + paddr = mb1->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr1 = vdupq_n_u64(paddr); vst1q_u64((uint64_t *)&rxdp++->read, dma_addr1); } @@ -515,7 +515,7 @@ vtx1(volatile struct i40e_tx_desc *txdp, ((uint64_t)flags << I40E_TXD_QW1_CMD_SHIFT) | ((uint64_t)pkt->data_len << I40E_TXD_QW1_TX_BUF_SZ_SHIFT)); - uint64x2_t descriptor = {pkt->buf_physaddr + pkt->data_off, high_qw}; + uint64x2_t descriptor = {pkt->buf_iova + pkt->data_off, high_qw}; vst1q_u64((uint64_t *)txdp, descriptor); } diff --git a/drivers/net/i40e/i40e_rxtx_vec_sse.c b/drivers/net/i40e/i40e_rxtx_vec_sse.c index 86e16ddf9..9d2d1f837 100644 --- a/drivers/net/i40e/i40e_rxtx_vec_sse.c +++ b/drivers/net/i40e/i40e_rxtx_vec_sse.c @@ -86,8 +86,8 @@ i40e_rxq_rearm(struct i40e_rx_queue *rxq) mb0 = rxep[0].mbuf; mb1 = rxep[1].mbuf; - /* load buf_addr(lo 64bit) and buf_physaddr(hi 64bit) */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_physaddr) != + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) != offsetof(struct rte_mbuf, buf_addr) + 8); vaddr0 = _mm_loadu_si128((__m128i *)&mb0->buf_addr); vaddr1 = _mm_loadu_si128((__m128i *)&mb1->buf_addr); @@ -549,7 +549,7 @@ vtx1(volatile struct i40e_tx_desc *txdp, ((uint64_t)pkt->data_len << I40E_TXD_QW1_TX_BUF_SZ_SHIFT)); __m128i descriptor = _mm_set_epi64x(high_qw, - pkt->buf_physaddr + pkt->data_off); + pkt->buf_iova + pkt->data_off); _mm_store_si128((__m128i *)txdp, descriptor); } diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c index 18e3d199f..2e87ffa0a 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c @@ -87,13 +87,13 @@ ixgbe_rxq_rearm(struct ixgbe_rx_queue *rxq) * Data to be rearmed is 6 bytes long. */ vst1_u8((uint8_t *)&mb0->rearm_data, p); - paddr = mb0->buf_physaddr + RTE_PKTMBUF_HEADROOM; + paddr = mb0->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr0 = vsetq_lane_u64(paddr, zero, 0); /* flush desc with pa dma_addr */ vst1q_u64((uint64_t *)&rxdp++->read, dma_addr0); vst1_u8((uint8_t *)&mb1->rearm_data, p); - paddr = mb1->buf_physaddr + RTE_PKTMBUF_HEADROOM; + paddr = mb1->buf_iova + RTE_PKTMBUF_HEADROOM; dma_addr1 = vsetq_lane_u64(paddr, zero, 0); vst1q_u64((uint64_t *)&rxdp++->read, dma_addr1); } @@ -414,7 +414,7 @@ vtx1(volatile union ixgbe_adv_tx_desc *txdp, struct rte_mbuf *pkt, uint64_t flags) { uint64x2_t descriptor = { - pkt->buf_physaddr + pkt->data_off, + pkt->buf_iova + pkt->data_off, (uint64_t)pkt->pkt_len << 46 | flags | pkt->data_len}; vst1q_u64((uint64_t *)&txdp->read, descriptor); diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c index 158ccd91d..a3b5b6bfb 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_sse.c @@ -86,8 +86,8 @@ ixgbe_rxq_rearm(struct ixgbe_rx_queue *rxq) mb0 = rxep[0].mbuf; mb1 = rxep[1].mbuf; - /* load buf_addr(lo 64bit) and buf_physaddr(hi 64bit) */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_physaddr) != + /* load buf_addr(lo 64bit) and buf_iova(hi 64bit) */ + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_iova) != offsetof(struct rte_mbuf, buf_addr) + 8); vaddr0 = _mm_loadu_si128((__m128i *)&(mb0->buf_addr)); vaddr1 = _mm_loadu_si128((__m128i *)&(mb1->buf_addr)); @@ -667,7 +667,7 @@ vtx1(volatile union ixgbe_adv_tx_desc *txdp, { __m128i descriptor = _mm_set_epi64x((uint64_t)pkt->pkt_len << 46 | flags | pkt->data_len, - pkt->buf_physaddr + pkt->data_off); + pkt->buf_iova + pkt->data_off); _mm_store_si128((__m128i *)&txdp->read, descriptor); } diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c index 7f1a026ac..e141627cc 100644 --- a/drivers/net/nfp/nfp_net.c +++ b/drivers/net/nfp/nfp_net.c @@ -106,7 +106,7 @@ static uint16_t nfp_net_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, #define NFP_QCP_MAX_ADD 0x7f #define RTE_MBUF_DMA_ADDR_DEFAULT(mb) \ - (uint64_t)((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM) + (uint64_t)((mb)->buf_iova + RTE_PKTMBUF_HEADROOM) /* nfp_qcp_ptr - Read or Write Pointer of a queue */ enum nfp_qcp_ptr { diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index c6756ed3f..d2576d5e0 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -493,7 +493,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx) * VIRTIO_MBUF_DATA_DMA_ADDR in virtqueue.h for more information. */ if (!hw->virtio_user_dev) - vq->offset = offsetof(struct rte_mbuf, buf_physaddr); + vq->offset = offsetof(struct rte_mbuf, buf_iova); else { vq->vq_ring_mem = (uintptr_t)mz->addr; vq->offset = offsetof(struct rte_mbuf, buf_addr); diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index 235f8c525..6c90d3779 100644 --- a/drivers/net/virtio/virtqueue.h +++ b/drivers/net/virtio/virtqueue.h @@ -79,7 +79,7 @@ struct rte_mbuf; #define VIRTIO_MBUF_ADDR(mb, vq) \ ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->offset))) #else -#define VIRTIO_MBUF_ADDR(mb, vq) ((mb)->buf_physaddr) +#define VIRTIO_MBUF_ADDR(mb, vq) ((mb)->buf_iova) #endif /** diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 8c483c1f4..5ee38e9ad 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -456,7 +456,7 @@ va2pa(struct rte_mbuf *m) { return (void *)((unsigned long)m - ((unsigned long)m->buf_addr - - (unsigned long)m->buf_physaddr)); + (unsigned long)m->buf_iova)); } static void diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c index c96943522..71c65a042 100644 --- a/lib/librte_mbuf/rte_mbuf.c +++ b/lib/librte_mbuf/rte_mbuf.c @@ -134,7 +134,7 @@ rte_pktmbuf_init(struct rte_mempool *mp, /* start of buffer is after mbuf structure and priv data */ m->priv_size = priv_size; m->buf_addr = (char *)m + mbuf_size; - m->buf_physaddr = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; m->buf_len = (uint16_t)buf_len; /* keep some headroom between start of buffer and data */ @@ -211,7 +211,7 @@ rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header) /* generic checks */ if (m->pool == NULL) rte_panic("bad mbuf pool\n"); - if (m->buf_physaddr == 0) + if (m->buf_iova == 0) rte_panic("bad phys addr\n"); if (m->buf_addr == NULL) rte_panic("bad virt addr\n"); @@ -244,7 +244,7 @@ rte_pktmbuf_dump(FILE *f, const struct rte_mbuf *m, unsigned dump_len) __rte_mbuf_sanity_check(m, 1); fprintf(f, "dump mbuf at %p, phys=%"PRIx64", buf_len=%u\n", - m, (uint64_t)m->buf_physaddr, (unsigned)m->buf_len); + m, (uint64_t)m->buf_iova, (unsigned)m->buf_len); fprintf(f, " pkt_len=%"PRIu32", ol_flags=%"PRIx64", nb_segs=%u, " "in_port=%u\n", m->pkt_len, m->ol_flags, (unsigned)m->nb_segs, (unsigned)m->port); diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index 0a555e2d5..7a4634fdd 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -432,7 +432,11 @@ struct rte_mbuf { * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes * working on vector drivers easier. */ - phys_addr_t buf_physaddr __rte_aligned(sizeof(phys_addr_t)); + RTE_STD_C11 + union { + rte_iova_t buf_iova; + rte_iova_t buf_physaddr; /**< deprecated */ + } __rte_aligned(sizeof(rte_iova_t)); /* next 8 bytes are initialised on RX descriptor rearm */ MARKER64 rearm_data; @@ -631,7 +635,7 @@ static inline uint16_t rte_pktmbuf_priv_size(struct rte_mempool *mp); static inline phys_addr_t rte_mbuf_data_dma_addr(const struct rte_mbuf *mb) { - return mb->buf_physaddr + mb->data_off; + return mb->buf_iova + mb->data_off; } /** @@ -649,7 +653,7 @@ rte_mbuf_data_dma_addr(const struct rte_mbuf *mb) static inline phys_addr_t rte_mbuf_data_dma_addr_default(const struct rte_mbuf *mb) { - return mb->buf_physaddr + RTE_PKTMBUF_HEADROOM; + return mb->buf_iova + RTE_PKTMBUF_HEADROOM; } /** @@ -840,7 +844,7 @@ rte_mbuf_sanity_check(const struct rte_mbuf *m, int is_header); * For standard needs, prefer rte_pktmbuf_alloc(). * * The caller can expect that the following fields of the mbuf structure - * are initialized: buf_addr, buf_physaddr, buf_len, refcnt=1, nb_segs=1, + * are initialized: buf_addr, buf_iova, buf_len, refcnt=1, nb_segs=1, * next=NULL, pool, priv_size. The other fields must be initialized * by the caller. * @@ -1250,7 +1254,7 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf *mi, struct rte_mbuf *m) rte_mbuf_refcnt_update(md, 1); mi->priv_size = m->priv_size; - mi->buf_physaddr = m->buf_physaddr; + mi->buf_iova = m->buf_iova; mi->buf_addr = m->buf_addr; mi->buf_len = m->buf_len; @@ -1298,7 +1302,7 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m) m->priv_size = priv_size; m->buf_addr = (char *)m + mbuf_size; - m->buf_physaddr = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; m->buf_len = (uint16_t)buf_len; rte_pktmbuf_reset_headroom(m); m->data_len = 0; @@ -1569,7 +1573,7 @@ static inline struct rte_mbuf *rte_pktmbuf_lastseg(struct rte_mbuf *m) * The offset into the data to calculate address from. */ #define rte_pktmbuf_mtophys_offset(m, o) \ - (phys_addr_t)((m)->buf_physaddr + (m)->data_off + (o)) + (rte_iova_t)((m)->buf_iova + (m)->data_off + (o)) /** * A macro that returns the physical address that points to the start of the diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index 3c6f1a531..6fee16e55 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -978,7 +978,7 @@ copy_desc_to_mbuf(struct virtio_net *dev, struct vhost_virtqueue *vq, cur->data_len = cpy_len; cur->data_off = 0; cur->buf_addr = (void *)(uintptr_t)desc_addr; - cur->buf_physaddr = hpa; + cur->buf_iova = hpa; /* * In zero copy mode, one mbuf can only reference data diff --git a/test/test/test_mbuf.c b/test/test/test_mbuf.c index 20e146b01..414509b20 100644 --- a/test/test/test_mbuf.c +++ b/test/test/test_mbuf.c @@ -906,7 +906,7 @@ test_failing_mbuf_sanity_check(struct rte_mempool *pktmbuf_pool) } badbuf = *buf; - badbuf.buf_physaddr = 0; + badbuf.buf_iova = 0; if (verify_mbuf_check_panics(&badbuf)) { printf("Error with bad-physaddr mbuf test\n"); return -1; -- 2.14.2