From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f175.google.com (mail-wi0-f175.google.com [209.85.212.175]) by dpdk.org (Postfix) with ESMTP id 1FBC6959 for ; Fri, 9 May 2014 16:50:56 +0200 (CEST) Received: by mail-wi0-f175.google.com with SMTP id f8so1452685wiw.14 for ; Fri, 09 May 2014 07:51:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2HbNT3bozSwJcRe+ULx3NdPqgMZadYjeu2t/fdAN8ew=; b=RyM51pf/90XulQ5UxWx3r3eYq56eVIUrqWXIexNzbxzQ5SL/SIuxoYLmLKoDJkKl9f h0PqurTRys64f4FjDz9ZJgilE6MXI0Hj8IiIVQ1QltaKgAgeOkA1v4IFIjX4hvVLcVAn RpsXUldgJf0Q5z3nNLlDWLhCA/Eq50RxO4ReOYjJCC3FpCS+qITUc68ue0FtbICicgOz VRG410+T0/8m28fanmeBoWZuekk53oVcZlkxJF1SwG3s+8FxSUFG9hrElVEJXjAEIjtw 6gb3uYXY9GPURIRX8lyLbwFg1CVikEFvRYhZdVAa3oiieQCEU4p3bsR0BQtCDwaGR+m7 Jatg== X-Gm-Message-State: ALoCoQlbINRzsXzZZI7w7Hk6XO3PnsmmZKE/ihLnXjKA5mV3Z1FnZufUUQg0ydh1rSkLkoDqlWOo X-Received: by 10.180.78.5 with SMTP id x5mr3792372wiw.12.1399647062349; Fri, 09 May 2014 07:51:02 -0700 (PDT) Received: from glumotte.dev.6wind.com (6wind.net2.nerim.net. [213.41.180.237]) by mx.google.com with ESMTPSA id c2sm5744122wja.18.2014.05.09.07.51.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 09 May 2014 07:51:01 -0700 (PDT) From: Olivier Matz To: dev@dpdk.org Date: Fri, 9 May 2014 16:50:33 +0200 Message-Id: <1399647038-15095-7-git-send-email-olivier.matz@6wind.com> X-Mailer: git-send-email 1.9.2 In-Reply-To: <1399647038-15095-1-git-send-email-olivier.matz@6wind.com> References: <1399647038-15095-1-git-send-email-olivier.matz@6wind.com> Subject: [dpdk-dev] [PATCH RFC 06/11] mbuf: replace data pointer by an offset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 14:50:56 -0000 The mbuf structure already contains a pointer to the beginning of the buffer (m->buf_addr). It is not needed to use 8 bytes again to store another pointer to the beginning of the data. Using a 16 bits unsigned integer is enough as we know that a mbuf is never longer than 64KB. We gain 6 bytes in the structure thanks to this modification. Signed-off-by: Olivier Matz --- app/test-pmd/csumonly.c | 2 +- app/test-pmd/macfwd-retry.c | 2 +- app/test-pmd/macfwd.c | 2 +- app/test-pmd/rxonly.c | 2 +- app/test-pmd/testpmd.c | 2 +- app/test-pmd/txonly.c | 7 ++-- app/test/test_mbuf.c | 6 ++-- examples/exception_path/main.c | 3 +- examples/vhost/main.c | 21 +++++++----- examples/vhost_xen/main.c | 2 +- lib/librte_mbuf/rte_mbuf.c | 7 ++-- lib/librte_mbuf/rte_mbuf.h | 62 ++++++++++++++++------------------- lib/librte_pmd_e1000/em_rxtx.c | 12 +++---- lib/librte_pmd_e1000/igb_rxtx.c | 13 ++++---- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 13 ++++---- lib/librte_pmd_ixgbe/ixgbe_rxtx.h | 3 +- lib/librte_pmd_virtio/virtio_rxtx.c | 2 +- lib/librte_pmd_virtio/virtqueue.h | 5 ++- lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c | 5 ++- 19 files changed, 85 insertions(+), 86 deletions(-) diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c index ee82eb6..3313b87 100644 --- a/app/test-pmd/csumonly.c +++ b/app/test-pmd/csumonly.c @@ -263,7 +263,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs) pkt_ol_flags = mb->ol_flags; ol_flags = (uint16_t) (pkt_ol_flags & (~PKT_TX_L4_MASK)); - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type = rte_be_to_cpu_16(eth_hdr->ether_type); if (eth_type == ETHER_TYPE_VLAN) { /* Only allow single VLAN label here */ diff --git a/app/test-pmd/macfwd-retry.c b/app/test-pmd/macfwd-retry.c index 687ff8d..7749c9e 100644 --- a/app/test-pmd/macfwd-retry.c +++ b/app/test-pmd/macfwd-retry.c @@ -119,7 +119,7 @@ pkt_burst_mac_retry_forward(struct fwd_stream *fs) fs->rx_packets += nb_rx; for (i = 0; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); ether_addr_copy(&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr, diff --git a/app/test-pmd/macfwd.c b/app/test-pmd/macfwd.c index 8d7612c..ab74d0c 100644 --- a/app/test-pmd/macfwd.c +++ b/app/test-pmd/macfwd.c @@ -110,7 +110,7 @@ pkt_burst_mac_forward(struct fwd_stream *fs) txp = &ports[fs->tx_port]; for (i = 0; i < nb_rx; i++) { mb = pkts_burst[i]; - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); ether_addr_copy(&peer_eth_addrs[fs->peer_addr], ð_hdr->d_addr); ether_addr_copy(&ports[fs->tx_port].eth_addr, diff --git a/app/test-pmd/rxonly.c b/app/test-pmd/rxonly.c index b77c8ce..5751b0b 100644 --- a/app/test-pmd/rxonly.c +++ b/app/test-pmd/rxonly.c @@ -149,7 +149,7 @@ pkt_burst_receive(struct fwd_stream *fs) rte_pktmbuf_free(mb); continue; } - eth_hdr = (struct ether_hdr *) mb->data; + eth_hdr = rte_pktmbuf_mtod(mb, struct ether_hdr *); eth_type = RTE_BE_TO_CPU_16(eth_hdr->ether_type); ol_flags = mb->ol_flags; print_ether_addr(" src=", ð_hdr->s_addr); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 1964020..572c3aa 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -395,7 +395,7 @@ testpmd_mbuf_ctor(struct rte_mempool *mp, mb_ctor_arg->seg_buf_offset); mb->buf_len = mb_ctor_arg->seg_buf_size; mb->ol_flags = 0; - mb->data = (char *) mb->buf_addr + RTE_PKTMBUF_HEADROOM; + mb->data_off = RTE_PKTMBUF_HEADROOM; mb->nb_segs = 1; mb->vlan_macip.data = 0; mb->hash.rss = 0; diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c index 3baa0c8..c28f3dd 100644 --- a/app/test-pmd/txonly.c +++ b/app/test-pmd/txonly.c @@ -111,13 +111,13 @@ copy_buf_to_pkt_segs(void* buf, unsigned len, struct rte_mbuf *pkt, seg = seg->next; } copy_len = seg->data_len - offset; - seg_buf = ((char *) seg->data + offset); + seg_buf = (rte_pktmbuf_mtod(seg, char *) + offset); while (len > copy_len) { rte_memcpy(seg_buf, buf, (size_t) copy_len); len -= copy_len; buf = ((char*) buf + copy_len); seg = seg->next; - seg_buf = seg->data; + seg_buf = rte_pktmbuf_mtod(seg, char *); } rte_memcpy(seg_buf, buf, (size_t) len); } @@ -126,7 +126,8 @@ static inline void copy_buf_to_pkt(void* buf, unsigned len, struct rte_mbuf *pkt, unsigned offset) { if (offset + len <= pkt->data_len) { - rte_memcpy(((char *) pkt->data + offset), buf, (size_t) len); + rte_memcpy((rte_pktmbuf_mtod(pkt, char *) + offset), + buf, (size_t) len); return; } copy_buf_to_pkt_segs(buf, len, pkt, offset); diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index 320d76f..f6ce8ac 100644 --- a/app/test/test_mbuf.c +++ b/app/test/test_mbuf.c @@ -432,7 +432,7 @@ test_pktmbuf_pool_ptr(void) printf("rte_pktmbuf_alloc() failed (%u)\n", i); ret = -1; } - m[i]->data = RTE_PTR_ADD(m[i]->data, 64); + m[i]->data_off += 64; } /* free them */ @@ -451,8 +451,8 @@ test_pktmbuf_pool_ptr(void) printf("rte_pktmbuf_alloc() failed (%u)\n", i); ret = -1; } - if (m[i]->data != RTE_PTR_ADD(m[i]->buf_addr, RTE_PKTMBUF_HEADROOM)) { - printf ("data pointer not set properly\n"); + if (m[i]->data_off != RTE_PKTMBUF_HEADROOM) { + printf ("invalid data_off\n"); ret = -1; } } diff --git a/examples/exception_path/main.c b/examples/exception_path/main.c index d9a85b5..ca65511 100644 --- a/examples/exception_path/main.c +++ b/examples/exception_path/main.c @@ -302,7 +302,8 @@ main_loop(__attribute__((unused)) void *arg) if (m == NULL) continue; - ret = read(tap_fd, m->data, MAX_PACKET_SZ); + ret = read(tap_fd, rte_pktmbuf_mtod(m, void *), + MAX_PACKET_SZ); lcore_stats[lcore_id].rx++; if (unlikely(ret < 0)) { FATAL_ERROR("Reading from %s interface failed", diff --git a/examples/vhost/main.c b/examples/vhost/main.c index 26cfc8e..c65f24d 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -815,7 +815,9 @@ virtio_dev_rx(struct virtio_net *dev, struct rte_mbuf **pkts, uint32_t count) vq->used->ring[res_cur_idx & (vq->size - 1)].len = packet_len; /* Copy mbuf data to buffer */ - rte_memcpy((void *)(uintptr_t)buff_addr, (const void*)buff->data, rte_pktmbuf_data_len(buff)); + rte_memcpy((void *)(uintptr_t)buff_addr, + rte_pktmbuf_mtod(buff, const void *), + rte_pktmbuf_data_len(buff)); res_cur_idx++; packet_success++; @@ -877,7 +879,7 @@ link_vmdq(struct virtio_net *dev, struct rte_mbuf *m) int i, ret; /* Learn MAC address of guest device from packet */ - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); dev_ll = ll_root_used; @@ -965,7 +967,7 @@ virtio_tx_local(struct virtio_net *dev, struct rte_mbuf *m) struct ether_hdr *pkt_hdr; uint64_t ret = 0; - pkt_hdr = (struct ether_hdr *)m->data; + pkt_hdr = rte_pktmbuf_mtod(m, struct ether_hdr *); /*get the used devices list*/ dev_ll = ll_root_used; @@ -1042,18 +1044,21 @@ virtio_tx_route(struct virtio_net* dev, struct rte_mbuf *m, struct rte_mempool * mbuf->pkt_len = mbuf->data_len; /* Copy ethernet header to mbuf. */ - rte_memcpy((void*)mbuf->data, (const void*)m->data, ETH_HLEN); + rte_memcpy(rte_pktmbuf_mtod(mbuf, void *), + rte_pktmbuf_mtod(m, const void *), + ETH_HLEN); /* Setup vlan header. Bytes need to be re-ordered for network with htons()*/ - vlan_hdr = (struct vlan_ethhdr *) mbuf->data; + vlan_hdr = rte_pktmbuf_mtod(mbuf, struct vlan_ethhdr *); vlan_hdr->h_vlan_encapsulated_proto = vlan_hdr->h_vlan_proto; vlan_hdr->h_vlan_proto = htons(ETH_P_8021Q); vlan_hdr->h_vlan_TCI = htons(vlan_tag); /* Copy the remaining packet contents to the mbuf. */ - rte_memcpy((void*) ((uint8_t*)mbuf->data + VLAN_ETH_HLEN), - (const void*) ((uint8_t*)m->data + ETH_HLEN), (m->data_len - ETH_HLEN)); + rte_memcpy((void *)(rte_pktmbuf_mtod(mbuf, uint8_t *) + VLAN_ETH_HLEN), + (const void *)(rte_pktmbuf_mtod(m, uint8_t *) + ETH_HLEN), + (m->data_len - ETH_HLEN)); tx_q->m_table[len] = mbuf; len++; if (enable_stats) { @@ -1144,7 +1149,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool) /* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; - m.data = (void*)(uintptr_t)buff_addr; + m.data_off = 0; PRINT_PACKET(dev, (uintptr_t)buff_addr, desc->len, 0); diff --git a/examples/vhost_xen/main.c b/examples/vhost_xen/main.c index 2cf0029..6da1b22 100644 --- a/examples/vhost_xen/main.c +++ b/examples/vhost_xen/main.c @@ -981,7 +981,7 @@ virtio_dev_tx(struct virtio_net* dev, struct rte_mempool *mbuf_pool) /* Setup dummy mbuf. This is copied to a real mbuf if transmitted out the physical port. */ m.data_len = desc->len; - m.data = (void*)(uintptr_t)buff_addr; + m.data_off = 0; m.nb_segs = 1; virtio_tx_route(dev, &m, mbuf_pool, 0); diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c index 9879095..43e6d32 100644 --- a/lib/librte_mbuf/rte_mbuf.c +++ b/lib/librte_mbuf/rte_mbuf.c @@ -104,8 +104,7 @@ rte_pktmbuf_init(struct rte_mempool *mp, m->buf_len = (uint16_t)buf_len; /* keep some headroom between start of buffer and data */ - m->data = (char*) m->buf_addr + RTE_MIN(RTE_PKTMBUF_HEADROOM, - (uint16_t)m->buf_len); + m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len); /* init some constant fields */ m->pool = mp; @@ -171,12 +170,12 @@ rte_pktmbuf_dump(const struct rte_mbuf *m, unsigned dump_len) __rte_mbuf_sanity_check(m, 0); printf(" segment at 0x%p, data=0x%p, data_len=%u\n", - m, m->data, (unsigned)m->data_len); + m, rte_pktmbuf_mtod(m, void *), (unsigned)m->data_len); len = dump_len; if (len > m->data_len) len = m->data_len; if (len != 0) - rte_hexdump(NULL, m->data, len); + rte_hexdump(NULL, rte_pktmbuf_mtod(m, void *), len); dump_len -= len; m = m->next; nb_segs --; diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index 275f6b2..8fa781b 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -132,6 +132,13 @@ struct rte_mbuf { void *buf_addr; /**< Virtual address of segment buffer. */ uint64_t buf_physaddr:48; /**< Physical address of segment buffer. */ uint64_t buf_len:16; /**< Length of segment buffer. */ + + /* valid for any segment */ + struct rte_mbuf *next; /**< Next segment of scattered packet. */ + uint16_t data_off; + uint16_t data_len; /**< Amount of data in segment buffer. */ + uint32_t pkt_len; /**< Total pkt len: sum of all segments. */ + #ifdef RTE_MBUF_REFCNT /** * 16-bit Reference counter. @@ -142,36 +149,30 @@ struct rte_mbuf { * config option. */ union { - rte_atomic16_t refcnt_atomic; /**< Atomically accessed refcnt */ - uint16_t refcnt; /**< Non-atomically accessed refcnt */ + rte_atomic16_t refcnt_atomic; /**< Atomically accessed refcnt */ + uint16_t refcnt; /**< Non-atomically accessed refcnt */ }; #else - uint16_t refcnt_reserved; /**< Do not use this field */ + uint16_t refcnt_reserved; /**< Do not use this field */ #endif - uint16_t ol_flags; /**< Offload features. */ - uint32_t reserved; /**< Unused field. Required for padding. */ - - /* valid for any segment */ - struct rte_mbuf *next; /**< Next segment of scattered packet. */ - void* data; /**< Start address of data in segment buffer. */ - uint16_t data_len; /**< Amount of data in segment buffer. */ - /* these fields are valid for first segment only */ - uint8_t nb_segs; /**< Number of segments. */ - uint8_t in_port; /**< Input port. */ - uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */ + uint8_t nb_segs; /**< Number of segments. */ + uint8_t in_port; /**< Input port. */ + uint16_t ol_flags; /**< Offload features. */ + uint16_t reserved; /**< Unused field. Required for padding. */ /* offload features, valid for first segment only */ union rte_vlan_macip vlan_macip; union { - uint32_t rss; /**< RSS hash result if RSS enabled */ + uint32_t rss; /**< RSS hash result if RSS enabled */ struct { uint16_t hash; uint16_t id; - } fdir; /**< Filter identifier if FDIR enabled */ - uint32_t sched; /**< Hierarchical scheduler */ - } hash; /**< hash information */ + } fdir; /**< Filter identifier if FDIR enabled */ + uint32_t sched; /**< Hierarchical scheduler */ + } hash; /**< hash information */ + uint64_t reserved2; /**< Unused field. Required for padding. */ } __rte_cache_aligned; /** @@ -436,8 +437,6 @@ void rte_pktmbuf_pool_init(struct rte_mempool *mp, void *opaque_arg); */ static inline void rte_pktmbuf_reset(struct rte_mbuf *m) { - uint32_t buf_ofs; - m->next = NULL; m->pkt_len = 0; m->vlan_macip.data = 0; @@ -445,9 +444,8 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf *m) m->in_port = 0xff; m->ol_flags = 0; - buf_ofs = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ? + m->data_off = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ? RTE_PKTMBUF_HEADROOM : m->buf_len; - m->data = (char*) m->buf_addr + buf_ofs; m->data_len = 0; __rte_mbuf_sanity_check(m, 1); @@ -504,7 +502,7 @@ static inline void rte_pktmbuf_attach(struct rte_mbuf *mi, struct rte_mbuf *md) mi->buf_len = md->buf_len; mi->next = md->next; - mi->data = md->data; + mi->data_off = md->data_off; mi->data_len = md->data_len; mi->in_port = md->in_port; mi->vlan_macip = md->vlan_macip; @@ -533,16 +531,14 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m) { const struct rte_mempool *mp = m->pool; void *buf = RTE_MBUF_TO_BADDR(m); - uint32_t buf_ofs; uint32_t buf_len = mp->elt_size - sizeof(*m); m->buf_physaddr = rte_mempool_virt2phy(mp, m) + sizeof (*m); m->buf_addr = buf; m->buf_len = (uint16_t)buf_len; - buf_ofs = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ? + m->data_off = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ? RTE_PKTMBUF_HEADROOM : m->buf_len; - m->data = (char*) m->buf_addr + buf_ofs; m->data_len = 0; } @@ -706,7 +702,7 @@ static inline void rte_pktmbuf_refcnt_update(struct rte_mbuf *m, int16_t v) static inline uint16_t rte_pktmbuf_headroom(const struct rte_mbuf *m) { __rte_mbuf_sanity_check(m, 1); - return (uint16_t) ((char*) m->data - (char*) m->buf_addr); + return m->data_off; } /** @@ -754,7 +750,7 @@ static inline struct rte_mbuf *rte_pktmbuf_lastseg(struct rte_mbuf *m) * @param t * The type to cast the result into. */ -#define rte_pktmbuf_mtod(m, t) ((t)((m)->data)) +#define rte_pktmbuf_mtod(m, t) ((t)((char *)(m)->buf_addr + (m)->data_off)) /** * A macro that returns the length of the packet. @@ -799,11 +795,11 @@ static inline char *rte_pktmbuf_prepend(struct rte_mbuf *m, if (unlikely(len > rte_pktmbuf_headroom(m))) return NULL; - m->data = (char*) m->data - len; + m->data_off -= len; m->data_len = (uint16_t)(m->data_len + len); m->pkt_len = (m->pkt_len + len); - return (char*) m->data; + return (char*) m->buf_addr + m->data_off; } /** @@ -832,7 +828,7 @@ static inline char *rte_pktmbuf_append(struct rte_mbuf *m, uint16_t len) if (unlikely(len > rte_pktmbuf_tailroom(m_last))) return NULL; - tail = (char*) m_last->data + m_last->data_len; + tail = (char*) m_last->buf_addr + m_last->data_off + m_last->data_len; m_last->data_len = (uint16_t)(m_last->data_len + len); m->pkt_len = (m->pkt_len + len); return (char*) tail; @@ -860,9 +856,9 @@ static inline char *rte_pktmbuf_adj(struct rte_mbuf *m, uint16_t len) return NULL; m->data_len = (uint16_t)(m->data_len - len); - m->data = ((char*) m->data + len); + m->data_off += len; m->pkt_len = (m->pkt_len - len); - return (char*) m->data; + return (char *)m->buf_addr + m->data_off; } /** diff --git a/lib/librte_pmd_e1000/em_rxtx.c b/lib/librte_pmd_e1000/em_rxtx.c index b9e66eb..b050a6b 100644 --- a/lib/librte_pmd_e1000/em_rxtx.c +++ b/lib/librte_pmd_e1000/em_rxtx.c @@ -90,8 +90,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) } #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off) #define RTE_MBUF_DATA_DMA_ADDR_DEFAULT(mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM) @@ -771,8 +770,8 @@ eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ pkt_len = (uint16_t) (rte_le_to_cpu_16(rxd.length) - rxq->crc_len); - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; - rte_packet_prefetch(rxm->data); + rxm->data_off = RTE_PKTMBUF_HEADROOM; + rte_packet_prefetch((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs = 1; rxm->next = NULL; rxm->pkt_len = pkt_len; @@ -941,7 +940,7 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ data_len = rte_le_to_cpu_16(rxd.length); rxm->data_len = data_len; - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; + rxm->data_off = RTE_PKTMBUF_HEADROOM; /* * If this is the first buffer of the received packet, @@ -1013,7 +1012,8 @@ eth_em_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, rxm->vlan_macip.f.vlan_tci = rte_le_to_cpu_16(rxd.special); /* Prefetch data of first segment, if configured to do so. */ - rte_packet_prefetch(first_seg->data); + rte_packet_prefetch((char *)first_seg->buf_addr + + first_seg->data_off); /* * Store the mbuf address into the next entry of the array diff --git a/lib/librte_pmd_e1000/igb_rxtx.c b/lib/librte_pmd_e1000/igb_rxtx.c index da33171..ab0ff01 100644 --- a/lib/librte_pmd_e1000/igb_rxtx.c +++ b/lib/librte_pmd_e1000/igb_rxtx.c @@ -84,9 +84,7 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) } #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - (uint64_t) ((mb)->buf_physaddr + \ - (uint64_t) ((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off) #define RTE_MBUF_DATA_DMA_ADDR_DEFAULT(mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM) @@ -742,8 +740,8 @@ eth_igb_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ pkt_len = (uint16_t) (rte_le_to_cpu_16(rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; - rte_packet_prefetch(rxm->data); + rxm->data_off = RTE_PKTMBUF_HEADROOM; + rte_packet_prefetch((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs = 1; rxm->next = NULL; rxm->pkt_len = pkt_len; @@ -919,7 +917,7 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ data_len = rte_le_to_cpu_16(rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; + rxm->data_off = RTE_PKTMBUF_HEADROOM; /* * If this is the first buffer of the received packet, @@ -1001,7 +999,8 @@ eth_igb_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, first_seg->ol_flags = pkt_flags; /* Prefetch data of first segment, if configured to do so. */ - rte_packet_prefetch(first_seg->data); + rte_packet_prefetch((char *)first_seg->buf_addr + + first_seg->data_off); /* * Store the mbuf address into the next entry of the array diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c index 6cb1640..3d3316d 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c @@ -988,7 +988,7 @@ ixgbe_rx_alloc_bufs(struct igb_rx_queue *rxq) mb = rxep[i].mbuf; rte_mbuf_refcnt_set(mb, 1); mb->next = NULL; - mb->data = (char *)mb->buf_addr + RTE_PKTMBUF_HEADROOM; + mb->data_off = RTE_PKTMBUF_HEADROOM; mb->nb_segs = 1; mb->in_port = rxq->port_id; @@ -1239,8 +1239,8 @@ ixgbe_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ pkt_len = (uint16_t) (rte_le_to_cpu_16(rxd.wb.upper.length) - rxq->crc_len); - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; - rte_packet_prefetch(rxm->data); + rxm->data_off = RTE_PKTMBUF_HEADROOM; + rte_packet_prefetch((char *)rxm->buf_addr + rxm->data_off); rxm->nb_segs = 1; rxm->next = NULL; rxm->pkt_len = pkt_len; @@ -1423,7 +1423,7 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, */ data_len = rte_le_to_cpu_16(rxd.wb.upper.length); rxm->data_len = data_len; - rxm->data = (char*) rxm->buf_addr + RTE_PKTMBUF_HEADROOM; + rxm->data_off = RTE_PKTMBUF_HEADROOM; /* * If this is the first buffer of the received packet, @@ -1515,7 +1515,8 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, } /* Prefetch data of first segment, if configured to do so. */ - rte_packet_prefetch(first_seg->data); + rte_packet_prefetch((char *)first_seg->buf_addr + + first_seg->data_off); /* * Store the mbuf address into the next entry of the array @@ -3084,7 +3085,7 @@ ixgbe_alloc_rx_queue_mbufs(struct igb_rx_queue *rxq) rte_mbuf_refcnt_set(mbuf, 1); mbuf->next = NULL; - mbuf->data = (char *)mbuf->buf_addr + RTE_PKTMBUF_HEADROOM; + mbuf->data_off = RTE_PKTMBUF_HEADROOM; mbuf->nb_segs = 1; mbuf->in_port = rxq->port_id; diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.h b/lib/librte_pmd_ixgbe/ixgbe_rxtx.h index e32a417..66e741f 100644 --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.h +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.h @@ -45,8 +45,7 @@ #endif #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off) #define RTE_MBUF_DATA_DMA_ADDR_DEFAULT(mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM) diff --git a/lib/librte_pmd_virtio/virtio_rxtx.c b/lib/librte_pmd_virtio/virtio_rxtx.c index 7deaeb6..ac1b7e0 100644 --- a/lib/librte_pmd_virtio/virtio_rxtx.c +++ b/lib/librte_pmd_virtio/virtio_rxtx.c @@ -268,7 +268,7 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) continue; } rxm->in_port = rxvq->port_id; - rxm->data = (char *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM; + rxm->data_off = RTE_PKTMBUF_HEADROOM; rxm->nb_segs = 1; rxm->next = NULL; rxm->pkt_len = (uint32_t)(len[i] - sizeof(struct virtio_net_hdr)); diff --git a/lib/librte_pmd_virtio/virtqueue.h b/lib/librte_pmd_virtio/virtqueue.h index 210944e..11f908c 100644 --- a/lib/librte_pmd_virtio/virtqueue.h +++ b/lib/librte_pmd_virtio/virtqueue.h @@ -59,8 +59,7 @@ #define VIRTQUEUE_MAX_NAME_SZ 32 #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off) #define VTNET_SQ_RQ_QUEUE_IDX 0 #define VTNET_SQ_TQ_QUEUE_IDX 1 @@ -363,7 +362,7 @@ virtqueue_dequeue_burst_rx(struct virtqueue *vq, struct rte_mbuf **rx_pkts, uint break; } rte_prefetch0(cookie); - rte_packet_prefetch(cookie->data); + rte_packet_prefetch((char *)cookie->buf_addr + cookie->data_off); rx_pkts[i] = cookie; vq->vq_used_cons_idx++; vq_ring_free_chain(vq, desc_idx); diff --git a/lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c b/lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c index 60f26ba..b5450b2 100644 --- a/lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c +++ b/lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c @@ -80,8 +80,7 @@ #define RTE_MBUF_DATA_DMA_ADDR(mb) \ - (uint64_t) ((mb)->buf_physaddr + (uint64_t)((char *)((mb)->data) - \ - (char *)(mb)->buf_addr)) + (uint64_t) ((mb)->buf_physaddr + (mb)->data_off) #define RTE_MBUF_DATA_DMA_ADDR_DEFAULT(mb) \ (uint64_t) ((mb)->buf_physaddr + RTE_PKTMBUF_HEADROOM) @@ -551,7 +550,7 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rxm->data_len = (uint16_t)rcd->len; rxm->in_port = rxq->port_id; rxm->vlan_macip.f.vlan_tci = 0; - rxm->data = (char *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM; + rxm->data_off = RTE_PKTMBUF_HEADROOM; rx_pkts[nb_rx++] = rxm; -- 1.9.2