From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id D5B88B523 for ; Sun, 15 Feb 2015 05:07:27 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP; 14 Feb 2015 20:03:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,579,1418112000"; d="scan'208";a="527723482" Received: from pgsmsx107.gar.corp.intel.com ([10.221.44.105]) by orsmga003.jf.intel.com with ESMTP; 14 Feb 2015 19:59:01 -0800 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by PGSMSX107.gar.corp.intel.com (10.221.44.105) with Microsoft SMTP Server (TLS) id 14.3.195.1; Sun, 15 Feb 2015 12:07:23 +0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.192]) by shsmsx102.ccr.corp.intel.com ([169.254.2.62]) with mapi id 14.03.0195.001; Sun, 15 Feb 2015 12:07:22 +0800 From: "Liu, Jijiang" To: Stephen Hemminger Thread-Topic: [dpdk-dev] [PATCH 1/4] xen: allow choosing dom0 support at runtime Thread-Index: AdBI04tcJbZDeS+eTKiYvBJ1V+grSQ== Date: Sun, 15 Feb 2015 04:07:21 +0000 Message-ID: <1ED644BD7E0A5F4091CF203DAFB8E4CC01DDC7CB@SHSMSX101.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" , Stephen Hemminger Subject: Re: [dpdk-dev] [PATCH 1/4] xen: allow choosing dom0 support at runtime X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Feb 2015 04:07:28 -0000 Hi Stephen, What do you mean ' allow choosing dom0 support at runtime'? If you mean user can choose DPDK to run Xen Dom0 or not on DOM0 by a runti= me flag, I don't think your change can achieve this goal. Thanks Jijiang Liu > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen Hemminger > Sent: Sunday, February 15, 2015 2:07 AM > To: dev@dpdk.org > Cc: Stephen Hemminger > Subject: [dpdk-dev] [PATCH 1/4] xen: allow choosing dom0 support at runti= me >=20 > The previous code would only allow building library and application so th= at it > ran on Xen DOM0 or not on DOM0. This changes that to a runtime flag. >=20 > Signed-off-by: Stephen Hemminger > --- > lib/librte_eal/common/include/rte_memory.h | 4 +++ > lib/librte_eal/linuxapp/eal/eal_memory.c | 7 ++++ > lib/librte_ether/rte_ethdev.c | 22 ++++++++++++ > lib/librte_ether/rte_ethdev.h | 23 ++++++++++++ > lib/librte_mempool/rte_mempool.c | 26 +++++++------- > lib/librte_pmd_e1000/em_rxtx.c | 30 +++------------- > lib/librte_pmd_e1000/igb_rxtx.c | 52 +++++++++---------------= --- > lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 58 +++++++++---------------= ------ > 8 files changed, 108 insertions(+), 114 deletions(-) >=20 > diff --git a/lib/librte_eal/common/include/rte_memory.h > b/lib/librte_eal/common/include/rte_memory.h > index 7f8103f..ab6c1ff 100644 > --- a/lib/librte_eal/common/include/rte_memory.h > +++ b/lib/librte_eal/common/include/rte_memory.h > @@ -176,6 +176,10 @@ unsigned rte_memory_get_nchannel(void); unsigned > rte_memory_get_nrank(void); >=20 > #ifdef RTE_LIBRTE_XEN_DOM0 > + > +/**< Internal use only - should DOM0 memory mapping be used */ extern > +int is_xen_dom0_supported(void); > + > /** > * Return the physical address of elt, which is an element of the pool m= p. > * > diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c > b/lib/librte_eal/linuxapp/eal/eal_memory.c > index a67a1b0..4afda2a 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_memory.c > +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c > @@ -98,6 +98,13 @@ > #include "eal_filesystem.h" > #include "eal_hugepages.h" >=20 > +#ifdef RTE_LIBRTE_XEN_DOM0 > +int is_xen_dom0_supported(void) > +{ > + return internal_config.xen_dom0_support; } #endif > + > /** > * @file > * Huge page mapping under linux > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.= c index > ea3a1fb..457e0bc 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -2825,6 +2825,27 @@ _rte_eth_dev_callback_process(struct rte_eth_dev > *dev, > } > rte_spinlock_unlock(&rte_eth_dev_cb_lock); > } > + > +const struct rte_memzone * > +rte_eth_dma_zone_reserve(const struct rte_eth_dev *dev, const char > *ring_name, > + uint16_t queue_id, size_t size, unsigned align, > + int socket_id) > +{ > + char z_name[RTE_MEMZONE_NAMESIZE]; > + const struct rte_memzone *mz; > + > + snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d", > + dev->driver->pci_drv.name, ring_name, > + dev->data->port_id, queue_id); > + > + mz =3D rte_memzone_lookup(z_name); > + if (mz) > + return mz; > + > + return rte_memzone_reserve_bounded(z_name, size, > + socket_id, 0, align, > RTE_PGSIZE_2M); } > + > #ifdef RTE_NIC_BYPASS > int rte_eth_dev_bypass_init(uint8_t port_id) { @@ -3003,6 +3024,7 @@ > rte_eth_dev_bypass_wd_reset(uint8_t port_id) > (*dev->dev_ops->bypass_wd_reset)(dev); > return 0; > } > + > #endif >=20 > int > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.= h index > 1200c1c..747acb5 100644 > --- a/lib/librte_ether/rte_ethdev.h > +++ b/lib/librte_ether/rte_ethdev.h > @@ -3664,6 +3664,29 @@ int rte_eth_dev_filter_supported(uint8_t port_id, > enum rte_filter_type filter_ty int rte_eth_dev_filter_ctrl(uint8_t port_= id, > enum rte_filter_type filter_type, > enum rte_filter_op filter_op, void *arg); >=20 > +/** > + * Create memzone for HW rings. > + * malloc can't be used as the physical address is needed. > + * If the memzone is already created, then this function returns a ptr > + * to the old one. > + * > + * @param eth_dev > + * The *eth_dev* pointer is the address of the *rte_eth_dev* structure > + * @param name > + * The name of the memory zone > + * @param queue_id > + * The index of the queue to add to name > + * @param size > + * The sizeof of the memory area > + * @param align > + * Alignment for resulting memzone. Must be a power of 2. > + * @param socket_id > + * The *socket_id* argument is the socket identifier in case of NUMA. > + */ > +const struct rte_memzone * > +rte_eth_dma_zone_reserve(const struct rte_eth_dev *eth_dev, const char > *name, > + uint16_t queue_id, size_t size, > + unsigned align, int socket_id); > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_mempool/rte_mempool.c > b/lib/librte_mempool/rte_mempool.c > index 4cf6c25..5056a4f 100644 > --- a/lib/librte_mempool/rte_mempool.c > +++ b/lib/librte_mempool/rte_mempool.c > @@ -372,19 +372,21 @@ rte_mempool_create(const char *name, unsigned n, > unsigned elt_size, > int socket_id, unsigned flags) > { > #ifdef RTE_LIBRTE_XEN_DOM0 > - return (rte_dom0_mempool_create(name, n, elt_size, > - cache_size, private_data_size, > - mp_init, mp_init_arg, > - obj_init, obj_init_arg, > - socket_id, flags)); > -#else > - return (rte_mempool_xmem_create(name, n, elt_size, > - cache_size, private_data_size, > - mp_init, mp_init_arg, > - obj_init, obj_init_arg, > - socket_id, flags, > - NULL, NULL, MEMPOOL_PG_NUM_DEFAULT, > MEMPOOL_PG_SHIFT_MAX)); > + if (is_xen_dom0_supported()) > + return (rte_dom0_mempool_create(name, n, elt_size, > + cache_size, private_data_size, > + mp_init, mp_init_arg, > + obj_init, obj_init_arg, > + socket_id, flags)); > + else > #endif > + return (rte_mempool_xmem_create(name, n, elt_size, > + cache_size, private_data_size, > + mp_init, mp_init_arg, > + obj_init, obj_init_arg, > + socket_id, flags, > + NULL, NULL, > MEMPOOL_PG_NUM_DEFAULT, > + MEMPOOL_PG_SHIFT_MAX)); > } >=20 > /* > diff --git a/lib/librte_pmd_e1000/em_rxtx.c b/lib/librte_pmd_e1000/em_rxt= x.c > index aa0b88c..9e09cfa 100644 > --- a/lib/librte_pmd_e1000/em_rxtx.c > +++ b/lib/librte_pmd_e1000/em_rxtx.c > @@ -1104,28 +1104,6 @@ eth_em_recv_scattered_pkts(void *rx_queue, > struct rte_mbuf **rx_pkts, > #define EM_MAX_BUF_SIZE 16384 > #define EM_RCTL_FLXBUF_STEP 1024 >=20 > -static const struct rte_memzone * > -ring_dma_zone_reserve(struct rte_eth_dev *dev, const char *ring_name, > - uint16_t queue_id, uint32_t ring_size, int socket_id) > -{ > - const struct rte_memzone *mz; > - char z_name[RTE_MEMZONE_NAMESIZE]; > - > - snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d", > - dev->driver->pci_drv.name, ring_name, dev->data->port_id, > - queue_id); > - > - if ((mz =3D rte_memzone_lookup(z_name)) !=3D 0) > - return (mz); > - > -#ifdef RTE_LIBRTE_XEN_DOM0 > - return rte_memzone_reserve_bounded(z_name, ring_size, > - socket_id, 0, RTE_CACHE_LINE_SIZE, RTE_PGSIZE_2M); > -#else > - return rte_memzone_reserve(z_name, ring_size, socket_id, 0); > -#endif > -} > - > static void > em_tx_queue_release_mbufs(struct em_tx_queue *txq) { @@ -1273,8 > +1251,8 @@ eth_em_tx_queue_setup(struct rte_eth_dev *dev, > * resizing in later calls to the queue setup function. > */ > tsize =3D sizeof (txq->tx_ring[0]) * EM_MAX_RING_DESC; > - if ((tz =3D ring_dma_zone_reserve(dev, "tx_ring", queue_idx, tsize, > - socket_id)) =3D=3D NULL) > + if ((tz =3D rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx, tsize, > + RTE_CACHE_LINE_SIZE, socket_id)) > =3D=3D NULL) > return (-ENOMEM); >=20 > /* Allocate the tx queue data structure. */ @@ -1400,8 +1378,8 @@ > eth_em_rx_queue_setup(struct rte_eth_dev *dev, >=20 > /* Allocate RX ring for max possible mumber of hardware descriptors. > */ > rsize =3D sizeof (rxq->rx_ring[0]) * EM_MAX_RING_DESC; > - if ((rz =3D ring_dma_zone_reserve(dev, "rx_ring", queue_idx, rsize, > - socket_id)) =3D=3D NULL) > + if ((rz =3D rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx, rsize, > + RTE_CACHE_LINE_SIZE, socket_id)) > =3D=3D NULL) > return (-ENOMEM); >=20 > /* Allocate the RX queue data structure. */ diff --git > a/lib/librte_pmd_e1000/igb_rxtx.c b/lib/librte_pmd_e1000/igb_rxtx.c index > 5c394a9..d36469b 100644 > --- a/lib/librte_pmd_e1000/igb_rxtx.c > +++ b/lib/librte_pmd_e1000/igb_rxtx.c > @@ -1109,29 +1109,6 @@ eth_igb_recv_scattered_pkts(void *rx_queue, > struct rte_mbuf **rx_pkts, #define IGB_MIN_RING_DESC 32 #define > IGB_MAX_RING_DESC 4096 >=20 > -static const struct rte_memzone * > -ring_dma_zone_reserve(struct rte_eth_dev *dev, const char *ring_name, > - uint16_t queue_id, uint32_t ring_size, int socket_id) > -{ > - char z_name[RTE_MEMZONE_NAMESIZE]; > - const struct rte_memzone *mz; > - > - snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d", > - dev->driver->pci_drv.name, ring_name, > - dev->data->port_id, queue_id); > - mz =3D rte_memzone_lookup(z_name); > - if (mz) > - return mz; > - > -#ifdef RTE_LIBRTE_XEN_DOM0 > - return rte_memzone_reserve_bounded(z_name, ring_size, > - socket_id, 0, IGB_ALIGN, RTE_PGSIZE_2M); > -#else > - return rte_memzone_reserve_aligned(z_name, ring_size, > - socket_id, 0, IGB_ALIGN); > -#endif > -} > - > static void > igb_tx_queue_release_mbufs(struct igb_tx_queue *txq) { @@ -1265,8 > +1242,8 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, > * resizing in later calls to the queue setup function. > */ > size =3D sizeof(union e1000_adv_tx_desc) * IGB_MAX_RING_DESC; > - tz =3D ring_dma_zone_reserve(dev, "tx_ring", queue_idx, > - size, socket_id); > + tz =3D rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx, size, > + IGB_ALIGN, socket_id); > if (tz =3D=3D NULL) { > igb_tx_queue_release(txq); > return (-ENOMEM); > @@ -1284,12 +1261,14 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev, > txq->port_id =3D dev->data->port_id; >=20 > txq->tdt_reg_addr =3D E1000_PCI_REG_ADDR(hw, E1000_TDT(txq- > >reg_idx)); -#ifndef RTE_LIBRTE_XEN_DOM0 > - txq->tx_ring_phys_addr =3D (uint64_t) tz->phys_addr; > -#else > - txq->tx_ring_phys_addr =3D rte_mem_phy2mch(tz->memseg_id, tz- > >phys_addr); > +#ifdef RTE_LIBRTE_XEN_DOM0 > + if (is_xen_dom0_supported()) > + txq->tx_ring_phys_addr =3D rte_mem_phy2mch(tz->memseg_id, > tz->phys_addr); > + else > #endif > - txq->tx_ring =3D (union e1000_adv_tx_desc *) tz->addr; > + txq->tx_ring_phys_addr =3D (uint64_t) tz->phys_addr; > + > + txq->tx_ring =3D (union e1000_adv_tx_desc *) tz->addr; > /* Allocate software ring */ > txq->sw_ring =3D rte_zmalloc("txq->sw_ring", > sizeof(struct igb_tx_entry) * nb_desc, @@ - > 1414,18 +1393,21 @@ eth_igb_rx_queue_setup(struct rte_eth_dev *dev, > * resizing in later calls to the queue setup function. > */ > size =3D sizeof(union e1000_adv_rx_desc) * IGB_MAX_RING_DESC; > - rz =3D ring_dma_zone_reserve(dev, "rx_ring", queue_idx, size, socket_id= ); > + rz =3D rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx, size, > + IGB_ALIGN, socket_id); > if (rz =3D=3D NULL) { > igb_rx_queue_release(rxq); > return (-ENOMEM); > } > rxq->rdt_reg_addr =3D E1000_PCI_REG_ADDR(hw, E1000_RDT(rxq- > >reg_idx)); > rxq->rdh_reg_addr =3D E1000_PCI_REG_ADDR(hw, E1000_RDH(rxq- > >reg_idx)); -#ifndef RTE_LIBRTE_XEN_DOM0 > - rxq->rx_ring_phys_addr =3D (uint64_t) rz->phys_addr; > -#else > - rxq->rx_ring_phys_addr =3D rte_mem_phy2mch(rz->memseg_id, rz- > >phys_addr); > +#ifdef RTE_LIBRTE_XEN_DOM0 > + if (is_xen_dom0_supported()) > + rxq->rx_ring_phys_addr =3D > + rte_mem_phy2mch(rz->memseg_id, rz->phys_addr); > + else > #endif > + rxq->rx_ring_phys_addr =3D (uint64_t) rz->phys_addr; > rxq->rx_ring =3D (union e1000_adv_rx_desc *) rz->addr; >=20 > /* Allocate software ring. */ > diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > index e6766b3..303144d 100644 > --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > @@ -1656,35 +1656,6 @@ ixgbe_recv_scattered_pkts(void *rx_queue, struct > rte_mbuf **rx_pkts, #define IXGBE_MIN_RING_DESC 32 #define > IXGBE_MAX_RING_DESC 4096 >=20 > -/* > - * Create memzone for HW rings. malloc can't be used as the physical add= ress > is > - * needed. If the memzone is already created, then this function returns= a ptr > - * to the old one. > - */ > -static const struct rte_memzone * > -ring_dma_zone_reserve(struct rte_eth_dev *dev, const char *ring_name, > - uint16_t queue_id, uint32_t ring_size, int socket_id) > -{ > - char z_name[RTE_MEMZONE_NAMESIZE]; > - const struct rte_memzone *mz; > - > - snprintf(z_name, sizeof(z_name), "%s_%s_%d_%d", > - dev->driver->pci_drv.name, ring_name, > - dev->data->port_id, queue_id); > - > - mz =3D rte_memzone_lookup(z_name); > - if (mz) > - return mz; > - > -#ifdef RTE_LIBRTE_XEN_DOM0 > - return rte_memzone_reserve_bounded(z_name, ring_size, > - socket_id, 0, IXGBE_ALIGN, RTE_PGSIZE_2M); > -#else > - return rte_memzone_reserve_aligned(z_name, ring_size, > - socket_id, 0, IXGBE_ALIGN); > -#endif > -} > - > static void > ixgbe_tx_queue_release_mbufs(struct igb_tx_queue *txq) { @@ -1920,9 > +1891,9 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev *dev, > * handle the maximum ring size is allocated in order to allow for > * resizing in later calls to the queue setup function. > */ > - tz =3D ring_dma_zone_reserve(dev, "tx_ring", queue_idx, > + tz =3D rte_eth_dma_zone_reserve(dev, "tx_ring", queue_idx, > sizeof(union ixgbe_adv_tx_desc) * > IXGBE_MAX_RING_DESC, > - socket_id); > + IXGBE_ALIGN, socket_id); > if (tz =3D=3D NULL) { > ixgbe_tx_queue_release(txq); > return (-ENOMEM); > @@ -1950,11 +1921,14 @@ ixgbe_dev_tx_queue_setup(struct rte_eth_dev > *dev, > txq->tdt_reg_addr =3D IXGBE_PCI_REG_ADDR(hw, > IXGBE_VFTDT(queue_idx)); > else > txq->tdt_reg_addr =3D IXGBE_PCI_REG_ADDR(hw, > IXGBE_TDT(txq->reg_idx)); > -#ifndef RTE_LIBRTE_XEN_DOM0 > - txq->tx_ring_phys_addr =3D (uint64_t) tz->phys_addr; > -#else > - txq->tx_ring_phys_addr =3D rte_mem_phy2mch(tz->memseg_id, tz- > >phys_addr); > + > +#ifdef RTE_LIBRTE_XEN_DOM0 > + if (is_xen_dom0_supported()) > + txq->tx_ring_phys_addr =3D > + rte_mem_phy2mch(tz->memseg_id, tz->phys_addr); > + else > #endif > + txq->tx_ring_phys_addr =3D (uint64_t) tz->phys_addr; > txq->tx_ring =3D (union ixgbe_adv_tx_desc *) tz->addr; >=20 > /* Allocate software ring */ > @@ -2195,8 +2169,8 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev *dev, > * handle the maximum ring size is allocated in order to allow for > * resizing in later calls to the queue setup function. > */ > - rz =3D ring_dma_zone_reserve(dev, "rx_ring", queue_idx, > - RX_RING_SZ, socket_id); > + rz =3D rte_eth_dma_zone_reserve(dev, "rx_ring", queue_idx, > + RX_RING_SZ, IXGBE_ALIGN, socket_id); > if (rz =3D=3D NULL) { > ixgbe_rx_queue_release(rxq); > return (-ENOMEM); > @@ -2223,11 +2197,13 @@ ixgbe_dev_rx_queue_setup(struct rte_eth_dev > *dev, > rxq->rdh_reg_addr =3D > IXGBE_PCI_REG_ADDR(hw, IXGBE_RDH(rxq->reg_idx)); > } > -#ifndef RTE_LIBRTE_XEN_DOM0 > - rxq->rx_ring_phys_addr =3D (uint64_t) rz->phys_addr; > -#else > - rxq->rx_ring_phys_addr =3D rte_mem_phy2mch(rz->memseg_id, rz- > >phys_addr); > +#ifdef RTE_LIBRTE_XEN_DOM0 > + if (is_xen_dom0_supported()) > + rxq->rx_ring_phys_addr =3D > + rte_mem_phy2mch(rz->memseg_id, rz->phys_addr); > + else > #endif > + rxq->rx_ring_phys_addr =3D (uint64_t) rz->phys_addr; > rxq->rx_ring =3D (union ixgbe_adv_rx_desc *) rz->addr; >=20 > /* > -- > 2.1.4