[dpdk-dev] [PATCH 0/7] virtio-net support on ppc64

[dpdk-dev] [PATCH 0/7] virtio-net support on ppc64

[Olivier Matz]

This patchset allows virtio-net pmd to run on ppc64 processors.
The main thing that as missing was the support of ioports in EAL.
It also fixes some endianess issues in PCI config accesses in
legacy mode.

This is validated with test-pmd:

=== HOST

mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 128 > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
qemu-system-ppc64le --enable-kvm -m 1G -mem-path "/mnt/huge" -cpu host -smp 3 \
  -serial telnet::58028,server,nowait -serial null \
  -device ne2k_pci,mac=de:ad:de:01:02:03,netdev=user.0,addr=03 \
  -netdev user,id=user.0,hostfwd=tcp::49680-:22 \
  -netdev type=tap,id=vhostnet0,script=no,vhost=on,queues=8 \
  -device virtio-net-pci,netdev=vhostnet0,ioeventfd=on,mq=on,vectors=17 \
  -hda /path/to/ubuntu-14.04-ppc64el.qcow2 \
  -vga none -display none
ip l set tap0 up
ip a a 1.1.1.1/24 dev tap0
# start guest, then:
ping 1.1.1.2

=== GUEST

cd dpdk.org/
make config T=ppc_64-power8-linuxapp-gcc
make
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
modprobe uio_pci_generic
python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:00.0
./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 -i --port-topology=chained
EAL: Detected 3 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:00:00.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
Interactive-mode selected
Configuring Port 0 (socket 0)
Port 0: 12:00:54:52:56:34
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd> set fwd icmpecho
Set icmpecho packet forwarding mode
testpmd> set verbose 1
Change verbose level from 0 to 1
testpmd> start
icmp_echo_config_setup fwd_cores=1 fwd_ports=1 fwd_streams=1
  core=0:
  stream=0 port=0 rxq=0 txq=0
  icmpecho packet forwarding - CRC stripping disabled - packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=1
  RX queues=1 - RX desc=128 - RX free threshold=0
  RX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=0
  TX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX RS bit threshold=0 - TXQ flags=0xf00
testpmd>
Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=12:00:54:52:56:34 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=1

Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=12:00:54:52:56:34 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=2




David Marchand (1):
  virtio: fix pci accesses for ppc64 in legacy mode

Olivier Matz (6):
  eal: fix typos in ioport API doxygen comments
  eal/linux: only call iopl on x86
  eal/linux: remove invalid comment
  eal/linux: split function parsing pci resources in sysfs
  eal/linux: mmap ioports on ppc64
  config: enable virtio-net pmd for ppc64

 config/defconfig_ppc_64-power8-linuxapp-gcc |   2 +-
 drivers/net/virtio/virtio_pci.c             |  44 ++++++++++
 lib/librte_eal/common/include/rte_pci.h     |  13 ++-
 lib/librte_eal/linuxapp/eal/eal.c           |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c       |  52 +++++++-----
 lib/librte_eal/linuxapp/eal/eal_pci_init.h  |   6 ++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c   | 121 ++++++++++++++++++++++------
 7 files changed, 191 insertions(+), 53 deletions(-)

-- 
2.8.0.rc3

[dpdk-dev] [PATCH 1/7] eal: fix typos in ioport API doxygen comments

[Olivier Matz]

Fix some typos and add missing comments related to ioports API in
rte_pci.h.

Fixes: 756ce64b1 ("eal: introduce PCI ioport API")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/common/include/rte_pci.h | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..fd049d1 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -521,12 +521,13 @@ struct rte_pci_ioport {
 };
 
 /**
- * Initialises a rte_pci_ioport object for a pci device io resource.
+ * Initialize a rte_pci_ioport object for a pci device io resource.
+ *
  * This object is then used to gain access to those io resources (see below).
  *
  * @param dev
- *   A pointer to a rte_pci_device structure describing the device.
- *   to use
+ *   A pointer to a rte_pci_device structure describing the device
+ *   to use.
  * @param bar
  *   Index of the io pci resource we want to access.
  * @param p
@@ -542,6 +543,8 @@ int rte_eal_pci_ioport_map(struct rte_pci_device *dev, int bar,
  *
  * @param p
  *   The rte_pci_ioport object to be uninitialized.
+ * @return
+ *  0 on success, negative on error.
  */
 int rte_eal_pci_ioport_unmap(struct rte_pci_ioport *p);
 
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 2/7] eal/linux: only call iopl on x86

[Olivier Matz]

>From iopl(2) man page: "This call is mostly for the x86 architecture. On
many other architectures it does not exist or will always return an
error".

This patch removes the call to iopl() in rte_eal_iopl_init() for
architectures other than x86, and always return 0 (success). This was
already done for ARM in
commit 0291476ae364 ("eal/linux: never check iopl for arm")

Next patches will introduce the support of memory mapped IO resources
for architectures != x86.

On BSD, there is nothing to do as open("/dev/io") already does the
proper thing. See man IO(4).

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index 8aafd51..bba8fea 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -715,12 +715,8 @@ rte_eal_iopl_init(void)
 #if defined(RTE_ARCH_X86)
 	if (iopl(3) != 0)
 		return -1;
-	return 0;
-#elif defined(RTE_ARCH_ARM) || defined(RTE_ARCH_ARM64)
-	return 0; /* iopl syscall not supported for ARM/ARM64 */
-#else
-	return -1;
 #endif
+	return 0;
 }
 
 /* Launch threads, called at application init(). */
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 3/7] eal/linux: remove invalid comment

[Olivier Matz]

In a previous commit, the file used to map the PCI resources changed
from "/dev/uio<x>" to "/sys/bus/pci/devices/<busaddr>/resource", making
the comment wrong. Remove it.

Fixes: 9e67561acd1a ("eal/linux: mmap uio resources using resourceX files")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..ac449c5 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -309,7 +309,7 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 		struct mapped_pci_resource *uio_res, int map_idx)
 {
 	int fd;
-	char devname[PATH_MAX]; /* contains the /dev/uioX */
+	char devname[PATH_MAX];
 	void *mapaddr;
 	struct rte_pci_addr *loc;
 	struct pci_map *maps;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 4/7] eal/linux: split function parsing pci resources in sysfs

[Olivier Matz]

Split pci_parse_sysfs_resource() and introduce
pci_parse_one_sysfs_resource() that parses one line of sysfs resource
file.

This new function will be exported and used in next commits when
mapping the ioports resources.

No functional change.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal_pci.c | 50 ++++++++++++++++++++++-------------
 1 file changed, 32 insertions(+), 18 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..1a93725 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -190,12 +190,13 @@ pci_find_max_end_va(void)
 	return RTE_PTR_ADD(last->addr, last->len);
 }
 
-/* parse the "resource" sysfs file */
+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
 static int
-pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+	uint64_t *end_addr, uint64_t *flags)
 {
-	FILE *f;
-	char buf[BUFSIZ];
 	union pci_resource_info {
 		struct {
 			char *phys_addr;
@@ -204,6 +205,31 @@ pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
 		};
 		char *ptrs[PCI_RESOURCE_FMT_NVAL];
 	} res_info;
+
+	if (rte_strsplit(line, len, res_info.ptrs, 3, ' ') != 3) {
+		RTE_LOG(ERR, EAL,
+			"%s(): bad resource format\n", __func__);
+		return -1;
+	}
+	errno = 0;
+	*phys_addr = strtoull(res_info.phys_addr, NULL, 16);
+	*end_addr = strtoull(res_info.end_addr, NULL, 16);
+	*flags = strtoull(res_info.flags, NULL, 16);
+	if (errno != 0) {
+		RTE_LOG(ERR, EAL,
+			"%s(): bad resource format\n", __func__);
+		return -1;
+	}
+
+	return 0;
+}
+
+/* parse the "resource" sysfs file */
+static int
+pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+{
+	FILE *f;
+	char buf[BUFSIZ];
 	int i;
 	uint64_t phys_addr, end_addr, flags;
 
@@ -220,21 +246,9 @@ pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
 				"%s(): cannot read resource\n", __func__);
 			goto error;
 		}
-
-		if (rte_strsplit(buf, sizeof(buf), res_info.ptrs, 3, ' ') != 3) {
-			RTE_LOG(ERR, EAL,
-				"%s(): bad resource format\n", __func__);
+		if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
+				&end_addr, &flags) < 0)
 			goto error;
-		}
-		errno = 0;
-		phys_addr = strtoull(res_info.phys_addr, NULL, 16);
-		end_addr = strtoull(res_info.end_addr, NULL, 16);
-		flags = strtoull(res_info.flags, NULL, 16);
-		if (errno != 0) {
-			RTE_LOG(ERR, EAL,
-				"%s(): bad resource format\n", __func__);
-			goto error;
-		}
 
 		if (flags & IORESOURCE_MEM) {
 			dev->mem_resource[i].phys_addr = phys_addr;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

On PPC64, the ioports are mapped in memory. Implement the missing part
of ioport API for PPC64 when using uio. This may also work on other
architectures but it has not been tested.

Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/common/include/rte_pci.h    |   4 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c      |   4 +-
 lib/librte_eal/linuxapp/eal/eal_pci_init.h |   6 ++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c  | 119 +++++++++++++++++++++++------
 4 files changed, 107 insertions(+), 26 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index fd049d1..4657396 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -105,7 +105,8 @@ extern struct pci_device_list pci_device_list; /**< Global list of PCI devices.
 /** Nb. of values in PCI resource format. */
 #define PCI_RESOURCE_FMT_NVAL 3
 
-/** IO resource type: memory address space */
+/** IO resource type: */
+#define IORESOURCE_IO         0x00000100
 #define IORESOURCE_MEM        0x00000200
 
 /**
@@ -518,6 +519,7 @@ int rte_eal_pci_write_config(const struct rte_pci_device *device,
 struct rte_pci_ioport {
 	struct rte_pci_device *dev;
 	uint64_t base;
+	uint64_t len; /* only filled for memory mapped ports */
 };
 
 /**
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 1a93725..f1ea52b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -191,9 +191,9 @@ pci_find_max_end_va(void)
 }
 
 /* parse one line of the "resource" sysfs file (note that the 'line'
- * string is modified)
+ * string is modified
  */
-static int
+int
 pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
 	uint64_t *end_addr, uint64_t *flags)
 {
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 7011753..fbe34c6 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -42,6 +42,12 @@
 extern void *pci_map_addr;
 void *pci_find_max_end_va(void);
 
+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
+int pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+	uint64_t *end_addr, uint64_t *flags);
+
 int pci_uio_alloc_resource(struct rte_pci_device *dev,
 		struct mapped_pci_resource **uio_res);
 void pci_uio_free_resource(struct rte_pci_device *dev,
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index ac449c5..077ad96 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -35,6 +35,7 @@
 #include <unistd.h>
 #include <fcntl.h>
 #include <dirent.h>
+#include <inttypes.h>
 #include <sys/stat.h>
 #include <sys/mman.h>
 #include <linux/pci_regs.h>
@@ -368,11 +369,11 @@ error:
 	return -1;
 }
 
+#if defined(RTE_ARCH_X86)
 int
 pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
 		   struct rte_pci_ioport *p)
 {
-#if defined(RTE_ARCH_X86)
 	char dirname[PATH_MAX];
 	char filename[PATH_MAX];
 	int uio_num;
@@ -411,81 +412,153 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
 	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);
 
 	p->base = start;
+	p->len = 0;
 	return 0;
+}
 #else
-	RTE_SET_USED(dev);
-	RTE_SET_USED(bar);
-	RTE_SET_USED(p);
+int
+pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
+		   struct rte_pci_ioport *p)
+{
+	FILE *f;
+	char buf[BUFSIZ];
+	char filename[PATH_MAX];
+	uint64_t phys_addr, end_addr, flags;
+	int fd, i;
+	void *addr;
+
+	/* open and read addresses of the corresponding resource in sysfs */
+	snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
+		SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
+		 dev->addr.devid, dev->addr.function);
+	f = fopen(filename, "r");
+	if (f == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
+			strerror(errno));
+		return -1;
+	}
+	for (i = 0; i < bar + 1; i++) {
+		if (fgets(buf, sizeof(buf), f) == NULL) {
+			RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
+			goto error;
+		}
+	}
+	if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
+			&end_addr, &flags) < 0)
+		goto error;
+	if ((flags & IORESOURCE_IO) == 0) {
+		RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
+		goto error;
+	}
+	snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
+		SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
+		 dev->addr.devid, dev->addr.function, bar);
+
+	/* mmap the pci resource */
+	fd = open(filename, O_RDWR);
+	if (fd < 0) {
+		RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
+			strerror(errno));
+		goto error;
+	}
+	addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
+		MAP_SHARED, fd, 0);
+	if (addr == MAP_FAILED) {
+		RTE_LOG(ERR, EAL, "Cannot mmap IO port resource: %s\n",
+			strerror(errno));
+		goto error;
+	}
+
+	/* strangely, the base address is mmap addr + phys_addr */
+	p->base = (uintptr_t)addr + phys_addr;
+	p->len = end_addr + 1;
+	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%"PRIx64"\n", p->base);
+	fclose(f);
+
+	return 0;
+
+error:
+	fclose(f);
 	return -1;
-#endif
 }
+#endif
 
 void
 pci_uio_ioport_read(struct rte_pci_ioport *p,
 		    void *data, size_t len, off_t offset)
 {
-#if defined(RTE_ARCH_X86)
 	uint8_t *d;
 	int size;
-	unsigned short reg = p->base + offset;
+	uintptr_t reg = p->base + offset;
 
 	for (d = data; len > 0; d += size, reg += size, len -= size) {
 		if (len >= 4) {
 			size = 4;
+#if defined(RTE_ARCH_X86)
 			*(uint32_t *)d = inl(reg);
+#else
+			*(uint32_t *)d = *(volatile uint32_t *)reg;
+#endif
 		} else if (len >= 2) {
 			size = 2;
+#if defined(RTE_ARCH_X86)
 			*(uint16_t *)d = inw(reg);
+#else
+			*(uint16_t *)d = *(volatile uint16_t *)reg;
+#endif
 		} else {
 			size = 1;
+#if defined(RTE_ARCH_X86)
 			*d = inb(reg);
-		}
-	}
 #else
-	RTE_SET_USED(p);
-	RTE_SET_USED(data);
-	RTE_SET_USED(len);
-	RTE_SET_USED(offset);
+			*d = *(volatile uint8_t *)reg;
 #endif
+		}
+	}
 }
 
 void
 pci_uio_ioport_write(struct rte_pci_ioport *p,
 		     const void *data, size_t len, off_t offset)
 {
-#if defined(RTE_ARCH_X86)
 	const uint8_t *s;
 	int size;
-	unsigned short reg = p->base + offset;
+	uintptr_t reg = p->base + offset;
 
 	for (s = data; len > 0; s += size, reg += size, len -= size) {
 		if (len >= 4) {
 			size = 4;
+#if defined(RTE_ARCH_X86)
 			outl_p(*(const uint32_t *)s, reg);
+#else
+			*(volatile uint32_t *)reg = *(const uint32_t *)s;
+#endif
 		} else if (len >= 2) {
 			size = 2;
+#if defined(RTE_ARCH_X86)
 			outw_p(*(const uint16_t *)s, reg);
+#else
+			*(volatile uint16_t *)reg = *(const uint16_t *)s;
+#endif
 		} else {
 			size = 1;
+#if defined(RTE_ARCH_X86)
 			outb_p(*s, reg);
-		}
-	}
 #else
-	RTE_SET_USED(p);
-	RTE_SET_USED(data);
-	RTE_SET_USED(len);
-	RTE_SET_USED(offset);
+			*(volatile uint8_t *)reg = *s;
 #endif
+		}
+	}
 }
 
 int
 pci_uio_ioport_unmap(struct rte_pci_ioport *p)
 {
-	RTE_SET_USED(p);
 #if defined(RTE_ARCH_X86)
+	RTE_SET_USED(p);
 	/* FIXME close intr fd ? */
 	return 0;
 #else
-	return -1;
+	return munmap((void *)(uintptr_t)p->base, p->len);
 #endif
 }
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Olivier Matz]

From: David Marchand <david.marchand@6wind.com>

Although ppc supports both endianesses, qemu supposes that the cpu is
big endian and enforces this for the virtio-net stuff.

Fix PCI accesses in legacy mode. Only ppc64le is supported at the moment.

Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 drivers/net/virtio/virtio_pci.c | 44 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..bdb89fd 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -55,18 +55,62 @@
  */
 #define VIRTIO_PCI_CONFIG(hw) (((hw)->use_msix) ? 24 : 20)
 
+/*
+ * Since we are in legacy mode:
+ * http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf
+ *
+ * "Note that this is possible because while the virtio header is PCI (i.e.
+ * little) endian, the device-specific region is encoded in the native endian of
+ * the guest (where such distinction is applicable)."
+ *
+ * For powerpc which supports both, qemu supposes that cpu is big endian and
+ * enforces this for the virtio-net stuff.
+ */
+
 static void
 legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
 		       void *dst, int length)
 {
 	rte_eal_pci_ioport_read(&hw->io, dst, length,
 				VIRTIO_PCI_CONFIG(hw) + offset);
+#ifdef RTE_ARCH_PPC_64
+	switch (length) {
+	case 4:
+		*(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
+		break;
+	case 2:
+		*(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
+		break;
+	default:
+		break;
+	}
+#endif
 }
 
 static void
 legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
 			const void *src, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+	union {
+		uint32_t u32;
+		uint16_t u16;
+	} tmp;
+	switch (length) {
+	case 4:
+		tmp.u32 = *(const uint32_t *)src;
+		tmp.u32 = rte_cpu_to_be_32(tmp.u32);
+		src = &tmp.u32;
+		break;
+	case 2:
+		tmp.u16 = *(const uint16_t *)src;
+		tmp.u16 = rte_cpu_to_be_16(tmp.u16);
+		src = &tmp.u16;
+		break;
+	default:
+		break;
+	}
+#endif
 	rte_eal_pci_ioport_write(&hw->io, src, length,
 				 VIRTIO_PCI_CONFIG(hw) + offset);
 }
-- 
2.8.0.rc3

[dpdk-dev] [PATCH 7/7] config: enable virtio-net pmd for ppc64

[Olivier Matz]

Now that virtio pmd is supported on ppc, enable it.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 config/defconfig_ppc_64-power8-linuxapp-gcc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc b/config/defconfig_ppc_64-power8-linuxapp-gcc
index 9eb0cc4..bef8f49 100644
--- a/config/defconfig_ppc_64-power8-linuxapp-gcc
+++ b/config/defconfig_ppc_64-power8-linuxapp-gcc
@@ -50,7 +50,7 @@ CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
 # Will turn on them only after the successful testing on Power
 CONFIG_RTE_LIBRTE_IXGBE_PMD=n
 CONFIG_RTE_LIBRTE_I40E_PMD=n
-CONFIG_RTE_LIBRTE_VIRTIO_PMD=n
+CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
 CONFIG_RTE_LIBRTE_VMXNET3_PMD=n
 CONFIG_RTE_LIBRTE_PMD_BOND=n
 CONFIG_RTE_LIBRTE_ENIC_PMD=n
-- 
2.8.0.rc3

Re: [dpdk-dev] [PATCH 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[David Marchand]

Hello Olivier,

On Fri, May 13, 2016 at 2:50 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
> From: David Marchand <david.marchand@6wind.com>
>
> Although ppc supports both endianesses, qemu supposes that the cpu is
> big endian and enforces this for the virtio-net stuff.
>
> Fix PCI accesses in legacy mode. Only ppc64le is supported at the moment.
>
> Signed-off-by: David Marchand <david.marchand@6wind.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
>  drivers/net/virtio/virtio_pci.c | 44 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 44 insertions(+)
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 9cdca06..bdb89fd 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -55,18 +55,62 @@
>   */
>  #define VIRTIO_PCI_CONFIG(hw) (((hw)->use_msix) ? 24 : 20)
>
> +/*
> + * Since we are in legacy mode:
> + * http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf
> + *
> + * "Note that this is possible because while the virtio header is PCI (i.e.
> + * little) endian, the device-specific region is encoded in the native endian of
> + * the guest (where such distinction is applicable)."
> + *
> + * For powerpc which supports both, qemu supposes that cpu is big endian and
> + * enforces this for the virtio-net stuff.
> + */
> +
>  static void
>  legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
>                        void *dst, int length)
>  {
>         rte_eal_pci_ioport_read(&hw->io, dst, length,
>                                 VIRTIO_PCI_CONFIG(hw) + offset);
> +#ifdef RTE_ARCH_PPC_64
> +       switch (length) {
> +       case 4:
> +               *(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
> +               break;
> +       case 2:
> +               *(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
> +               break;
> +       default:
> +               break;
> +       }
> +#endif
>  }

I think that, in the original patch, I was handling lengths different
than 1, 2 and 4 ;-)
Idem for write.


-- 
David Marchand

Re: [dpdk-dev] [PATCH 5/7] eal/linux: mmap ioports on ppc64

[David Marchand]

On Fri, May 13, 2016 at 2:50 PM, Olivier Matz <olivier.matz@6wind.com> wrote:
> On PPC64, the ioports are mapped in memory. Implement the missing part
> of ioport API for PPC64 when using uio. This may also work on other
> architectures but it has not been tested.
>
> Signed-off-by: David Marchand <david.marchand@6wind.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
>  lib/librte_eal/common/include/rte_pci.h    |   4 +-
>  lib/librte_eal/linuxapp/eal/eal_pci.c      |   4 +-
>  lib/librte_eal/linuxapp/eal/eal_pci_init.h |   6 ++
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c  | 119 +++++++++++++++++++++++------
>  4 files changed, 107 insertions(+), 26 deletions(-)
>
> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
> index fd049d1..4657396 100644
> --- a/lib/librte_eal/common/include/rte_pci.h
> +++ b/lib/librte_eal/common/include/rte_pci.h
> @@ -105,7 +105,8 @@ extern struct pci_device_list pci_device_list; /**< Global list of PCI devices.
>  /** Nb. of values in PCI resource format. */
>  #define PCI_RESOURCE_FMT_NVAL 3
>
> -/** IO resource type: memory address space */
> +/** IO resource type: */
> +#define IORESOURCE_IO         0x00000100
>  #define IORESOURCE_MEM        0x00000200
>
>  /**

This could go in a linux-specific header.

> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index 1a93725..f1ea52b 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -191,9 +191,9 @@ pci_find_max_end_va(void)
>  }
>
>  /* parse one line of the "resource" sysfs file (note that the 'line'
> - * string is modified)
> + * string is modified
>   */

Garbage ?

-- 
David Marchand

Re: [dpdk-dev] [PATCH 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Olivier Matz]

Hi David,

On 05/13/2016 04:28 PM, David Marchand wrote:
>> +#ifdef RTE_ARCH_PPC_64
>> +       switch (length) {
>> +       case 4:
>> +               *(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
>> +               break;
>> +       case 2:
>> +               *(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
>> +               break;
>> +       default:
>> +               break;
>> +       }
>> +#endif
>>  }
> 
> I think that, in the original patch, I was handling lengths different
> than 1, 2 and 4 ;-)
> Idem for write.

Oh oh, spotted :)
I'll fix that in the v2.

Re: [dpdk-dev] [PATCH 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

On 05/13/2016 04:33 PM, David Marchand wrote:
>> --- a/lib/librte_eal/common/include/rte_pci.h
>> +++ b/lib/librte_eal/common/include/rte_pci.h
>> @@ -105,7 +105,8 @@ extern struct pci_device_list pci_device_list; /**< Global list of PCI devices.
>>  /** Nb. of values in PCI resource format. */
>>  #define PCI_RESOURCE_FMT_NVAL 3
>>
>> -/** IO resource type: memory address space */
>> +/** IO resource type: */
>> +#define IORESOURCE_IO         0x00000100
>>  #define IORESOURCE_MEM        0x00000200
>>
>>  /**
> 
> This could go in a linux-specific header.

Yep, I'll do a separate patch for that in v2.

> 
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> index 1a93725..f1ea52b 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
>> @@ -191,9 +191,9 @@ pci_find_max_end_va(void)
>>  }
>>
>>  /* parse one line of the "resource" sysfs file (note that the 'line'
>> - * string is modified)
>> + * string is modified
>>   */
> 
> Garbage ?

Indeed :)


Thanks for reviewing!

[dpdk-dev] [PATCH v2 0/7] virtio-net support on ppc64

[Olivier Matz]

This patchset allows virtio-net pmd to run on ppc64 processors.
The main thing that as missing was the support of ioports in EAL.
It also fixes some endianess issues in PCI config accesses in
legacy mode.

v1 -> v2:
  - fix read/write lengths management in virtio (this fixes
    the reading of mac address)
  - fix garbage in comment
  - move linux specific definition into a linux specific
    header
  - replay the test plan


This is validated with test-pmd:

=== HOST

mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 128 > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
qemu-system-ppc64le --enable-kvm -m 1G -mem-path "/mnt/huge" -cpu host -smp 3 \
  -serial telnet::58028,server,nowait -serial null \
  -device ne2k_pci,mac=de:ad:de:01:02:03,netdev=user.0,addr=03 \
  -netdev user,id=user.0,hostfwd=tcp::49680-:22 \
  -netdev type=tap,id=vhostnet0,script=no,vhost=on,queues=8 \
  -device virtio-net-pci,netdev=vhostnet0,ioeventfd=on,mq=on,vectors=17 \
  -hda /path/to/ubuntu-14.04-ppc64el.qcow2 \
  -vga none -display none
ip l set tap0 up
ip a a 1.1.1.1/24 dev tap0
# start guest, then:
ping 1.1.1.2

=== GUEST

cd dpdk.org/
make config T=ppc_64-power8-linuxapp-gcc
make
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
modprobe uio_pci_generic
python tools/dpdk_nic_bind.py -b uio_pci_generic 0000:00:00.0
./build/app/testpmd -l 0,1 --log-level 8 -- --total-num-mbufs=16384 -i --port-topology=chained
EAL: Detected 3 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:00:00.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
Interactive-mode selected
Configuring Port 0 (socket 0)
Port 0: 52:54:00:12:34:56
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd> set fwd icmpecho
Set icmpecho packet forwarding mode
testpmd> set verbose 1
Change verbose level from 0 to 1
testpmd> start
icmp_echo_config_setup fwd_cores=1 fwd_ports=1 fwd_streams=1
  core=0:
  stream=0 port=0 rxq=0 txq=0
  icmpecho packet forwarding - CRC stripping disabled - packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=1
  RX queues=1 - RX desc=128 - RX free threshold=0
  RX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=0
  TX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX RS bit threshold=0 - TXQ flags=0xf00
testpmd>
Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=52:54:00:12:34:56 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=1

Port 0 pkt-len=98 nb-segs=1
  ETH:  src=46:A9:44:17:62:F1 dst=52:54:00:12:34:56 type=0x0800
  IPV4: src=1.1.1.1 dst=1.1.1.2 proto=1 (ICMP)
  ICMP: echo request seq id=2


David Marchand (1):
  virtio: fix pci accesses for ppc64 in legacy mode

Olivier Matz (6):
  eal: fix typos in ioport API doxygen comments
  eal/linux: only call iopl on x86
  eal/linux: remove invalid comment
  eal/linux: split function parsing pci resources in sysfs
  eal/linux: mmap ioports on ppc64
  config: enable virtio-net pmd for ppc64

 config/defconfig_ppc_64-power8-linuxapp-gcc |   2 +-
 drivers/net/virtio/virtio_pci.c             |  68 ++++++++++++++++
 lib/librte_eal/common/include/rte_pci.h     |  13 +--
 lib/librte_eal/linuxapp/eal/eal.c           |   6 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c       |  52 +++++++-----
 lib/librte_eal/linuxapp/eal/eal_pci_init.h  |  10 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c   | 121 ++++++++++++++++++++++------
 7 files changed, 217 insertions(+), 55 deletions(-)

-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 1/7] eal: fix typos in ioport API doxygen comments

[Olivier Matz]

Fix some typos and add missing comments related to ioports API in
rte_pci.h.

Fixes: 756ce64b1 ("eal: introduce PCI ioport API")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/common/include/rte_pci.h | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index 8fa2712..fd049d1 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -521,12 +521,13 @@ struct rte_pci_ioport {
 };
 
 /**
- * Initialises a rte_pci_ioport object for a pci device io resource.
+ * Initialize a rte_pci_ioport object for a pci device io resource.
+ *
  * This object is then used to gain access to those io resources (see below).
  *
  * @param dev
- *   A pointer to a rte_pci_device structure describing the device.
- *   to use
+ *   A pointer to a rte_pci_device structure describing the device
+ *   to use.
  * @param bar
  *   Index of the io pci resource we want to access.
  * @param p
@@ -542,6 +543,8 @@ int rte_eal_pci_ioport_map(struct rte_pci_device *dev, int bar,
  *
  * @param p
  *   The rte_pci_ioport object to be uninitialized.
+ * @return
+ *  0 on success, negative on error.
  */
 int rte_eal_pci_ioport_unmap(struct rte_pci_ioport *p);
 
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 2/7] eal/linux: only call iopl on x86

[Olivier Matz]

>From iopl(2) man page: "This call is mostly for the x86 architecture. On
many other architectures it does not exist or will always return an
error".

This patch removes the call to iopl() in rte_eal_iopl_init() for
architectures other than x86, and always return 0 (success). This was
already done for ARM in
commit 0291476ae364 ("eal/linux: never check iopl for arm")

Next patches will introduce the support of memory mapped IO resources
for architectures != x86.

On BSD, there is nothing to do as open("/dev/io") already does the
proper thing. See man IO(4).

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index 8aafd51..bba8fea 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -715,12 +715,8 @@ rte_eal_iopl_init(void)
 #if defined(RTE_ARCH_X86)
 	if (iopl(3) != 0)
 		return -1;
-	return 0;
-#elif defined(RTE_ARCH_ARM) || defined(RTE_ARCH_ARM64)
-	return 0; /* iopl syscall not supported for ARM/ARM64 */
-#else
-	return -1;
 #endif
+	return 0;
 }
 
 /* Launch threads, called at application init(). */
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 3/7] eal/linux: remove invalid comment

[Olivier Matz]

In a previous commit, the file used to map the PCI resources changed
from "/dev/uio<x>" to "/sys/bus/pci/devices/<busaddr>/resource", making
the comment wrong. Remove it.

Fixes: 9e67561acd1a ("eal/linux: mmap uio resources using resourceX files")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index 068694d..ac449c5 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -309,7 +309,7 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx,
 		struct mapped_pci_resource *uio_res, int map_idx)
 {
 	int fd;
-	char devname[PATH_MAX]; /* contains the /dev/uioX */
+	char devname[PATH_MAX];
 	void *mapaddr;
 	struct rte_pci_addr *loc;
 	struct pci_map *maps;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 4/7] eal/linux: split function parsing pci resources in sysfs

[Olivier Matz]

Split pci_parse_sysfs_resource() and introduce
pci_parse_one_sysfs_resource() that parses one line of sysfs resource
file.

This new function will be exported and used in next commits when
mapping the ioports resources.

No functional change.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/linuxapp/eal/eal_pci.c | 50 ++++++++++++++++++++++-------------
 1 file changed, 32 insertions(+), 18 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index bdc08a0..1a93725 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -190,12 +190,13 @@ pci_find_max_end_va(void)
 	return RTE_PTR_ADD(last->addr, last->len);
 }
 
-/* parse the "resource" sysfs file */
+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
 static int
-pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+	uint64_t *end_addr, uint64_t *flags)
 {
-	FILE *f;
-	char buf[BUFSIZ];
 	union pci_resource_info {
 		struct {
 			char *phys_addr;
@@ -204,6 +205,31 @@ pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
 		};
 		char *ptrs[PCI_RESOURCE_FMT_NVAL];
 	} res_info;
+
+	if (rte_strsplit(line, len, res_info.ptrs, 3, ' ') != 3) {
+		RTE_LOG(ERR, EAL,
+			"%s(): bad resource format\n", __func__);
+		return -1;
+	}
+	errno = 0;
+	*phys_addr = strtoull(res_info.phys_addr, NULL, 16);
+	*end_addr = strtoull(res_info.end_addr, NULL, 16);
+	*flags = strtoull(res_info.flags, NULL, 16);
+	if (errno != 0) {
+		RTE_LOG(ERR, EAL,
+			"%s(): bad resource format\n", __func__);
+		return -1;
+	}
+
+	return 0;
+}
+
+/* parse the "resource" sysfs file */
+static int
+pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
+{
+	FILE *f;
+	char buf[BUFSIZ];
 	int i;
 	uint64_t phys_addr, end_addr, flags;
 
@@ -220,21 +246,9 @@ pci_parse_sysfs_resource(const char *filename, struct rte_pci_device *dev)
 				"%s(): cannot read resource\n", __func__);
 			goto error;
 		}
-
-		if (rte_strsplit(buf, sizeof(buf), res_info.ptrs, 3, ' ') != 3) {
-			RTE_LOG(ERR, EAL,
-				"%s(): bad resource format\n", __func__);
+		if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
+				&end_addr, &flags) < 0)
 			goto error;
-		}
-		errno = 0;
-		phys_addr = strtoull(res_info.phys_addr, NULL, 16);
-		end_addr = strtoull(res_info.end_addr, NULL, 16);
-		flags = strtoull(res_info.flags, NULL, 16);
-		if (errno != 0) {
-			RTE_LOG(ERR, EAL,
-				"%s(): bad resource format\n", __func__);
-			goto error;
-		}
 
 		if (flags & IORESOURCE_MEM) {
 			dev->mem_resource[i].phys_addr = phys_addr;
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

On PPC64, the ioports are mapped in memory. Implement the missing part
of ioport API for PPC64 when using uio. This may also work on other
architectures but it has not been tested.

Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/common/include/rte_pci.h    |   4 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c      |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci_init.h |  10 +++
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c  | 119 +++++++++++++++++++++++------
 4 files changed, 108 insertions(+), 27 deletions(-)

diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
index fd049d1..565bf38 100644
--- a/lib/librte_eal/common/include/rte_pci.h
+++ b/lib/librte_eal/common/include/rte_pci.h
@@ -105,9 +105,6 @@ extern struct pci_device_list pci_device_list; /**< Global list of PCI devices.
 /** Nb. of values in PCI resource format. */
 #define PCI_RESOURCE_FMT_NVAL 3
 
-/** IO resource type: memory address space */
-#define IORESOURCE_MEM        0x00000200
-
 /**
  * A structure describing a PCI resource.
  */
@@ -518,6 +515,7 @@ int rte_eal_pci_write_config(const struct rte_pci_device *device,
 struct rte_pci_ioport {
 	struct rte_pci_device *dev;
 	uint64_t base;
+	uint64_t len; /* only filled for memory mapped ports */
 };
 
 /**
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
index 1a93725..30e2328 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
@@ -193,7 +193,7 @@ pci_find_max_end_va(void)
 /* parse one line of the "resource" sysfs file (note that the 'line'
  * string is modified)
  */
-static int
+int
 pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
 	uint64_t *end_addr, uint64_t *flags)
 {
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
index 7011753..f72a254 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
@@ -36,12 +36,22 @@
 
 #include "eal_vfio.h"
 
+/** IO resource type: */
+#define IORESOURCE_IO         0x00000100
+#define IORESOURCE_MEM        0x00000200
+
 /*
  * Helper function to map PCI resources right after hugepages in virtual memory
  */
 extern void *pci_map_addr;
 void *pci_find_max_end_va(void);
 
+/* parse one line of the "resource" sysfs file (note that the 'line'
+ * string is modified)
+ */
+int pci_parse_one_sysfs_resource(char *line, size_t len, uint64_t *phys_addr,
+	uint64_t *end_addr, uint64_t *flags);
+
 int pci_uio_alloc_resource(struct rte_pci_device *dev,
 		struct mapped_pci_resource **uio_res);
 void pci_uio_free_resource(struct rte_pci_device *dev,
diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
index ac449c5..077ad96 100644
--- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
@@ -35,6 +35,7 @@
 #include <unistd.h>
 #include <fcntl.h>
 #include <dirent.h>
+#include <inttypes.h>
 #include <sys/stat.h>
 #include <sys/mman.h>
 #include <linux/pci_regs.h>
@@ -368,11 +369,11 @@ error:
 	return -1;
 }
 
+#if defined(RTE_ARCH_X86)
 int
 pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
 		   struct rte_pci_ioport *p)
 {
-#if defined(RTE_ARCH_X86)
 	char dirname[PATH_MAX];
 	char filename[PATH_MAX];
 	int uio_num;
@@ -411,81 +412,153 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
 	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);
 
 	p->base = start;
+	p->len = 0;
 	return 0;
+}
 #else
-	RTE_SET_USED(dev);
-	RTE_SET_USED(bar);
-	RTE_SET_USED(p);
+int
+pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
+		   struct rte_pci_ioport *p)
+{
+	FILE *f;
+	char buf[BUFSIZ];
+	char filename[PATH_MAX];
+	uint64_t phys_addr, end_addr, flags;
+	int fd, i;
+	void *addr;
+
+	/* open and read addresses of the corresponding resource in sysfs */
+	snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
+		SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
+		 dev->addr.devid, dev->addr.function);
+	f = fopen(filename, "r");
+	if (f == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
+			strerror(errno));
+		return -1;
+	}
+	for (i = 0; i < bar + 1; i++) {
+		if (fgets(buf, sizeof(buf), f) == NULL) {
+			RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
+			goto error;
+		}
+	}
+	if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
+			&end_addr, &flags) < 0)
+		goto error;
+	if ((flags & IORESOURCE_IO) == 0) {
+		RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
+		goto error;
+	}
+	snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
+		SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
+		 dev->addr.devid, dev->addr.function, bar);
+
+	/* mmap the pci resource */
+	fd = open(filename, O_RDWR);
+	if (fd < 0) {
+		RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
+			strerror(errno));
+		goto error;
+	}
+	addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
+		MAP_SHARED, fd, 0);
+	if (addr == MAP_FAILED) {
+		RTE_LOG(ERR, EAL, "Cannot mmap IO port resource: %s\n",
+			strerror(errno));
+		goto error;
+	}
+
+	/* strangely, the base address is mmap addr + phys_addr */
+	p->base = (uintptr_t)addr + phys_addr;
+	p->len = end_addr + 1;
+	RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%"PRIx64"\n", p->base);
+	fclose(f);
+
+	return 0;
+
+error:
+	fclose(f);
 	return -1;
-#endif
 }
+#endif
 
 void
 pci_uio_ioport_read(struct rte_pci_ioport *p,
 		    void *data, size_t len, off_t offset)
 {
-#if defined(RTE_ARCH_X86)
 	uint8_t *d;
 	int size;
-	unsigned short reg = p->base + offset;
+	uintptr_t reg = p->base + offset;
 
 	for (d = data; len > 0; d += size, reg += size, len -= size) {
 		if (len >= 4) {
 			size = 4;
+#if defined(RTE_ARCH_X86)
 			*(uint32_t *)d = inl(reg);
+#else
+			*(uint32_t *)d = *(volatile uint32_t *)reg;
+#endif
 		} else if (len >= 2) {
 			size = 2;
+#if defined(RTE_ARCH_X86)
 			*(uint16_t *)d = inw(reg);
+#else
+			*(uint16_t *)d = *(volatile uint16_t *)reg;
+#endif
 		} else {
 			size = 1;
+#if defined(RTE_ARCH_X86)
 			*d = inb(reg);
-		}
-	}
 #else
-	RTE_SET_USED(p);
-	RTE_SET_USED(data);
-	RTE_SET_USED(len);
-	RTE_SET_USED(offset);
+			*d = *(volatile uint8_t *)reg;
 #endif
+		}
+	}
 }
 
 void
 pci_uio_ioport_write(struct rte_pci_ioport *p,
 		     const void *data, size_t len, off_t offset)
 {
-#if defined(RTE_ARCH_X86)
 	const uint8_t *s;
 	int size;
-	unsigned short reg = p->base + offset;
+	uintptr_t reg = p->base + offset;
 
 	for (s = data; len > 0; s += size, reg += size, len -= size) {
 		if (len >= 4) {
 			size = 4;
+#if defined(RTE_ARCH_X86)
 			outl_p(*(const uint32_t *)s, reg);
+#else
+			*(volatile uint32_t *)reg = *(const uint32_t *)s;
+#endif
 		} else if (len >= 2) {
 			size = 2;
+#if defined(RTE_ARCH_X86)
 			outw_p(*(const uint16_t *)s, reg);
+#else
+			*(volatile uint16_t *)reg = *(const uint16_t *)s;
+#endif
 		} else {
 			size = 1;
+#if defined(RTE_ARCH_X86)
 			outb_p(*s, reg);
-		}
-	}
 #else
-	RTE_SET_USED(p);
-	RTE_SET_USED(data);
-	RTE_SET_USED(len);
-	RTE_SET_USED(offset);
+			*(volatile uint8_t *)reg = *s;
 #endif
+		}
+	}
 }
 
 int
 pci_uio_ioport_unmap(struct rte_pci_ioport *p)
 {
-	RTE_SET_USED(p);
 #if defined(RTE_ARCH_X86)
+	RTE_SET_USED(p);
 	/* FIXME close intr fd ? */
 	return 0;
 #else
-	return -1;
+	return munmap((void *)(uintptr_t)p->base, p->len);
 #endif
 }
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Olivier Matz]

From: David Marchand <david.marchand@6wind.com>

Although ppc supports both endianesses, qemu supposes that the cpu is
big endian and enforces this for the virtio-net stuff.

Fix PCI accesses in legacy mode. Only ppc64le is supported at the moment.

Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 drivers/net/virtio/virtio_pci.c | 68 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
index 9cdca06..ebf4cf7 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -55,20 +55,88 @@
  */
 #define VIRTIO_PCI_CONFIG(hw) (((hw)->use_msix) ? 24 : 20)
 
+/*
+ * Since we are in legacy mode:
+ * http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf
+ *
+ * "Note that this is possible because while the virtio header is PCI (i.e.
+ * little) endian, the device-specific region is encoded in the native endian of
+ * the guest (where such distinction is applicable)."
+ *
+ * For powerpc which supports both, qemu supposes that cpu is big endian and
+ * enforces this for the virtio-net stuff.
+ */
+
 static void
 legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
 		       void *dst, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+	int size;
+
+	while (length > 0) {
+		if (length >= 4) {
+			size = 4;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+			*(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t *)dst);
+		} else if (length >= 2) {
+			size = 2;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+			*(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t *)dst);
+		} else {
+			size = 1;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		}
+
+		dst = (char *)dst + size;
+		offset += size;
+		length -= size;
+	}
+#else
 	rte_eal_pci_ioport_read(&hw->io, dst, length,
 				VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }
 
 static void
 legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
 			const void *src, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+	union {
+		uint32_t u32;
+		uint16_t u16;
+	} tmp;
+	int size;
+
+	while (length > 0) {
+		if (length >= 4) {
+			size = 4;
+			tmp.u32 = rte_cpu_to_be_32(*(const uint32_t *)src);
+			rte_eal_pci_ioport_write(&hw->io, &tmp.u32, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		} else if (length >= 2) {
+			size = 2;
+			tmp.u16 = rte_cpu_to_be_16(*(const uint16_t *)src);
+			rte_eal_pci_ioport_write(&hw->io, &tmp.u16, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		} else {
+			size = 1;
+			rte_eal_pci_ioport_write(&hw->io, src, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		}
+
+		src = (const char *)src + size;
+		offset += size;
+		length -= size;
+	}
+#else
 	rte_eal_pci_ioport_write(&hw->io, src, length,
 				 VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }
 
 static uint64_t
-- 
2.8.0.rc3

[dpdk-dev] [PATCH v2 7/7] config: enable virtio-net pmd for ppc64

[Olivier Matz]

Now that virtio pmd is supported on ppc, enable it.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 config/defconfig_ppc_64-power8-linuxapp-gcc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc b/config/defconfig_ppc_64-power8-linuxapp-gcc
index 9eb0cc4..bef8f49 100644
--- a/config/defconfig_ppc_64-power8-linuxapp-gcc
+++ b/config/defconfig_ppc_64-power8-linuxapp-gcc
@@ -50,7 +50,7 @@ CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
 # Will turn on them only after the successful testing on Power
 CONFIG_RTE_LIBRTE_IXGBE_PMD=n
 CONFIG_RTE_LIBRTE_I40E_PMD=n
-CONFIG_RTE_LIBRTE_VIRTIO_PMD=n
+CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
 CONFIG_RTE_LIBRTE_VMXNET3_PMD=n
 CONFIG_RTE_LIBRTE_PMD_BOND=n
 CONFIG_RTE_LIBRTE_ENIC_PMD=n
-- 
2.8.0.rc3

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[David Marchand]

Hello Olivier,

On Tue, May 17, 2016 at 11:59 AM, Olivier Matz <olivier.matz@6wind.com> wrote:
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index ac449c5..077ad96 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -411,81 +412,153 @@ pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
>         RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%lx\n", start);
>
>         p->base = start;
> +       p->len = 0;
>         return 0;
> +}
>  #else
> -       RTE_SET_USED(dev);
> -       RTE_SET_USED(bar);
> -       RTE_SET_USED(p);
> +int
> +pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
> +                  struct rte_pci_ioport *p)
> +{
> +       FILE *f;
> +       char buf[BUFSIZ];
> +       char filename[PATH_MAX];
> +       uint64_t phys_addr, end_addr, flags;
> +       int fd, i;
> +       void *addr;
> +
> +       /* open and read addresses of the corresponding resource in sysfs */
> +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
> +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> +                dev->addr.devid, dev->addr.function);
> +       f = fopen(filename, "r");
> +       if (f == NULL) {
> +               RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
> +                       strerror(errno));
> +               return -1;
> +       }
> +       for (i = 0; i < bar + 1; i++) {
> +               if (fgets(buf, sizeof(buf), f) == NULL) {
> +                       RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
> +                       goto error;
> +               }
> +       }
> +       if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
> +                       &end_addr, &flags) < 0)
> +               goto error;
> +       if ((flags & IORESOURCE_IO) == 0) {
> +               RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
> +               goto error;
> +       }
> +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
> +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> +                dev->addr.devid, dev->addr.function, bar);
> +
> +       /* mmap the pci resource */
> +       fd = open(filename, O_RDWR);
> +       if (fd < 0) {
> +               RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
> +                       strerror(errno));
> +               goto error;
> +       }
> +       addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
> +               MAP_SHARED, fd, 0);

Sorry, did not catch it in v1, but a close(fd) is missing here.
With this, I think the patchset looks good.

Just missing some opinion from the virtio maintainers ?


> +       if (addr == MAP_FAILED) {
> +               RTE_LOG(ERR, EAL, "Cannot mmap IO port resource: %s\n",
> +                       strerror(errno));
> +               goto error;
> +       }
> +
> +       /* strangely, the base address is mmap addr + phys_addr */
> +       p->base = (uintptr_t)addr + phys_addr;
> +       p->len = end_addr + 1;
> +       RTE_LOG(DEBUG, EAL, "PCI Port IO found start=0x%"PRIx64"\n", p->base);
> +       fclose(f);
> +
> +       return 0;
> +
> +error:
> +       fclose(f);
>         return -1;
> -#endif
>  }
> +#endif


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

Hi David,

On 05/17/2016 05:54 PM, David Marchand wrote:
> On Tue, May 17, 2016 at 11:59 AM, Olivier Matz <olivier.matz@6wind.com> wrote:
>> +       /* mmap the pci resource */
>> +       fd = open(filename, O_RDWR);
>> +       if (fd < 0) {
>> +               RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
>> +                       strerror(errno));
>> +               goto error;
>> +       }
>> +       addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
>> +               MAP_SHARED, fd, 0);
> 
> Sorry, did not catch it in v1, but a close(fd) is missing here.
> With this, I think the patchset looks good.

Good catch, I'm fixing it.

Thanks,
Olivier

Re: [dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Chao Zhu]

Olivier,

Thanks for the patches! 
Just one comment:
POWER8 machine only supports little endian OS on bare metal. In VM guest, it
can support both little endian and big endian OS. Did you try to run it on
both host (little endian) and guest (big endian and little endian)?

-----Original Message-----
From: Olivier Matz [mailto:olivier.matz@6wind.com] 
Sent: 2016年5月17日 18:00
To: dev@dpdk.org
Cc: david.marchand@6wind.com; chaozhu@linux.vnet.ibm.com; yuanhan.liu@linux.
intel.com; huawei.xie@intel.com
Subject: [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

From: David Marchand <david.marchand@6wind.com>

Although ppc supports both endianesses, qemu supposes that the cpu is big
endian and enforces this for the virtio-net stuff.

Fix PCI accesses in legacy mode. Only ppc64le is supported at the moment.

Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 drivers/net/virtio/virtio_pci.c | 68
+++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/drivers/net/virtio/virtio_pci.c
b/drivers/net/virtio/virtio_pci.c index 9cdca06..ebf4cf7 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -55,20 +55,88 @@
  */
 #define VIRTIO_PCI_CONFIG(hw) (((hw)->use_msix) ? 24 : 20)

+/*
+ * Since we are in legacy mode:
+ * http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf
+ *
+ * "Note that this is possible because while the virtio header is PCI (i.e.
+ * little) endian, the device-specific region is encoded in the native 
+endian of
+ * the guest (where such distinction is applicable)."
+ *
+ * For powerpc which supports both, qemu supposes that cpu is big 
+endian and
+ * enforces this for the virtio-net stuff.
+ */
+
 static void
 legacy_read_dev_config(struct virtio_hw *hw, size_t offset,
 		       void *dst, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+	int size;
+
+	while (length > 0) {
+		if (length >= 4) {
+			size = 4;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+			*(uint32_t *)dst = rte_be_to_cpu_32(*(uint32_t
*)dst);
+		} else if (length >= 2) {
+			size = 2;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+			*(uint16_t *)dst = rte_be_to_cpu_16(*(uint16_t
*)dst);
+		} else {
+			size = 1;
+			rte_eal_pci_ioport_read(&hw->io, dst, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		}
+
+		dst = (char *)dst + size;
+		offset += size;
+		length -= size;
+	}
+#else
 	rte_eal_pci_ioport_read(&hw->io, dst, length,
 				VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }

 static void
 legacy_write_dev_config(struct virtio_hw *hw, size_t offset,
 			const void *src, int length)
 {
+#ifdef RTE_ARCH_PPC_64
+	union {
+		uint32_t u32;
+		uint16_t u16;
+	} tmp;
+	int size;
+
+	while (length > 0) {
+		if (length >= 4) {
+			size = 4;
+			tmp.u32 = rte_cpu_to_be_32(*(const uint32_t *)src);
+			rte_eal_pci_ioport_write(&hw->io, &tmp.u32, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		} else if (length >= 2) {
+			size = 2;
+			tmp.u16 = rte_cpu_to_be_16(*(const uint16_t *)src);
+			rte_eal_pci_ioport_write(&hw->io, &tmp.u16, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		} else {
+			size = 1;
+			rte_eal_pci_ioport_write(&hw->io, src, size,
+				VIRTIO_PCI_CONFIG(hw) + offset);
+		}
+
+		src = (const char *)src + size;
+		offset += size;
+		length -= size;
+	}
+#else
 	rte_eal_pci_ioport_write(&hw->io, src, length,
 				 VIRTIO_PCI_CONFIG(hw) + offset);
+#endif
 }

 static uint64_t
--
2.8.0.rc3

Re: [dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Olivier Matz]

Hi Chao,

On 05/19/2016 11:13 AM, Chao Zhu wrote:
> Olivier,
> 
> Thanks for the patches! 
> Just one comment:
> POWER8 machine only supports little endian OS on bare metal. In VM guest, it
> can support both little endian and big endian OS. Did you try to run it on
> both host (little endian) and guest (big endian and little endian)?

No I didn't test big endian in the guest.

I don't have any big endian VM image. Is it required for the
test? I mean, is it possible to run a big endian DPDK userland
application on a little endian kernel? (and if yes, is it
known to work?)

Maybe I could replace in the patch:
  #ifdef RTE_ARCH_PPC_64
By something like :
  #if defined(RTE_ARCH_PPC_64) && (RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN)

If you have the ability to test it easily, it would be appreciated :)

Thanks for the review!
Olivier

Re: [dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Olivier Matz]

On 05/20/2016 02:11 PM, Olivier Matz wrote:
> Maybe I could replace in the patch:
>   #ifdef RTE_ARCH_PPC_64
> By something like :
>   #if defined(RTE_ARCH_PPC_64) && (RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN)

Hmm it's probably useless since we already do a
rte_be_to_cpu_*(), so it should do nothing on BE
targets.

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Yuanhan Liu]

On Tue, May 17, 2016 at 05:54:01PM +0200, David Marchand wrote:
> > +pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
> > +                  struct rte_pci_ioport *p)
> > +{
> > +       FILE *f;
> > +       char buf[BUFSIZ];
> > +       char filename[PATH_MAX];
> > +       uint64_t phys_addr, end_addr, flags;
> > +       int fd, i;
> > +       void *addr;
> > +
> > +       /* open and read addresses of the corresponding resource in sysfs */
> > +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
> > +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> > +                dev->addr.devid, dev->addr.function);
> > +       f = fopen(filename, "r");
> > +       if (f == NULL) {
> > +               RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
> > +                       strerror(errno));
> > +               return -1;
> > +       }
> > +       for (i = 0; i < bar + 1; i++) {
> > +               if (fgets(buf, sizeof(buf), f) == NULL) {
> > +                       RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
> > +                       goto error;
> > +               }
> > +       }
> > +       if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
> > +                       &end_addr, &flags) < 0)
> > +               goto error;
> > +       if ((flags & IORESOURCE_IO) == 0) {
> > +               RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
> > +               goto error;
> > +       }
> > +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
> > +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
> > +                dev->addr.devid, dev->addr.function, bar);
> > +
> > +       /* mmap the pci resource */
> > +       fd = open(filename, O_RDWR);
> > +       if (fd < 0) {
> > +               RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
> > +                       strerror(errno));
> > +               goto error;
> > +       }
> > +       addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
> > +               MAP_SHARED, fd, 0);
> 
> Sorry, did not catch it in v1, but a close(fd) is missing here.
> With this, I think the patchset looks good.
> 
> Just missing some opinion from the virtio maintainers ?

Apologize for being late for review. Assuming you have done proper
test, this patch set looks good to me. (well, I don't quite like
the tons of "#ifdef ... #else ..#end" block though)

A side note is that I noticed an ABI breakage introduced in this
patch, so, this release is not a good fit?

	--yliu

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

Hi Yuanhan,

On 05/23/2016 03:07 PM, Yuanhan Liu wrote:
> On Tue, May 17, 2016 at 05:54:01PM +0200, David Marchand wrote:
>>> +pci_uio_ioport_map(struct rte_pci_device *dev, int bar,
>>> +                  struct rte_pci_ioport *p)
>>> +{
>>> +       FILE *f;
>>> +       char buf[BUFSIZ];
>>> +       char filename[PATH_MAX];
>>> +       uint64_t phys_addr, end_addr, flags;
>>> +       int fd, i;
>>> +       void *addr;
>>> +
>>> +       /* open and read addresses of the corresponding resource in sysfs */
>>> +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource",
>>> +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
>>> +                dev->addr.devid, dev->addr.function);
>>> +       f = fopen(filename, "r");
>>> +       if (f == NULL) {
>>> +               RTE_LOG(ERR, EAL, "Cannot open sysfs resource: %s\n",
>>> +                       strerror(errno));
>>> +               return -1;
>>> +       }
>>> +       for (i = 0; i < bar + 1; i++) {
>>> +               if (fgets(buf, sizeof(buf), f) == NULL) {
>>> +                       RTE_LOG(ERR, EAL, "Cannot read sysfs resource\n");
>>> +                       goto error;
>>> +               }
>>> +       }
>>> +       if (pci_parse_one_sysfs_resource(buf, sizeof(buf), &phys_addr,
>>> +                       &end_addr, &flags) < 0)
>>> +               goto error;
>>> +       if ((flags & IORESOURCE_IO) == 0) {
>>> +               RTE_LOG(ERR, EAL, "BAR %d is not an IO resource\n", bar);
>>> +               goto error;
>>> +       }
>>> +       snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource%d",
>>> +               SYSFS_PCI_DEVICES, dev->addr.domain, dev->addr.bus,
>>> +                dev->addr.devid, dev->addr.function, bar);
>>> +
>>> +       /* mmap the pci resource */
>>> +       fd = open(filename, O_RDWR);
>>> +       if (fd < 0) {
>>> +               RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", filename,
>>> +                       strerror(errno));
>>> +               goto error;
>>> +       }
>>> +       addr = mmap(NULL, end_addr + 1, PROT_READ | PROT_WRITE,
>>> +               MAP_SHARED, fd, 0);
>>
>> Sorry, did not catch it in v1, but a close(fd) is missing here.
>> With this, I think the patchset looks good.
>>
>> Just missing some opinion from the virtio maintainers ?
> 
> Apologize for being late for review. Assuming you have done proper
> test, this patch set looks good to me. (well, I don't quite like
> the tons of "#ifdef ... #else ..#end" block though)
> 
> A side note is that I noticed an ABI breakage introduced in this
> patch, so, this release is not a good fit?

Thank you for the review.

For reference, here is the report of the ABI checker for EAL:

[−] struct rte_pci_ioport (2)

 1 Field len has been added to this type.
   1) This field will not be initialized by old clients.
   2) Size of the inclusive type has been changed.
      NOTE: this field should be accessed only from the new library
      functions, otherwise it may result in crash or incorrect behavior
      of applications.
 2 Size of this type has been changed from 16 bytes to 24 bytes. 	
   The fields or parameters of such data type may be incorrectly
   initialized or accessed by old client applications.

[−] affected symbols (4)
 rte_eal_pci_ioport_map ( struct rte_pci_device* dev, int bar,
    struct rte_pci_ioport* p ) @@ DPDK_16.04
 3rd parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
 rte_eal_pci_ioport_read ( struct rte_pci_ioport* p, void* data,
    size_t len, off_t offset ) @@ DPDK_16.04
 1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
 rte_eal_pci_ioport_unmap ( struct rte_pci_ioport* p ) @@ DPDK_16.04
 1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
 rte_eal_pci_ioport_write ( struct rte_pci_ioport* p, void const* data,
    size_t len, off_t offset ) @@ DPDK_16.04
 1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.


My understanding of the comment for this structure is that it's
internal to EAL:

/**
 * A structure used to access io resources for a pci device.
 * rte_pci_ioport is arch, os, driver specific, and should not be used
outside
 * of pci ioport api.
 */
struct rte_pci_ioport {
 ...
}

So I'd say it's ok to have it integrated for 16.07.

Regards,
Olivier

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Yuanhan Liu]

On Mon, May 23, 2016 at 03:40:58PM +0200, Olivier Matz wrote:
> For reference, here is the report of the ABI checker for EAL:
> 
> [−] struct rte_pci_ioport (2)
> 
>  1 Field len has been added to this type.
>    1) This field will not be initialized by old clients.
>    2) Size of the inclusive type has been changed.
>       NOTE: this field should be accessed only from the new library
>       functions, otherwise it may result in crash or incorrect behavior
>       of applications.
>  2 Size of this type has been changed from 16 bytes to 24 bytes. 	
>    The fields or parameters of such data type may be incorrectly
>    initialized or accessed by old client applications.
> 
> [−] affected symbols (4)
>  rte_eal_pci_ioport_map ( struct rte_pci_device* dev, int bar,
>     struct rte_pci_ioport* p ) @@ DPDK_16.04
>  3rd parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>  rte_eal_pci_ioport_read ( struct rte_pci_ioport* p, void* data,
>     size_t len, off_t offset ) @@ DPDK_16.04
>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>  rte_eal_pci_ioport_unmap ( struct rte_pci_ioport* p ) @@ DPDK_16.04
>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>  rte_eal_pci_ioport_write ( struct rte_pci_ioport* p, void const* data,
>     size_t len, off_t offset ) @@ DPDK_16.04
>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
> 
> 
> My understanding of the comment for this structure is that it's
> internal to EAL:

I'm not quite sure that is enough. Cc'ed Panu, the guru on ABI stuff,
hopefully he could shed some light on it.

> /**
>  * A structure used to access io resources for a pci device.
>  * rte_pci_ioport is arch, os, driver specific, and should not be used
> outside
>  * of pci ioport api.
>  */
> struct rte_pci_ioport {
>  ...
> }
> 
> So I'd say it's ok to have it integrated for 16.07.

I'll let Thomas to decide it :)

	--yliu

Re: [dpdk-dev] [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy mode

[Chao Zhu]

Olivier,

The patch set looks good. I think little endian is good enough. We
internally decide to only support little endian previously. You can keep the
endian conversion such kind of things in the code.

-----Original Message-----
From: Olivier Matz [mailto:olivier.matz@6wind.com] 
Sent: 2016年5月20日 20:11
To: Chao Zhu <chaozhu@linux.vnet.ibm.com>; dev@dpdk.org
Cc: david.marchand@6wind.com; yuanhan.liu@linux.intel.com; huawei.xie@intel.
com
Subject: Re: [PATCH v2 6/7] virtio: fix pci accesses for ppc64 in legacy
mode

Hi Chao,

On 05/19/2016 11:13 AM, Chao Zhu wrote:
> Olivier,
> 
> Thanks for the patches! 
> Just one comment:
> POWER8 machine only supports little endian OS on bare metal. In VM 
> guest, it can support both little endian and big endian OS. Did you 
> try to run it on both host (little endian) and guest (big endian and
little endian)?

No I didn't test big endian in the guest.

I don't have any big endian VM image. Is it required for the test? I mean,
is it possible to run a big endian DPDK userland application on a little
endian kernel? (and if yes, is it known to work?)

Maybe I could replace in the patch:
  #ifdef RTE_ARCH_PPC_64
By something like :
  #if defined(RTE_ARCH_PPC_64) && (RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN)

If you have the ability to test it easily, it would be appreciated :)

Thanks for the review!
Olivier

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Olivier Matz]

On 05/24/2016 07:15 AM, Yuanhan Liu wrote:
> On Mon, May 23, 2016 at 03:40:58PM +0200, Olivier Matz wrote:
>> For reference, here is the report of the ABI checker for EAL:
>>
>> [−] struct rte_pci_ioport (2)
>>
>>  1 Field len has been added to this type.
>>    1) This field will not be initialized by old clients.
>>    2) Size of the inclusive type has been changed.
>>       NOTE: this field should be accessed only from the new library
>>       functions, otherwise it may result in crash or incorrect behavior
>>       of applications.
>>  2 Size of this type has been changed from 16 bytes to 24 bytes. 	
>>    The fields or parameters of such data type may be incorrectly
>>    initialized or accessed by old client applications.
>>
>> [−] affected symbols (4)
>>  rte_eal_pci_ioport_map ( struct rte_pci_device* dev, int bar,
>>     struct rte_pci_ioport* p ) @@ DPDK_16.04
>>  3rd parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>>  rte_eal_pci_ioport_read ( struct rte_pci_ioport* p, void* data,
>>     size_t len, off_t offset ) @@ DPDK_16.04
>>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>>  rte_eal_pci_ioport_unmap ( struct rte_pci_ioport* p ) @@ DPDK_16.04
>>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>>  rte_eal_pci_ioport_write ( struct rte_pci_ioport* p, void const* data,
>>     size_t len, off_t offset ) @@ DPDK_16.04
>>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
>>
>>
>> My understanding of the comment for this structure is that it's
>> internal to EAL:
> 
> I'm not quite sure that is enough. Cc'ed Panu, the guru on ABI stuff,
> hopefully he could shed some light on it.
> 
>> /**
>>  * A structure used to access io resources for a pci device.
>>  * rte_pci_ioport is arch, os, driver specific, and should not be used
>> outside
>>  * of pci ioport api.
>>  */
>> struct rte_pci_ioport {
>>  ...
>> }
>>
>> So I'd say it's ok to have it integrated for 16.07.
> 
> I'll let Thomas to decide it :)

Panu or Thomas, do you have any comment on this?

Thanks,
Olivier

Re: [dpdk-dev] [PATCH v2 5/7] eal/linux: mmap ioports on ppc64

[Thomas Monjalon]

2016-05-30 10:45, Olivier Matz:
> On 05/24/2016 07:15 AM, Yuanhan Liu wrote:
> > On Mon, May 23, 2016 at 03:40:58PM +0200, Olivier Matz wrote:
> >> For reference, here is the report of the ABI checker for EAL:
> >>
> >> [−] struct rte_pci_ioport (2)
> >>
> >>  1 Field len has been added to this type.
> >>    1) This field will not be initialized by old clients.
> >>    2) Size of the inclusive type has been changed.
> >>       NOTE: this field should be accessed only from the new library
> >>       functions, otherwise it may result in crash or incorrect behavior
> >>       of applications.
> >>  2 Size of this type has been changed from 16 bytes to 24 bytes. 	
> >>    The fields or parameters of such data type may be incorrectly
> >>    initialized or accessed by old client applications.
> >>
> >> [−] affected symbols (4)
> >>  rte_eal_pci_ioport_map ( struct rte_pci_device* dev, int bar,
> >>     struct rte_pci_ioport* p ) @@ DPDK_16.04
> >>  3rd parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
> >>  rte_eal_pci_ioport_read ( struct rte_pci_ioport* p, void* data,
> >>     size_t len, off_t offset ) @@ DPDK_16.04
> >>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
> >>  rte_eal_pci_ioport_unmap ( struct rte_pci_ioport* p ) @@ DPDK_16.04
> >>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
> >>  rte_eal_pci_ioport_write ( struct rte_pci_ioport* p, void const* data,
> >>     size_t len, off_t offset ) @@ DPDK_16.04
> >>  1st parameter 'p' (pointer) has base type 'struct rte_pci_ioport'.
> >>
> >>
> >> My understanding of the comment for this structure is that it's
> >> internal to EAL:
> > 
> > I'm not quite sure that is enough. Cc'ed Panu, the guru on ABI stuff,
> > hopefully he could shed some light on it.
> > 
> >> /**
> >>  * A structure used to access io resources for a pci device.
> >>  * rte_pci_ioport is arch, os, driver specific, and should not be used
> >> outside
> >>  * of pci ioport api.
> >>  */
> >> struct rte_pci_ioport {
> >>  ...
> >> }
> >>
> >> So I'd say it's ok to have it integrated for 16.07.
> > 
> > I'll let Thomas to decide it :)
> 
> Panu or Thomas, do you have any comment on this?

The user of this struct is virtio.
The ABI policy does not apply to drivers:
	- A means Application
	- external drivers must be rebuilt for each new release

Thus no problem here.

Re: [dpdk-dev] [PATCH v2 0/7] virtio-net support on ppc64

[Thomas Monjalon]

> David Marchand (1):
>   virtio: fix pci accesses for ppc64 in legacy mode
> 
> Olivier Matz (6):
>   eal: fix typos in ioport API doxygen comments
>   eal/linux: only call iopl on x86
>   eal/linux: remove invalid comment
>   eal/linux: split function parsing pci resources in sysfs
>   eal/linux: mmap ioports on ppc64
>   config: enable virtio-net pmd for ppc64

Rebased on top of "pci: allow to override sysfs path" and
Applied, thanks