* [dpdk-dev] DPDK on IBM Power9
@ 2019-01-09 22:25 Daniel Waddington
2019-01-09 22:52 ` Wiles, Keith
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Daniel Waddington @ 2019-01-09 22:25 UTC (permalink / raw)
To: dev
Hi,
I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
Thx,
Daniel
Below is what I get:
$ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
EAL: Detected 128 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:1583 net_i40e
EAL: using IOMMU type 7 (sPAPR)
EAL: cannot create new DMA window, error 22 (Invalid argument)
EAL: Could not create new DMA window
EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
EAL: Requested device 0000:01:00.0 cannot be used
Number of NICs: 0
EAL: Error - exiting with code: 1
Cause: No available NIC ports!
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 22:25 [dpdk-dev] DPDK on IBM Power9 Daniel Waddington
@ 2019-01-09 22:52 ` Wiles, Keith
2019-01-09 23:22 ` Daniel Waddington
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Wiles, Keith @ 2019-01-09 22:52 UTC (permalink / raw)
To: Daniel Waddington; +Cc: dev
> On Jan 9, 2019, at 4:25 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>
> Hi,
> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
> Thx,
> Daniel
> Below is what I get:
>
> $ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
> EAL: Detected 128 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1583 net_i40e
> EAL: using IOMMU type 7 (sPAPR)
> EAL: cannot create new DMA window, error 22 (Invalid argument)
> EAL: Could not create new DMA window
> EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
> EAL: Requested device 0000:01:00.0 cannot be used
> Number of NICs: 0
> EAL: Error - exiting with code: 1
> Cause: No available NIC ports!
>
>From the message above you need support NIC for DPDK is all or you a NIC needs to be bound to something like igb_uio for linux. I have never run DPDK on power before and not sure what is next other then using a supported NIC.
>
>
Regards,
Keith
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 22:25 [dpdk-dev] DPDK on IBM Power9 Daniel Waddington
2019-01-09 22:52 ` Wiles, Keith
@ 2019-01-09 23:22 ` Daniel Waddington
2019-01-09 23:35 ` Wiles, Keith
2019-01-09 23:58 ` Daniel Waddington
2019-01-10 0:07 ` Thomas Monjalon
2019-01-10 8:51 ` Burakov, Anatoly
3 siblings, 2 replies; 9+ messages in thread
From: Daniel Waddington @ 2019-01-09 23:22 UTC (permalink / raw)
To: Wiles, Keith; +Cc: dev
This NIC (000:01:00.0) is supported (Intel XL710). I assume that I can use VFIO instead of UIO. UIO is really bad from a security perspective.
I think its the "cannot create new DMA window, error 22" which is the first indicator of the problem.
Daniel
-----"Wiles, Keith" <keith.wiles@intel.com> wrote: -----
To: Daniel Waddington <Daniel.Waddington@ibm.com>
From: "Wiles, Keith" <keith.wiles@intel.com>
Date: 01/09/2019 02:52PM
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK on IBM Power9
> On Jan 9, 2019, at 4:25 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>
> Hi,
> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
> Thx,
> Daniel
> Below is what I get:
>
> $ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
> EAL: Detected 128 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1583 net_i40e
> EAL: using IOMMU type 7 (sPAPR)
> EAL: cannot create new DMA window, error 22 (Invalid argument)
> EAL: Could not create new DMA window
> EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
> EAL: Requested device 0000:01:00.0 cannot be used
> Number of NICs: 0
> EAL: Error - exiting with code: 1
> Cause: No available NIC ports!
>
>From the message above you need support NIC for DPDK is all or you a NIC needs to be bound to something like igb_uio for linux. I have never run DPDK on power before and not sure what is next other then using a supported NIC.
>
>
Regards,
Keith
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 23:22 ` Daniel Waddington
@ 2019-01-09 23:35 ` Wiles, Keith
2019-01-09 23:58 ` Daniel Waddington
1 sibling, 0 replies; 9+ messages in thread
From: Wiles, Keith @ 2019-01-09 23:35 UTC (permalink / raw)
To: Daniel Waddington; +Cc: dev
> On Jan 9, 2019, at 5:22 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>
> This NIC (000:01:00.0) is supported (Intel XL710). I assume that I can use VFIO instead of UIO. UIO is really bad from a security perspective.
>
> I think its the "cannot create new DMA window, error 22" which is the first indicator of the problem.
>
> Daniel
>
> -----"Wiles, Keith" <keith.wiles@intel.com> wrote: -----
> To: Daniel Waddington <Daniel.Waddington@ibm.com>
> From: "Wiles, Keith" <keith.wiles@intel.com>
> Date: 01/09/2019 02:52PM
> Cc: "dev@dpdk.org" <dev@dpdk.org>
> Subject: Re: [dpdk-dev] DPDK on IBM Power9
>
>
>> On Jan 9, 2019, at 4:25 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>>
>> Hi,
>> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
>> Thx,
>> Daniel
>> Below is what I get:
>>
>> $ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
>> EAL: Detected 128 lcore(s)
>> EAL: Detected 2 NUMA nodes
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: Probing VFIO support...
>> EAL: VFIO support initialized
>> EAL: PCI device 0000:01:00.0 on NUMA socket 0
>> EAL: probe driver: 8086:1583 net_i40e
>> EAL: using IOMMU type 7 (sPAPR)
>> EAL: cannot create new DMA window, error 22 (Invalid argument)
>> EAL: Could not create new DMA window
>> EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
>> EAL: Requested device 0000:01:00.0 cannot be used
>> Number of NICs: 0
>> EAL: Error - exiting with code: 1
>> Cause: No available NIC ports!
>>
>
> From the message above you need support NIC for DPDK is all or you a NIC needs to be bound to something like igb_uio for linux. I have never run DPDK on power before and not sure what is next other then using a supported NIC.
>
Yes you should be able to use VFIO. Did you bind the NIC to VFIO ?
There should be some docs about how that is done.
Beyond this I have not idea?
>>
>>
>
> Regards,
> Keith
>
>
Regards,
Keith
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 23:22 ` Daniel Waddington
2019-01-09 23:35 ` Wiles, Keith
@ 2019-01-09 23:58 ` Daniel Waddington
1 sibling, 0 replies; 9+ messages in thread
From: Daniel Waddington @ 2019-01-09 23:58 UTC (permalink / raw)
To: Wiles, Keith; +Cc: dev
Also, this is what the kernel shows.
....
[ 2812.456773] pci 0000:01 : [PE# 00] Enabling 64-bit DMA bypass
[ 2876.395586] pci 0000:01 : [PE# 00] Disabling 64-bit DMA bypass
[ 2876.395593] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 2876.395705] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 2876.395711] pci 0000:01 : [PE# 00] Removing DMA window #1
[ 2876.399657] pci 0000:01 : [PE# 00] Setting up window#0 0..7fffffff pg=1000
[ 2876.399663] pci 0000:01 : [PE# 00] Enabling 64-bit DMA bypass
[ 6519.671051] pci 0000:01 : [PE# 00] Disabling 64-bit DMA bypass
[ 6519.671059] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 6519.671174] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 6519.671180] pci 0000:01 : [PE# 00] Removing DMA window #1
[ 6519.675520] pci 0000:01 : [PE# 00] Setting up window#0 0..7fffffff pg=1000
[ 6519.675526] pci 0000:01 : [PE# 00] Enabling 64-bit DMA bypass
[ 9015.064426] pci 0000:01 : [PE# 00] Disabling 64-bit DMA bypass
[ 9015.064432] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 9015.064527] pci 0000:01 : [PE# 00] Removing DMA window #0
[ 9015.064531] pci 0000:01 : [PE# 00] Removing DMA window #1
[ 9015.067168] pci 0000:01 : [PE# 00] Setting up window#0 0..7fffffff pg=1000
[ 9015.067173] pci 0000:01 : [PE# 00] Enabling 64-bit DMA bypass
Daniel
-----"Wiles, Keith" <keith.wiles@intel.com> wrote: -----
To: Daniel Waddington <Daniel.Waddington@ibm.com>
From: "Wiles, Keith" <keith.wiles@intel.com>
Date: 01/09/2019 03:35PM
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK on IBM Power9
> On Jan 9, 2019, at 5:22 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>
> This NIC (000:01:00.0) is supported (Intel XL710). I assume that I can use VFIO instead of UIO. UIO is really bad from a security perspective.
>
> I think its the "cannot create new DMA window, error 22" which is the first indicator of the problem.
>
> Daniel
>
> -----"Wiles, Keith" <keith.wiles@intel.com> wrote: -----
> To: Daniel Waddington <Daniel.Waddington@ibm.com>
> From: "Wiles, Keith" <keith.wiles@intel.com>
> Date: 01/09/2019 02:52PM
> Cc: "dev@dpdk.org" <dev@dpdk.org>
> Subject: Re: [dpdk-dev] DPDK on IBM Power9
>
>
>> On Jan 9, 2019, at 4:25 PM, Daniel Waddington <Daniel.Waddington@ibm.com> wrote:
>>
>> Hi,
>> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
>> Thx,
>> Daniel
>> Below is what I get:
>>
>> $ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
>> EAL: Detected 128 lcore(s)
>> EAL: Detected 2 NUMA nodes
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: Probing VFIO support...
>> EAL: VFIO support initialized
>> EAL: PCI device 0000:01:00.0 on NUMA socket 0
>> EAL: probe driver: 8086:1583 net_i40e
>> EAL: using IOMMU type 7 (sPAPR)
>> EAL: cannot create new DMA window, error 22 (Invalid argument)
>> EAL: Could not create new DMA window
>> EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
>> EAL: Requested device 0000:01:00.0 cannot be used
>> Number of NICs: 0
>> EAL: Error - exiting with code: 1
>> Cause: No available NIC ports!
>>
>
> From the message above you need support NIC for DPDK is all or you a NIC needs to be bound to something like igb_uio for linux. I have never run DPDK on power before and not sure what is next other then using a supported NIC.
>
Yes you should be able to use VFIO. Did you bind the NIC to VFIO ?
There should be some docs about how that is done.
Beyond this I have not idea?
>>
>>
>
> Regards,
> Keith
>
>
Regards,
Keith
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 22:25 [dpdk-dev] DPDK on IBM Power9 Daniel Waddington
2019-01-09 22:52 ` Wiles, Keith
2019-01-09 23:22 ` Daniel Waddington
@ 2019-01-10 0:07 ` Thomas Monjalon
2019-01-10 0:23 ` Wiles, Keith
2019-01-10 7:40 ` Olga Shern
2019-01-10 8:51 ` Burakov, Anatoly
3 siblings, 2 replies; 9+ messages in thread
From: Thomas Monjalon @ 2019-01-10 0:07 UTC (permalink / raw)
To: Daniel Waddington; +Cc: dev
Hi,
09/01/2019 23:25, Daniel Waddington:
> Hi,
> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
As mentioned in the release notes, Power9 is supported:
http://doc.dpdk.org/guides/rel_notes/release_18_11.html#known-issues
We are missing a strong support from IBM.
Several requests were already sent.
Given your email address, you should be able to forward the need ;)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-10 0:07 ` Thomas Monjalon
@ 2019-01-10 0:23 ` Wiles, Keith
2019-01-10 7:40 ` Olga Shern
1 sibling, 0 replies; 9+ messages in thread
From: Wiles, Keith @ 2019-01-10 0:23 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: Daniel Waddington, dev
> On Jan 9, 2019, at 6:07 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
>
> Hi,
>
> 09/01/2019 23:25, Daniel Waddington:
>> Hi,
>> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
>
> As mentioned in the release notes, Power9 is supported:
> http://doc.dpdk.org/guides/rel_notes/release_18_11.html#known-issues
>
> We are missing a strong support from IBM.
> Several requests were already sent.
> Given your email address, you should be able to forward the need ;)
>
>From the MAINTAINERS file:
IBM POWER (alpha)
M: Chao Zhu <chaozhu@linux.vnet.ibm.com>
F: lib/librte_eal/common/arch/ppc_64/
F: lib/librte_eal/common/include/arch/ppc_64/
F: drivers/net/i40e/i40e_rxtx_vec_altivec.c
F: examples/l3fwd/*altivec.h
So Chao Zhu IBM is the maintainer of the ppc code. That was the only reference I could find.
Regards,
Keith
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-10 0:07 ` Thomas Monjalon
2019-01-10 0:23 ` Wiles, Keith
@ 2019-01-10 7:40 ` Olga Shern
1 sibling, 0 replies; 9+ messages in thread
From: Olga Shern @ 2019-01-10 7:40 UTC (permalink / raw)
To: Thomas Monjalon, Daniel Waddington; +Cc: dev
Daniel,
What exactly is not working for you?
Mellanox is testing Power9 for 19.02 release.
Thanks,
Olga
-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
Sent: Thursday, January 10, 2019 2:08 AM
To: Daniel Waddington <Daniel.Waddington@ibm.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK on IBM Power9
Hi,
09/01/2019 23:25, Daniel Waddington:
> Hi,
> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
As mentioned in the release notes, Power9 is supported:
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Frel_notes%2Frelease_18_11.html%23known-issues&data=02%7C01%7Colgas%40mellanox.com%7C6c8d43623a0348adae1408d6768fb1b1%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636826756880461285&sdata=WzB%2Fz2zsUUZUawHT1W42BPI2M%2FfDu943xNgDI4w90lU%3D&reserved=0
We are missing a strong support from IBM.
Several requests were already sent.
Given your email address, you should be able to forward the need ;)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dpdk-dev] DPDK on IBM Power9
2019-01-09 22:25 [dpdk-dev] DPDK on IBM Power9 Daniel Waddington
` (2 preceding siblings ...)
2019-01-10 0:07 ` Thomas Monjalon
@ 2019-01-10 8:51 ` Burakov, Anatoly
3 siblings, 0 replies; 9+ messages in thread
From: Burakov, Anatoly @ 2019-01-10 8:51 UTC (permalink / raw)
To: Daniel Waddington, dev
On 09-Jan-19 10:25 PM, Daniel Waddington wrote:
> Hi,
> I'm trying to run DPDK 18.11 on IBM Power9. Should it run? (I'n new to DPDK on Power).
> Thx,
> Daniel
> Below is what I get:
>
> $ sudo ./examples/ethtool/ethtool-app/ppc_64-power8-linuxapp-gcc/app/ethtool --iova-mode=pa -c7 -w 0000:01:00.0
> EAL: Detected 128 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1583 net_i40e
> EAL: using IOMMU type 7 (sPAPR)
> EAL: cannot create new DMA window, error 22 (Invalid argument)
> EAL: Could not create new DMA window
> EAL: 0000:01:00.0 DMA remapping failed, error 22 (Invalid argument)
> EAL: Requested device 0000:01:00.0 cannot be used
> Number of NICs: 0
> EAL: Error - exiting with code: 1
> Cause: No available NIC ports!
>
I know _where_ this happens, but as to _why_... You're IBM, you tell me!
There have been numerous issues with VFIO/memory subsystem being pretty
much unmaintained as far as IBM platforms support is concerned, but
there's little i, as a maintainer, can do about it. Thomas is right, one
more reason to ping the relevant people within IBM :)
--
Thanks,
Anatoly
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-01-10 8:51 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-09 22:25 [dpdk-dev] DPDK on IBM Power9 Daniel Waddington
2019-01-09 22:52 ` Wiles, Keith
2019-01-09 23:22 ` Daniel Waddington
2019-01-09 23:35 ` Wiles, Keith
2019-01-09 23:58 ` Daniel Waddington
2019-01-10 0:07 ` Thomas Monjalon
2019-01-10 0:23 ` Wiles, Keith
2019-01-10 7:40 ` Olga Shern
2019-01-10 8:51 ` Burakov, Anatoly
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).