DPDK patches and discussions
 help / color / mirror / Atom feed
From: David Christensen <drc@linux.vnet.ibm.com>
To: Nithin Dabilpuram <nithind1988@gmail.com>,
	anatoly.burakov@intel.com, david.marchand@redhat.com
Cc: jerinj@marvell.com, dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v6 0/4] fix issue with partial DMA unmap
Date: Mon, 4 Jan 2021 14:29:23 -0800	[thread overview]
Message-ID: <7bf59410-6d2b-e3b2-bb98-4ca9290e1abb@linux.vnet.ibm.com> (raw)
In-Reply-To: <X+LR89T6P9bKI93N@gmail.com>



On 12/22/20 9:13 PM, Nithin Dabilpuram wrote:
> Ping.

Tested the patches and they generate a failure on my P9 system:

EAL:   cannot map vaddr for IOMMU, error 22 (Invalid argument)

I'm looking at it now to see what the problem might be.  I'm assuing 
it's related to the size paramter (see 
https://elixir.bootlin.com/linux/v4.18/source/drivers/vfio/vfio_iommu_spapr_tce.c#L906) 
but I'm not sure yet.  See below for a bit more of the debug output from 
the failure.

Dave


sudo gdb --args /home/drc/src/dpdk/build/app/test/dpdk-test 
--log="eal,debug" --iova-mode=va -a 0034:01:00.0 -a 0034:01:00.1 -l 
64-127 -n 4
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-12.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "ppc64le-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
     <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/drc/src/dpdk/build/app/test/dpdk-test...done.
(gdb) b test_vfio
Breakpoint 1 at 0x1066ae34: file ../app/test/test_vfio.c, line 96.
(gdb) start
Temporary breakpoint 2 at 0x1000e724: file ../app/test/test.c, line 100.
Starting program: /home/drc/src/dpdk/build/app/test/dpdk-test 
--log=eal,debug --iova-mode=va -a 0034:01:00.0 -a 0034:01:00.1 -l 64-127 
-n 4
Missing separate debuginfos, use: yum debuginfo-install 
glibc-2.28-127.el8.ppc64le
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/power9/libthread_db.so.1".

Temporary breakpoint 2, main (argc=11, argv=0x7ffffffff098) at 
../app/test/test.c:100
100		extra_args = getenv("DPDK_TEST_PARAMS");
Missing separate debuginfos, use: yum debuginfo-install 
elfutils-libelf-0.180-1.el8.ppc64le jansson-2.11-3.el8.ppc64le 
libibverbs-29.0-3.el8.ppc64le libnl3-3.5.0-1.el8.ppc64le 
libpcap-1.9.1-4.el8.ppc64le numactl-libs-2.0.12-11.el8.ppc64le 
openssl-libs-1.1.1g-11.el8.ppc64le zlib-1.2.11-16.el8_2.ppc64le
(gdb) c
Continuing.
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 0 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 4 on socket 0
EAL: Detected lcore 6 as core 4 on socket 0
EAL: Detected lcore 7 as core 4 on socket 0
EAL: Detected lcore 8 as core 8 on socket 0
EAL: Detected lcore 9 as core 8 on socket 0
EAL: Detected lcore 10 as core 8 on socket 0
EAL: Detected lcore 11 as core 8 on socket 0
EAL: Detected lcore 12 as core 12 on socket 0
EAL: Detected lcore 13 as core 12 on socket 0
EAL: Detected lcore 14 as core 12 on socket 0
EAL: Detected lcore 15 as core 12 on socket 0
EAL: Detected lcore 16 as core 16 on socket 0
EAL: Detected lcore 17 as core 16 on socket 0
EAL: Detected lcore 18 as core 16 on socket 0
EAL: Detected lcore 19 as core 16 on socket 0
EAL: Detected lcore 20 as core 20 on socket 0
EAL: Detected lcore 21 as core 20 on socket 0
EAL: Detected lcore 22 as core 20 on socket 0
EAL: Detected lcore 23 as core 20 on socket 0
EAL: Detected lcore 24 as core 24 on socket 0
EAL: Detected lcore 25 as core 24 on socket 0
EAL: Detected lcore 26 as core 24 on socket 0
EAL: Detected lcore 27 as core 24 on socket 0
EAL: Detected lcore 28 as core 28 on socket 0
EAL: Detected lcore 29 as core 28 on socket 0
EAL: Detected lcore 30 as core 28 on socket 0
EAL: Detected lcore 31 as core 28 on socket 0
EAL: Detected lcore 32 as core 32 on socket 0
EAL: Detected lcore 33 as core 32 on socket 0
EAL: Detected lcore 34 as core 32 on socket 0
EAL: Detected lcore 35 as core 32 on socket 0
EAL: Detected lcore 36 as core 36 on socket 0
EAL: Detected lcore 37 as core 36 on socket 0
EAL: Detected lcore 38 as core 36 on socket 0
EAL: Detected lcore 39 as core 36 on socket 0
EAL: Detected lcore 40 as core 48 on socket 0
EAL: Detected lcore 41 as core 48 on socket 0
EAL: Detected lcore 42 as core 48 on socket 0
EAL: Detected lcore 43 as core 48 on socket 0
EAL: Detected lcore 44 as core 52 on socket 0
EAL: Detected lcore 45 as core 52 on socket 0
EAL: Detected lcore 46 as core 52 on socket 0
EAL: Detected lcore 47 as core 52 on socket 0
EAL: Detected lcore 48 as core 72 on socket 0
EAL: Detected lcore 49 as core 72 on socket 0
EAL: Detected lcore 50 as core 72 on socket 0
EAL: Detected lcore 51 as core 72 on socket 0
EAL: Detected lcore 52 as core 76 on socket 0
EAL: Detected lcore 53 as core 76 on socket 0
EAL: Detected lcore 54 as core 76 on socket 0
EAL: Detected lcore 55 as core 76 on socket 0
EAL: Detected lcore 56 as core 80 on socket 0
EAL: Detected lcore 57 as core 80 on socket 0
EAL: Detected lcore 58 as core 80 on socket 0
EAL: Detected lcore 59 as core 80 on socket 0
EAL: Detected lcore 60 as core 84 on socket 0
EAL: Detected lcore 61 as core 84 on socket 0
EAL: Detected lcore 62 as core 84 on socket 0
EAL: Detected lcore 63 as core 84 on socket 0
EAL: Detected lcore 64 as core 2048 on socket 8
EAL: Detected lcore 65 as core 2048 on socket 8
EAL: Detected lcore 66 as core 2048 on socket 8
EAL: Detected lcore 67 as core 2048 on socket 8
EAL: Detected lcore 68 as core 2052 on socket 8
EAL: Detected lcore 69 as core 2052 on socket 8
EAL: Detected lcore 70 as core 2052 on socket 8
EAL: Detected lcore 71 as core 2052 on socket 8
EAL: Detected lcore 72 as core 2056 on socket 8
EAL: Detected lcore 73 as core 2056 on socket 8
EAL: Detected lcore 74 as core 2056 on socket 8
EAL: Detected lcore 75 as core 2056 on socket 8
EAL: Detected lcore 76 as core 2060 on socket 8
EAL: Detected lcore 77 as core 2060 on socket 8
EAL: Detected lcore 78 as core 2060 on socket 8
EAL: Detected lcore 79 as core 2060 on socket 8
EAL: Detected lcore 80 as core 2072 on socket 8
EAL: Detected lcore 81 as core 2072 on socket 8
EAL: Detected lcore 82 as core 2072 on socket 8
EAL: Detected lcore 83 as core 2072 on socket 8
EAL: Detected lcore 84 as core 2076 on socket 8
EAL: Detected lcore 85 as core 2076 on socket 8
EAL: Detected lcore 86 as core 2076 on socket 8
EAL: Detected lcore 87 as core 2076 on socket 8
EAL: Detected lcore 88 as core 2080 on socket 8
EAL: Detected lcore 89 as core 2080 on socket 8
EAL: Detected lcore 90 as core 2080 on socket 8
EAL: Detected lcore 91 as core 2080 on socket 8
EAL: Detected lcore 92 as core 2084 on socket 8
EAL: Detected lcore 93 as core 2084 on socket 8
EAL: Detected lcore 94 as core 2084 on socket 8
EAL: Detected lcore 95 as core 2084 on socket 8
EAL: Detected lcore 96 as core 2088 on socket 8
EAL: Detected lcore 97 as core 2088 on socket 8
EAL: Detected lcore 98 as core 2088 on socket 8
EAL: Detected lcore 99 as core 2088 on socket 8
EAL: Detected lcore 100 as core 2092 on socket 8
EAL: Detected lcore 101 as core 2092 on socket 8
EAL: Detected lcore 102 as core 2092 on socket 8
EAL: Detected lcore 103 as core 2092 on socket 8
EAL: Detected lcore 104 as core 2096 on socket 8
EAL: Detected lcore 105 as core 2096 on socket 8
EAL: Detected lcore 106 as core 2096 on socket 8
EAL: Detected lcore 107 as core 2096 on socket 8
EAL: Detected lcore 108 as core 2100 on socket 8
EAL: Detected lcore 109 as core 2100 on socket 8
EAL: Detected lcore 110 as core 2100 on socket 8
EAL: Detected lcore 111 as core 2100 on socket 8
EAL: Detected lcore 112 as core 2120 on socket 8
EAL: Detected lcore 113 as core 2120 on socket 8
EAL: Detected lcore 114 as core 2120 on socket 8
EAL: Detected lcore 115 as core 2120 on socket 8
EAL: Detected lcore 116 as core 2124 on socket 8
EAL: Detected lcore 117 as core 2124 on socket 8
EAL: Detected lcore 118 as core 2124 on socket 8
EAL: Detected lcore 119 as core 2124 on socket 8
EAL: Detected lcore 120 as core 2136 on socket 8
EAL: Detected lcore 121 as core 2136 on socket 8
EAL: Detected lcore 122 as core 2136 on socket 8
EAL: Detected lcore 123 as core 2136 on socket 8
EAL: Detected lcore 124 as core 2140 on socket 8
EAL: Detected lcore 125 as core 2140 on socket 8
EAL: Detected lcore 126 as core 2140 on socket 8
EAL: Detected lcore 127 as core 2140 on socket 8
EAL: Support maximum 1536 logical core(s) by configuration.
EAL: Detected 128 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x10000)
[New Thread 0x7ffff74ad090 (LWP 140091)]
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
[New Thread 0x7ffff6c9d090 (LWP 140092)]
EAL: DPAA Bus not present. Skipping.
EAL: VFIO PCI modules not loaded
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: Probing VFIO support...
EAL:   IOMMU type 1 (Type 1) is not supported
EAL:   IOMMU type 7 (sPAPR) is supported
EAL:   IOMMU type 8 (No-IOMMU) is not supported
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x30000 bytes
EAL: Virtual area found at 0x100010000 (size = 0x30000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 32768
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Detected memory type: socket_id:8 hugepage_sz:1073741824
EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x100040000 (size = 0x10000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x140000000, size 800000000
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x10000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x980000000, size 800000000
EAL: Creating 2 segment lists: n_segs:32 socket_id:8 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x1180000000 (size = 0x10000)
EAL: Memseg list allocated at socket 8, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x11c0000000, size 800000000
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x19c0000000 (size = 0x10000)
EAL: Memseg list allocated at socket 8, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x1a00000000, size 800000000
EAL: TSC frequency is ~510000 KHz
EAL: Main lcore 64 is ready (tid=7ffff7ff8890;cpuset=[64])
[New Thread 0x7ffff648d090 (LWP 140093)]
[New Thread 0x7ffff5c7d090 (LWP 140094)]
EAL: lcore 65 is ready (tid=7ffff648d090;cpuset=[65])
[New Thread 0x7ffff546d090 (LWP 140095)]
EAL: lcore 66 is ready (tid=7ffff5c7d090;cpuset=[66])
[New Thread 0x7ffff4c5d090 (LWP 140096)]
EAL: lcore 67 is ready (tid=7ffff546d090;cpuset=[67])
[New Thread 0x7fffdfffd090 (LWP 140097)]
EAL: lcore 68 is ready (tid=7ffff4c5d090;cpuset=[68])
[New Thread 0x7fffdf7ed090 (LWP 140098)]
EAL: lcore 69 is ready (tid=7fffdfffd090;cpuset=[69])
[New Thread 0x7fffdefdd090 (LWP 140099)]
EAL: lcore 70 is ready (tid=7fffdf7ed090;cpuset=[70])
[New Thread 0x7fffde7cd090 (LWP 140100)]
EAL: lcore 71 is ready (tid=7fffdefdd090;cpuset=[71])
[New Thread 0x7fffddfbd090 (LWP 140101)]
EAL: lcore 72 is ready (tid=7fffde7cd090;cpuset=[72])
[New Thread 0x7fffdd7ad090 (LWP 140102)]
EAL: lcore 73 is ready (tid=7fffddfbd090;cpuset=[73])
[New Thread 0x7fffdcf9d090 (LWP 140103)]
EAL: lcore 74 is ready (tid=7fffdd7ad090;cpuset=[74])
[New Thread 0x7fffbfffd090 (LWP 140104)]
EAL: lcore 75 is ready (tid=7fffdcf9d090;cpuset=[75])
EAL: lcore 76 is ready (tid=7fffbfffd090;cpuset=[76])
[New Thread 0x7fffbf7ed090 (LWP 140105)]
[New Thread 0x7fffbefdd090 (LWP 140106)]
EAL: lcore 77 is ready (tid=7fffbf7ed090;cpuset=[77])
[New Thread 0x7fffbe7cd090 (LWP 140107)]
EAL: lcore 78 is ready (tid=7fffbefdd090;cpuset=[78])
[New Thread 0x7fffbdfbd090 (LWP 140108)]
EAL: lcore 79 is ready (tid=7fffbe7cd090;cpuset=[79])
[New Thread 0x7fffbd7ad090 (LWP 140109)]
EAL: lcore 80 is ready (tid=7fffbdfbd090;cpuset=[80])
[New Thread 0x7fffbcf9d090 (LWP 140110)]
EAL: lcore 81 is ready (tid=7fffbd7ad090;cpuset=[81])
[New Thread 0x7fff9fffd090 (LWP 140111)]
EAL: lcore 82 is ready (tid=7fffbcf9d090;cpuset=[82])
[New Thread 0x7fff9f7ed090 (LWP 140112)]
EAL: lcore 83 is ready (tid=7fff9fffd090;cpuset=[83])
[New Thread 0x7fff9efdd090 (LWP 140113)]
EAL: lcore 84 is ready (tid=7fff9f7ed090;cpuset=[84])
[New Thread 0x7fff9e7cd090 (LWP 140114)]
EAL: lcore 85 is ready (tid=7fff9efdd090;cpuset=[85])
[New Thread 0x7fff9dfbd090 (LWP 140115)]
EAL: lcore 86 is ready (tid=7fff9e7cd090;cpuset=[86])
[New Thread 0x7fff9d7ad090 (LWP 140116)]
EAL: lcore 87 is ready (tid=7fff9dfbd090;cpuset=[87])
[New Thread 0x7fff9cf9d090 (LWP 140117)]
EAL: lcore 88 is ready (tid=7fff9d7ad090;cpuset=[88])
[New Thread 0x7fff7fffd090 (LWP 140118)]
EAL: lcore 89 is ready (tid=7fff9cf9d090;cpuset=[89])
[New Thread 0x7fff7f7ed090 (LWP 140119)]
EAL: lcore 90 is ready (tid=7fff7fffd090;cpuset=[90])
[New Thread 0x7fff7efdd090 (LWP 140120)]
EAL: lcore 91 is ready (tid=7fff7f7ed090;cpuset=[91])
EAL: lcore 92 is ready (tid=7fff7efdd090;cpuset=[92])
[New Thread 0x7fff7e7cd090 (LWP 140121)]
[New Thread 0x7fff7dfbd090 (LWP 140122)]
EAL: lcore 93 is ready (tid=7fff7e7cd090;cpuset=[93])
[New Thread 0x7fff7d7ad090 (LWP 140123)]
EAL: lcore 94 is ready (tid=7fff7dfbd090;cpuset=[94])
[New Thread 0x7fff7cf9d090 (LWP 140124)]
EAL: lcore 95 is ready (tid=7fff7d7ad090;cpuset=[95])
[New Thread 0x7fff5fffd090 (LWP 140125)]
EAL: lcore 96 is ready (tid=7fff7cf9d090;cpuset=[96])
[New Thread 0x7fff5f7ed090 (LWP 140126)]
EAL: lcore 97 is ready (tid=7fff5fffd090;cpuset=[97])
[New Thread 0x7fff5efdd090 (LWP 140127)]
EAL: lcore 98 is ready (tid=7fff5f7ed090;cpuset=[98])
[New Thread 0x7fff5e7cd090 (LWP 140128)]
EAL: lcore 99 is ready (tid=7fff5efdd090;cpuset=[99])
[New Thread 0x7fff5dfbd090 (LWP 140129)]
EAL: lcore 100 is ready (tid=7fff5e7cd090;cpuset=[100])
[New Thread 0x7fff5d7ad090 (LWP 140130)]
EAL: lcore 101 is ready (tid=7fff5dfbd090;cpuset=[101])
[New Thread 0x7fff5cf9d090 (LWP 140131)]
EAL: lcore 102 is ready (tid=7fff5d7ad090;cpuset=[102])
[New Thread 0x7fff3fffd090 (LWP 140132)]
EAL: lcore 103 is ready (tid=7fff5cf9d090;cpuset=[103])
[New Thread 0x7fff3f7ed090 (LWP 140133)]
EAL: lcore 104 is ready (tid=7fff3fffd090;cpuset=[104])
[New Thread 0x7fff3efdd090 (LWP 140134)]
EAL: lcore 105 is ready (tid=7fff3f7ed090;cpuset=[105])
[New Thread 0x7fff3e7cd090 (LWP 140135)]
EAL: lcore 106 is ready (tid=7fff3efdd090;cpuset=[106])
[New Thread 0x7fff3dfbd090 (LWP 140136)]
EAL: lcore 107 is ready (tid=7fff3e7cd090;cpuset=[107])
[New Thread 0x7fff3d7ad090 (LWP 140137)]
EAL: lcore 108 is ready (tid=7fff3dfbd090;cpuset=[108])
[New Thread 0x7fff3cf9d090 (LWP 140138)]
EAL: lcore 109 is ready (tid=7fff3d7ad090;cpuset=[109])
[New Thread 0x7fff1fffd090 (LWP 140139)]
EAL: lcore 110 is ready (tid=7fff3cf9d090;cpuset=[110])
[New Thread 0x7fff1f7ed090 (LWP 140140)]
EAL: lcore 111 is ready (tid=7fff1fffd090;cpuset=[111])
[New Thread 0x7fff1efdd090 (LWP 140141)]
EAL: lcore 112 is ready (tid=7fff1f7ed090;cpuset=[112])
[New Thread 0x7fff1e7cd090 (LWP 140142)]
EAL: lcore 113 is ready (tid=7fff1efdd090;cpuset=[113])
[New Thread 0x7fff1dfbd090 (LWP 140143)]
EAL: lcore 114 is ready (tid=7fff1e7cd090;cpuset=[114])
[New Thread 0x7fff1d7ad090 (LWP 140144)]
EAL: lcore 115 is ready (tid=7fff1dfbd090;cpuset=[115])
[New Thread 0x7fff1cf9d090 (LWP 140145)]
EAL: lcore 116 is ready (tid=7fff1d7ad090;cpuset=[116])
[New Thread 0x7ffeffffd090 (LWP 140146)]
EAL: lcore 117 is ready (tid=7fff1cf9d090;cpuset=[117])
[New Thread 0x7ffeff7ed090 (LWP 140147)]
EAL: lcore 118 is ready (tid=7ffeffffd090;cpuset=[118])
[New Thread 0x7ffefefdd090 (LWP 140148)]
EAL: lcore 119 is ready (tid=7ffeff7ed090;cpuset=[119])
[New Thread 0x7ffefe7cd090 (LWP 140149)]
EAL: lcore 120 is ready (tid=7ffefefdd090;cpuset=[120])
[New Thread 0x7ffefdfbd090 (LWP 140150)]
EAL: lcore 121 is ready (tid=7ffefe7cd090;cpuset=[121])
[New Thread 0x7ffefd7ad090 (LWP 140151)]
EAL: lcore 122 is ready (tid=7ffefdfbd090;cpuset=[122])
[New Thread 0x7ffefcf9d090 (LWP 140152)]
EAL: lcore 123 is ready (tid=7ffefd7ad090;cpuset=[123])
EAL: lcore 124 is ready (tid=7ffefcf9d090;cpuset=[124])
[New Thread 0x7ffedfffd090 (LWP 140153)]
[New Thread 0x7ffedf7ed090 (LWP 140154)]
EAL: lcore 125 is ready (tid=7ffedfffd090;cpuset=[125])
[New Thread 0x7ffedefdd090 (LWP 140155)]
EAL: lcore 126 is ready (tid=7ffedf7ed090;cpuset=[126])
EAL: lcore 127 is ready (tid=7ffedefdd090;cpuset=[127])
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 8
EAL: Restoring previous memory policy: 0
EAL: request: mp_malloc_sync
EAL: Heap on socket 8 was expanded by 1024MB
EAL: PCI device 0034:01:00.0 on NUMA socket 8
EAL:   probe driver: 8086:1583 net_i40e
EAL:   set IOMMU type 1 (Type 1) failed, error 19 (No such device)
EAL:   using IOMMU type 7 (sPAPR)
EAL: Highest VA address in memseg list is 0x2200000000
EAL: Setting DMA window size to 0x4000000000
EAL: Mem event callback 'vfio_mem_event_clb:(nil)' registered
EAL: Installed memory event callback for VFIO
EAL: VFIO reports MSI-X BAR as mappable
EAL:   PCI memory mapped at 0x2200000000
EAL:   PCI memory mapped at 0x2200800000
EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.0 (socket 8)
EAL: PCI device 0034:01:00.1 on NUMA socket 8
EAL:   probe driver: 8086:1583 net_i40e
EAL: VFIO reports MSI-X BAR as mappable
EAL:   PCI memory mapped at 0x2200810000
EAL:   PCI memory mapped at 0x2201010000
EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.1 (socket 8)
[New Thread 0x7ffede7cd090 (LWP 140156)]
[New Thread 0x7ffeddfbd090 (LWP 140157)]
APP: HPET is not enabled, using TSC as default timer
RTE>>vfio_autotest

Thread 1 "dpdk-test" hit Breakpoint 1, test_vfio () at 
../app/test/test_vfio.c:96
96		ret = test_memory_vfio_dma_map();
(gdb) s
test_memory_vfio_dma_map () at ../app/test/test_vfio.c:24
24		uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
(gdb) n
31		alloc_mem = malloc(sz * 2);
(gdb) p sz
$1 = 131072
(gdb) n
32		if (!alloc_mem) {
(gdb) n
39		memset(alloc_mem, 0, sz * 2);
(gdb) n
41		mem = RTE_PTR_ALIGN(alloc_mem, rte_mem_page_size());
(gdb) p/x mem
$2 = 0x12c37400
(gdb) n
44		ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
(gdb) p/x mem
$3 = 0x24d20000
(gdb) s
rte_vfio_container_dma_map (container_fd=-1, vaddr=617742336, 
iova=617742336, len=131072)
     at ../lib/librte_eal/linux/eal_vfio.c:2038
2038		if (len == 0) {
(gdb) p/x iova
$4 = 0x24d20000
(gdb) n
2043		vfio_cfg = get_vfio_cfg_by_container_fd(container_fd);
(gdb) n
2044		if (vfio_cfg == NULL) {
(gdb) n
2049		return container_dma_map(vfio_cfg, vaddr, iova, len);
(gdb) s
container_dma_map (vfio_cfg=0x23faba88 <vfio_cfgs>, vaddr=617742336, 
iova=617742336, len=131072)
     at ../lib/librte_eal/linux/eal_vfio.c:1784
1784		int ret = 0;
(gdb) n
1786		user_mem_maps = &vfio_cfg->mem_maps;
(gdb) n
1787		rte_spinlock_recursive_lock(&user_mem_maps->lock);
(gdb) n
1788		if (user_mem_maps->n_maps == VFIO_MAX_USER_MEM_MAPS) {
(gdb) n
1795		if (vfio_dma_mem_map(vfio_cfg, vaddr, iova, len, 1)) {
(gdb) s
vfio_dma_mem_map (vfio_cfg=0x23faba88 <vfio_cfgs>, vaddr=617742336, 
iova=617742336, len=131072, do_map=1)
     at ../lib/librte_eal/linux/eal_vfio.c:1758
1758		const struct vfio_iommu_type *t = vfio_cfg->vfio_iommu_type;
(gdb) n
1760		if (!t) {
(gdb) n
1766		if (!t->dma_user_map_func) {
(gdb) n
1774		return t->dma_user_map_func(vfio_cfg->vfio_container_fd, vaddr, iova,
(gdb) s
vfio_spapr_dma_mem_map (vfio_container_fd=11, vaddr=617742336, 
iova=617742336, len=131072, do_map=1)
     at ../lib/librte_eal/linux/eal_vfio.c:1703
1703		int ret = 0;
(gdb) n
1705		if (do_map) {
(gdb) n
1706			if (vfio_spapr_dma_do_map(vfio_container_fd,
(gdb) s
vfio_spapr_dma_do_map (vfio_container_fd=11, vaddr=617742336, 
iova=617742336, len=131072, do_map=1)
     at ../lib/librte_eal/linux/eal_vfio.c:1399
1399		struct vfio_iommu_spapr_register_memory reg = {
(gdb) n
1407		if (do_map != 0) {
(gdb) n
1410			if (iova + len > spapr_dma_win_len) {
(gdb) p/x iova
$5 = 0x24d20000
(gdb) p/x spapr_dma_win_len
$6 = 0x4000000000
(gdb) n
1415			ret = ioctl(vfio_container_fd,
(gdb) n
1417			if (ret) {
(gdb) p ret
$7 = 0
(gdb) n
1423			memset(&dma_map, 0, sizeof(dma_map));
(gdb) n
1424			dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
(gdb) n
1425			dma_map.vaddr = vaddr;
(gdb) n
1426			dma_map.size = len;
(gdb) n
1427			dma_map.iova = iova;
(gdb) n
1428			dma_map.flags = VFIO_DMA_MAP_FLAG_READ |
(gdb) n
1431			ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA, &dma_map);
(gdb) p/x dma_map
$8 = {argsz = 0x20, flags = 0x3, vaddr = 0x24d20000, iova = 0x24d20000, 
size = 0x20000}
(gdb) n
1432			if (ret) {
(gdb) p ret
$9 = -1
(gdb) n
1433				RTE_LOG(ERR, EAL, "  cannot map vaddr for IOMMU, error %i (%s)\n",
(gdb) n
EAL:   cannot map vaddr for IOMMU, error 22 (Invalid argument)
1435				return -1;
(gdb) quit

  reply	other threads:[~2021-01-04 22:29 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-12  8:11 [dpdk-dev] [PATCH 0/2] " Nithin Dabilpuram
2020-10-12  8:11 ` [dpdk-dev] [PATCH 1/2] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-10-14 14:39   ` Burakov, Anatoly
2020-10-15  9:54     ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-10-12  8:11 ` [dpdk-dev] [PATCH 2/2] vfio: fix partial DMA unmapping for VFIO type1 Nithin Dabilpuram
2020-10-14 15:07   ` Burakov, Anatoly
2020-10-15  6:09     ` [dpdk-dev] [EXT] " Nithin Dabilpuram
2020-10-15 10:00       ` Burakov, Anatoly
2020-10-15 11:38         ` Nithin Dabilpuram
2020-10-15 11:50         ` Nithin Dabilpuram
2020-10-15 11:57         ` Nithin Dabilpuram
2020-10-15 15:10           ` Burakov, Anatoly
2020-10-16  7:10             ` Nithin Dabilpuram
2020-10-17 16:14               ` Burakov, Anatoly
2020-10-19  9:43                 ` Nithin Dabilpuram
2020-10-22 12:13                   ` Nithin Dabilpuram
2020-10-28 13:04                     ` Burakov, Anatoly
2020-10-28 14:17                       ` Nithin Dabilpuram
2020-10-28 16:07                         ` Burakov, Anatoly
2020-10-28 16:31                           ` Nithin Dabilpuram
2020-11-05  9:04 ` [dpdk-dev] [PATCH v2 0/3] fix issue with partial DMA unmap Nithin Dabilpuram
2020-11-05  9:04   ` [dpdk-dev] [PATCH v2 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2020-11-05  9:04   ` [dpdk-dev] [PATCH v2 2/3] vfio: fix DMA mapping granularity for type1 iova as va Nithin Dabilpuram
2020-11-10 14:04     ` Burakov, Anatoly
2020-11-10 14:22       ` Burakov, Anatoly
2020-11-10 14:17     ` Burakov, Anatoly
2020-11-11  5:08       ` Nithin Dabilpuram
2020-11-11 10:00         ` Burakov, Anatoly
2020-11-05  9:04   ` [dpdk-dev] [PATCH v2 3/3] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-01 19:32 ` [dpdk-dev] [PATCH v3 0/4] fix issue with partial DMA unmap Nithin Dabilpuram
2020-12-01 19:32   ` [dpdk-dev] [PATCH v3 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2020-12-01 19:33   ` [dpdk-dev] [PATCH v3 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2020-12-01 19:33   ` [dpdk-dev] [PATCH v3 3/4] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-01 19:33   ` [dpdk-dev] [PATCH v3 4/4] test: change external memory test to use system page sz Nithin Dabilpuram
2020-12-01 23:23     ` David Christensen
2020-12-02  5:40       ` Nithin Dabilpuram
2020-12-02  5:46 ` [dpdk-dev] [PATCH v4 0/4] fix issue with partial DMA unmap Nithin Dabilpuram
2020-12-02  5:46   ` [dpdk-dev] [PATCH v4 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2020-12-02 18:36     ` David Christensen
2020-12-02  5:46   ` [dpdk-dev] [PATCH v4 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2020-12-02 18:38     ` David Christensen
2020-12-02  5:46   ` [dpdk-dev] [PATCH v4 3/4] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-02 19:23     ` David Christensen
2020-12-03  7:14       ` Nithin Dabilpuram
2020-12-14  8:24         ` Nithin Dabilpuram
2020-12-02  5:46   ` [dpdk-dev] [PATCH v4 4/4] test: change external memory test to use system page sz Nithin Dabilpuram
2020-12-14  8:19 ` [dpdk-dev] [PATCH v5 0/4] fix issue with partial DMA unmap Nithin Dabilpuram
2020-12-14  8:19   ` [dpdk-dev] [PATCH v5 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2020-12-14  8:19   ` [dpdk-dev] [PATCH v5 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2020-12-14  8:19   ` [dpdk-dev] [PATCH v5 3/4] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-14  8:19   ` [dpdk-dev] [PATCH v5 4/4] test: change external memory test to use system page sz Nithin Dabilpuram
2020-12-17 19:06 ` [dpdk-dev] [PATCH v6 0/4] fix issue with partial DMA unmap Nithin Dabilpuram
2020-12-17 19:06   ` [dpdk-dev] [PATCH v6 1/4] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2020-12-17 19:06   ` [dpdk-dev] [PATCH v6 2/4] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2020-12-17 19:06   ` [dpdk-dev] [PATCH v6 3/4] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-17 19:10     ` Nithin Dabilpuram
2021-01-05 19:33       ` David Christensen
2021-01-06  8:40         ` Nithin Dabilpuram
2021-01-06 21:20           ` David Christensen
2020-12-17 19:06   ` [dpdk-dev] [PATCH v6 4/4] test: change external memory test to use system page sz Nithin Dabilpuram
2020-12-23  5:13   ` [dpdk-dev] [PATCH v6 0/4] fix issue with partial DMA unmap Nithin Dabilpuram
2021-01-04 22:29     ` David Christensen [this message]
2021-01-12 17:39 ` [dpdk-dev] [PATCH v7 0/3] " Nithin Dabilpuram
2021-01-12 17:39   ` [dpdk-dev] [PATCH v7 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2021-01-12 17:39   ` [dpdk-dev] [PATCH v7 2/3] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2021-01-12 17:39   ` [dpdk-dev] [PATCH v7 3/3] test: change external memory test to use system page sz Nithin Dabilpuram
2021-01-14 16:30     ` David Marchand
2021-01-15  6:57       ` Nithin Dabilpuram
2021-01-15  7:32 ` [dpdk-dev] [PATCH v8 0/3] fix issue with partial DMA unmap Nithin Dabilpuram
2021-01-15  7:32   ` [dpdk-dev] [PATCH v8 1/3] vfio: revert changes for map contiguous areas in one go Nithin Dabilpuram
2021-03-05  7:50     ` David Marchand
2021-03-05 13:54       ` Burakov, Anatoly
2021-03-05 15:50         ` Nithin Dabilpuram
2021-04-01 11:27           ` Burakov, Anatoly
2021-01-15  7:32   ` [dpdk-dev] [PATCH v8 2/3] vfio: fix DMA mapping granularity for type1 IOVA as VA Nithin Dabilpuram
2021-01-15  7:32   ` [dpdk-dev] [PATCH v8 3/3] test: change external memory test to use system page sz Nithin Dabilpuram
2021-02-11 11:21     ` Burakov, Anatoly
2021-02-16 13:14   ` [dpdk-dev] [PATCH v8 0/3] fix issue with partial DMA unmap Burakov, Anatoly
2021-02-22  9:41     ` Nithin Dabilpuram
2021-02-22 10:06       ` David Marchand
2021-03-01 12:13   ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7bf59410-6d2b-e3b2-bb98-4ca9290e1abb@linux.vnet.ibm.com \
    --to=drc@linux.vnet.ibm.com \
    --cc=anatoly.burakov@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinj@marvell.com \
    --cc=nithind1988@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).