* [dpdk-dev] Question about “PCI memorymmap”
@ 2020-07-08 1:26 陈毅强
0 siblings, 0 replies; 4+ messages in thread
From: 陈毅强 @ 2020-07-08 1:26 UTC (permalink / raw)
To: dev, Neil Horman, Bruce Richardson, Thomas Monjalon, robin.jarry,
Tal Shnaiderman, Dmitry Kozlyuk
Hi,ALL
We fail to start dpdk in a newly deployed machine.Failed progress works in proc-type=secondary mode..Some logs are shown below
=========================================================================================
Primary progress:
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fffec3ff000
EAL: PCI memory mapped at 0x7ffff7ff4000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.1 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffed25d9000
EAL: PCI memory mapped at 0x7ffff7ff0000
Secondary progress logs:
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=0.
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: ixgbe_set_tx_function(): Using simple tx code path
PMD: ixgbe_set_rx_function(): Port[1] doesn't meet Vector Rx preconditions or RTE_IXGBE_INC_VECTOR is not enabled
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=1.
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff65d8000
EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:82:00.0/resource0 to address: 0x7fffec3ff000
EAL: Error - exiting with code: 1
Cause: Requested device 0000:82:00.0 cannot be used
==================================================================================
Relevant code is list be below:
pci_uio_map_secondary(struct rte_pci_device *dev)
{
int fd, i;
struct mapped_pci_resource *uio_res;
struct mapped_pci_res_list *uio_res_list =
RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list);
TAILQ_FOREACH(uio_res, uio_res_list, next) {
/* skip this element if it doesn't match our PCI address */
if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
continue;
for (i = 0; i != uio_res->nb_maps; i++) {
/*
* open devname, to mmap it
*/
fd = open(uio_res->maps[i].path, O_RDWR);
if (fd < 0) {
RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
uio_res->maps[i].path, strerror(errno));
return -1;
}
void *mapaddr = pci_map_resource(uio_res->maps[i].addr,
fd, (off_t)uio_res->maps[i].offset,
(size_t)uio_res->maps[i].size, 0);
/* fd is not needed in slave process, close it */
close(fd);
if (mapaddr != uio_res->maps[i].addr) {
RTE_LOG(ERR, EAL,
"Cannot mmap device resource file %s to address: %p\n",
uio_res->maps[i].path,
uio_res->maps[i].addr);
return -1;
}
}
return 0;
}
RTE_LOG(ERR, EAL, "Cannot find resource for device\n");
return 1;
}
And We check the system config . ASLR is not enabled on our system. I wondered why mapaddr != uio_res->maps[i].addr .Can anyone tell me why?
Thanks in advance.
^ permalink raw reply [flat|nested] 4+ messages in thread* [dpdk-dev] Question about “PCI memorymmap”
@ 2020-07-02 2:49 陈毅强
2020-09-25 15:15 ` Burakov, Anatoly
0 siblings, 1 reply; 4+ messages in thread
From: 陈毅强 @ 2020-07-02 2:49 UTC (permalink / raw)
To: dev
Hi,ALL
We fail to start dpdk in a newly deployed machine.Failed progress works in proc-type=secondary mode. (DPDK version 16.04).Some logs are shown below
=========================================================================================
Primary progress:
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fffec3ff000
EAL: PCI memory mapped at 0x7ffff7ff4000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.1 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffed25d9000
EAL: PCI memory mapped at 0x7ffff7ff0000
Secondary progress logs:
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=0.
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: ixgbe_set_tx_function(): Using simple tx code path
PMD: ixgbe_set_rx_function(): Port[1] doesn't meet Vector Rx preconditions or RTE_IXGBE_INC_VECTOR is not enabled
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=1.
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff65d8000
EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:82:00.0/resource0 to address: 0x7fffec3ff000
EAL: Error - exiting with code: 1
Cause: Requested device 0000:82:00.0 cannot be used
==================================================================================
Relevant code is list be below:
pci_uio_map_secondary(struct rte_pci_device *dev)
{
int fd, i;
struct mapped_pci_resource *uio_res;
struct mapped_pci_res_list *uio_res_list =
RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list);
TAILQ_FOREACH(uio_res, uio_res_list, next) {
/* skip this element if it doesn't match our PCI address */
if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
continue;
for (i = 0; i != uio_res->nb_maps; i++) {
/*
* open devname, to mmap it
*/
fd = open(uio_res->maps[i].path, O_RDWR);
if (fd < 0) {
RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
uio_res->maps[i].path, strerror(errno));
return -1;
}
void *mapaddr = pci_map_resource(uio_res->maps[i].addr,
fd, (off_t)uio_res->maps[i].offset,
(size_t)uio_res->maps[i].size, 0);
/* fd is not needed in slave process, close it */
close(fd);
if (mapaddr != uio_res->maps[i].addr) {
RTE_LOG(ERR, EAL,
"Cannot mmap device resource file %s to address: %p\n",
uio_res->maps[i].path,
uio_res->maps[i].addr);
return -1;
}
}
return 0;
}
RTE_LOG(ERR, EAL, "Cannot find resource for device\n");
return 1;
}
And We check the system config . ASLR is not enabled on our system. I wondered why mapaddr != uio_res->maps[i].addr .Can anyone tell me why?
Thanks in advance.
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [dpdk-dev] Question about “PCI memorymmap”
2020-07-02 2:49 陈毅强
@ 2020-09-25 15:15 ` Burakov, Anatoly
0 siblings, 0 replies; 4+ messages in thread
From: Burakov, Anatoly @ 2020-09-25 15:15 UTC (permalink / raw)
To: 陈毅强, dev
On 02-Jul-20 3:49 AM, 陈毅强 wrote:
> Hi,ALL
>
> We fail to start dpdk in a newly deployed machine.Failed progress works in proc-type=secondary mode. (DPDK version 16.04).Some logs are shown below
>
> =========================================================================================
>
> Primary progress:
>
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI memory mapped at 0x7ffff5c04000
> EAL: PCI memory mapped at 0x7ffff6004000
> PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
> PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
> EAL: PCI device 0000:82:00.0 on NUMA socket 1
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI memory mapped at 0x7fffec3ff000
> EAL: PCI memory mapped at 0x7ffff7ff4000
> PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
> PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
> EAL: PCI device 0000:82:00.1 on NUMA socket 1
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI memory mapped at 0x7ffed25d9000
> EAL: PCI memory mapped at 0x7ffff7ff0000
>
>
>
>
> Secondary progress logs:
>
>
> PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=0.
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI memory mapped at 0x7ffff5c04000
> EAL: PCI memory mapped at 0x7ffff6004000
> PMD: ixgbe_set_tx_function(): Using simple tx code path
> PMD: ixgbe_set_rx_function(): Port[1] doesn't meet Vector Rx preconditions or RTE_IXGBE_INC_VECTOR is not enabled
> PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=1.
> EAL: PCI device 0000:82:00.0 on NUMA socket 1
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: PCI memory mapped at 0x7ffff65d8000
> EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:82:00.0/resource0 to address: 0x7fffec3ff000
> EAL: Error - exiting with code: 1
> Cause: Requested device 0000:82:00.0 cannot be used
>
> ==================================================================================
>
> Relevant code is list be below:
>
>
>
>
> pci_uio_map_secondary(struct rte_pci_device *dev)
> {
> int fd, i;
> struct mapped_pci_resource *uio_res;
> struct mapped_pci_res_list *uio_res_list =
> RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list);
>
>
> TAILQ_FOREACH(uio_res, uio_res_list, next) {
>
>
> /* skip this element if it doesn't match our PCI address */
> if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
> continue;
>
>
> for (i = 0; i != uio_res->nb_maps; i++) {
> /*
> * open devname, to mmap it
> */
> fd = open(uio_res->maps[i].path, O_RDWR);
> if (fd < 0) {
> RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
> uio_res->maps[i].path, strerror(errno));
> return -1;
> }
>
>
> void *mapaddr = pci_map_resource(uio_res->maps[i].addr,
> fd, (off_t)uio_res->maps[i].offset,
> (size_t)uio_res->maps[i].size, 0);
> /* fd is not needed in slave process, close it */
> close(fd);
> if (mapaddr != uio_res->maps[i].addr) {
> RTE_LOG(ERR, EAL,
> "Cannot mmap device resource file %s to address: %p\n",
> uio_res->maps[i].path,
> uio_res->maps[i].addr);
> return -1;
> }
> }
> return 0;
> }
>
>
> RTE_LOG(ERR, EAL, "Cannot find resource for device\n");
> return 1;
> }
>
>
>
>
>
>
>
> And We check the system config . ASLR is not enabled on our system. I wondered why mapaddr != uio_res->maps[i].addr .Can anyone tell me why?
>
>
>
>
>
>
>
> Thanks in advance.
>
Hi,
This is a known issue in DPDK and unfortunately it's not really
solvable. I've made attempts in the past to address it with an RFC [1]
patchset to enable deterministic hotplug, but it seems that this isn't a
problem the majority care about enough to fix it.
You might want to try with different --base-virtaddr values, but the
root cause is that we do not control our memory address space, kernel
does. Since secondary processes are predicated upon duplicating primary
process's address space, this inevitably means that sometimes the
duplication isn't possible because something else has already taken the
address, and we cannot do anything about it.
Enabling/disabling ASLR will not help or hurt with this issue; rather,
it'll make it more deterministic (i.e. it'll either always work or
always fail). So, playing around with various values for --base-virtaddr
is pretty much your only option at this point.
[1] http://patches.dpdk.org/bundle/aburakov/reliable_device_map/
--
Thanks,
Anatoly
^ permalink raw reply [flat|nested] 4+ messages in thread
* [dpdk-dev] Question about “PCI memorymmap”
@ 2020-07-02 2:43 陈毅强
0 siblings, 0 replies; 4+ messages in thread
From: 陈毅强 @ 2020-07-02 2:43 UTC (permalink / raw)
To: dev
Hi,ALL
We fail to start dpdk in a newly deployed machine.Failed progress works in proc-type=secondary mode. (DPDK version 16.04).Some logs are shown below
=========================================================================================
Primary progress:
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fffec3ff000
EAL: PCI memory mapped at 0x7ffff7ff4000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.1 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffed25d9000
EAL: PCI memory mapped at 0x7ffff7ff0000
Secondary progress logs:
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=0.
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: ixgbe_set_tx_function(): Using simple tx code path
PMD: ixgbe_set_rx_function(): Port[1] doesn't meet Vector Rx preconditions or RTE_IXGBE_INC_VECTOR is not enabled
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=1.
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff65d8000
EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:82:00.0/resource0 to address: 0x7fffec3ff000
EAL: Error - exiting with code: 1
Cause: Requested device 0000:82:00.0 cannot be used
==================================================================================
Relevant code is list be below:
pci_uio_map_secondary(struct rte_pci_device *dev)
{
int fd, i;
struct mapped_pci_resource *uio_res;
struct mapped_pci_res_list *uio_res_list =
RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list);
TAILQ_FOREACH(uio_res, uio_res_list, next) {
/* skip this element if it doesn't match our PCI address */
if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
continue;
for (i = 0; i != uio_res->nb_maps; i++) {
/*
* open devname, to mmap it
*/
fd = open(uio_res->maps[i].path, O_RDWR);
if (fd < 0) {
RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
uio_res->maps[i].path, strerror(errno));
return -1;
}
void *mapaddr = pci_map_resource(uio_res->maps[i].addr,
fd, (off_t)uio_res->maps[i].offset,
(size_t)uio_res->maps[i].size, 0);
/* fd is not needed in slave process, close it */
close(fd);
if (mapaddr != uio_res->maps[i].addr) {
RTE_LOG(ERR, EAL,
"Cannot mmap device resource file %s to address: %p\n",
uio_res->maps[i].path,
uio_res->maps[i].addr);
return -1;
}
}
return 0;
}
RTE_LOG(ERR, EAL, "Cannot find resource for device\n");
return 1;
}
And We check the system config . ASLR is not enabled on our system. I wondered why mapaddr != uio_res->maps[i].addr .Can anyone tell me why?
Thanks in advance.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-09-25 15:15 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-08 1:26 [dpdk-dev] Question about “PCI memorymmap” 陈毅强
-- strict thread matches above, loose matches on Subject: below --
2020-07-02 2:49 陈毅强
2020-09-25 15:15 ` Burakov, Anatoly
2020-07-02 2:43 陈毅强
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).