* [dpdk-dev] [Bug 580] [dpdk-20.11] testpmd: Ubuntu 16.04: Failed to start port as mlx5_pci can't allocate hash list.
@ 2020-11-19 17:45 6% bugzilla
0 siblings, 0 replies; 15+ results
From: bugzilla @ 2020-11-19 17:45 UTC (permalink / raw)
To: dev
https://bugs.dpdk.org/show_bug.cgi?id=580
Bug ID: 580
Summary: [dpdk-20.11] testpmd: Ubuntu 16.04: Failed to start
port as mlx5_pci can't allocate hash list.
Product: DPDK
Version: 20.11
Hardware: x86
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: testpmd
Assignee: dev@dpdk.org
Reporter: abhimarathe17@gmail.com
Target Milestone: ---
Hi,
Facing issue with Ubuntu 16.04 & Ubuntu 18.04 with mlx5 driver on Azure
platform.
For Ubuntu 16.04(4.15.0-1098-azure), testpmd failed to start port. mlx5_pci:
Can't allocate hash list mlx5_1_flow_table entry.
testpmd output given below:
EAL: Probing VFIO support...
EAL: Probe PCI driver: mlx5_pci (15b3:1016) device: 40db:00:02.0 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
net_vdev_netvsc: probably using routed NetVSC interface "eth1" (index 3)
EAL: No legacy callbacks, legacy socket not created
Set io packet forwarding mode
Warning: NUMA should be configured manually by using --port-numa-config and
--ring-numa-config parameters along with --numa.
testpmd: create a new mbuf pool <mb_pool_0>: n=180224, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 1 (socket 0)
mlx5_pci: Failed to init cache list NIC_ingress_0_matcher_cache entry (nil).
mlx5_pci: port 0 failed to set defaults flows
Fail to start port 1
Please stop the ports first
Done
For Ubuntu 18.04(5.4.0-1031-azure),Getting similar errors but ports are not
blocked/stopped.
testpmd output given below:
--------------------------mlx5-----------------------------------------
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1016) device: 0002:00:02.0 (socket 0)
mlx5_pci: Retrying to allocate Rx DevX UAR
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
mlx5_pci: Can't allocate hash list mlx5_1_flow_table entry.
net_vdev_netvsc: probably using routed NetVSC interface "eth1" (index 3)
tap_nl_dump_ext_ack(): Cannot delete qdisc with handle of zero
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
tap_nl_dump_ext_ack(): Failed to find qdisc with specified classid
EAL: No legacy callbacks, legacy socket not created
Set txonly packet forwarding mode
Warning: NUMA should be configured manually by using --port-numa-config and
--ring-numa-config parameters along with --numa.
testpmd: create a new mbuf pool <mb_pool_0>: n=163840, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 1 (socket 0)
Port 1: 00:0D:3A:C4:E8:BB
Checking link statuses...
Done
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v4 3/4] test: add test case to validate VFIO DMA map/unmap
@ 2020-12-02 19:23 6% ` David Christensen
2020-12-03 7:14 0% ` Nithin Dabilpuram
0 siblings, 1 reply; 15+ results
From: David Christensen @ 2020-12-02 19:23 UTC (permalink / raw)
To: Nithin Dabilpuram, anatoly.burakov, david.marchand; +Cc: jerinj, dev
On 12/1/20 9:46 PM, Nithin Dabilpuram wrote:
> Test case mmap's system pages and tries to performs a user
> DMA map and unmap both partially and fully.
>
> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> app/test/meson.build | 1 +
> app/test/test_vfio.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 104 insertions(+)
> create mode 100644 app/test/test_vfio.c
>
> diff --git a/app/test/meson.build b/app/test/meson.build
> index 94fd39f..d9eedb6 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -139,6 +139,7 @@ test_sources = files('commands.c',
> 'test_trace_register.c',
> 'test_trace_perf.c',
> 'test_version.c',
> + 'test_vfio.c',
> 'virtual_pmd.c'
> )
>
> diff --git a/app/test/test_vfio.c b/app/test/test_vfio.c
> new file mode 100644
> index 0000000..00626d4
> --- /dev/null
> +++ b/app/test/test_vfio.c
> @@ -0,0 +1,103 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(C) 2020 Marvell.
> + */
> +
> +#include <stdio.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <sys/mman.h>
> +#include <unistd.h>
> +
> +#include <rte_common.h>
> +#include <rte_eal.h>
> +#include <rte_eal_paging.h>
> +#include <rte_errno.h>
> +#include <rte_memory.h>
> +#include <rte_vfio.h>
> +
> +#include "test.h"
> +
> +static int
> +test_memory_vfio_dma_map(void)
> +{
> + uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
> + uint64_t unmap1, unmap2;
> + uint8_t *mem;
> + int ret;
> +
> + /* Allocate twice size of page */
> + mem = mmap(NULL, sz, PROT_READ | PROT_WRITE,
> + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> + if (mem == MAP_FAILED) {
> + printf("Failed to allocate memory for external heap\n");
> + return -1;
> + }
> +
> + /* Force page allocation */
> + memset(mem, 0, sz);
> +
> + /* map the whole region */
> + ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
> + (uintptr_t)mem, (rte_iova_t)mem, sz);
> + if (ret) {
> + /* Check if VFIO is not available or no device is probed */
> + if (rte_errno == ENOTSUP || rte_errno == ENODEV) {
> + ret = 1;
> + goto fail;
> + }
> + printf("Failed to dma map whole region, ret=%d(%s)\n",
> + ret, rte_strerror(rte_errno));
> + goto fail;
> + }
> +
> + unmap1 = (uint64_t)mem + (sz / 2);
> + sz1 = sz / 2;
> + unmap2 = (uint64_t)mem;
> + sz2 = sz / 2;
> + /* unmap the partial region */
> + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> + unmap1, (rte_iova_t)unmap1, sz1);
> + if (ret) {
> + if (rte_errno == ENOTSUP) {
> + printf("Partial dma unmap not supported\n");
> + unmap2 = (uint64_t)mem;
> + sz2 = sz;
> + } else {
> + printf("Failed to unmap second half region, ret=%d(%s)\n",
> + ret, rte_strerror(rte_errno));
> + goto fail;
> + }
> + }
> +
> + /* unmap the remaining region */
> + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> + unmap2, (rte_iova_t)unmap2, sz2);
> + if (ret) {
> + printf("Failed to unmap remaining region, ret=%d(%s)\n", ret,
> + rte_strerror(rte_errno));
> + goto fail;
> + }
> +
> +fail:
> + munmap(mem, sz);
> + return ret;
> +}
> +
> +static int
> +test_vfio(void)
> +{
> + int ret;
> +
> + /* test for vfio dma map/unmap */
> + ret = test_memory_vfio_dma_map();
> + if (ret == 1) {
> + printf("VFIO dma map/unmap unsupported\n");
> + } else if (ret < 0) {
> + printf("Error vfio dma map/unmap, ret=%d\n", ret);
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +REGISTER_TEST_COMMAND(vfio_autotest, test_vfio);
>
The test as written fails on a POWER9 system (see below for debug output).
IOMMU on POWER systems requires that a DMA window be defined and that
all DMA mappings reside within that window. In this test, the DMA
window is defined as 0x0 to 0x4000000000, but the VA allocated in your
test is 0x7fffb8680000, well outside that range.
I recently submitted a change in the 20.11 release which scans the
memseg list in order to set the DMA window. That test can be seen here:
EAL: Highest VA address in memseg list is 0x2200000000
EAL: Setting DMA window size to 0x4000000000
Can we modify the test to allocate memory out of the exsitng memseg
allocations?
Dave
$ sudo ~/src/dpdk/build/app/test/dpdk-test --log="eal,debug"
--iova-mode=va -l 64-127
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 0 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 4 on socket 0
EAL: Detected lcore 6 as core 4 on socket 0
EAL: Detected lcore 7 as core 4 on socket 0
EAL: Detected lcore 8 as core 8 on socket 0
EAL: Detected lcore 9 as core 8 on socket 0
EAL: Detected lcore 10 as core 8 on socket 0
EAL: Detected lcore 11 as core 8 on socket 0
EAL: Detected lcore 12 as core 12 on socket 0
EAL: Detected lcore 13 as core 12 on socket 0
EAL: Detected lcore 14 as core 12 on socket 0
EAL: Detected lcore 15 as core 12 on socket 0
EAL: Detected lcore 16 as core 16 on socket 0
EAL: Detected lcore 17 as core 16 on socket 0
EAL: Detected lcore 18 as core 16 on socket 0
EAL: Detected lcore 19 as core 16 on socket 0
EAL: Detected lcore 20 as core 20 on socket 0
EAL: Detected lcore 21 as core 20 on socket 0
EAL: Detected lcore 22 as core 20 on socket 0
EAL: Detected lcore 23 as core 20 on socket 0
EAL: Detected lcore 24 as core 24 on socket 0
EAL: Detected lcore 25 as core 24 on socket 0
EAL: Detected lcore 26 as core 24 on socket 0
EAL: Detected lcore 27 as core 24 on socket 0
EAL: Detected lcore 28 as core 28 on socket 0
EAL: Detected lcore 29 as core 28 on socket 0
EAL: Detected lcore 30 as core 28 on socket 0
EAL: Detected lcore 31 as core 28 on socket 0
EAL: Detected lcore 32 as core 32 on socket 0
EAL: Detected lcore 33 as core 32 on socket 0
EAL: Detected lcore 34 as core 32 on socket 0
EAL: Detected lcore 35 as core 32 on socket 0
EAL: Detected lcore 36 as core 36 on socket 0
EAL: Detected lcore 37 as core 36 on socket 0
EAL: Detected lcore 38 as core 36 on socket 0
EAL: Detected lcore 39 as core 36 on socket 0
EAL: Detected lcore 40 as core 48 on socket 0
EAL: Detected lcore 41 as core 48 on socket 0
EAL: Detected lcore 42 as core 48 on socket 0
EAL: Detected lcore 43 as core 48 on socket 0
EAL: Detected lcore 44 as core 52 on socket 0
EAL: Detected lcore 45 as core 52 on socket 0
EAL: Detected lcore 46 as core 52 on socket 0
EAL: Detected lcore 47 as core 52 on socket 0
EAL: Detected lcore 48 as core 72 on socket 0
EAL: Detected lcore 49 as core 72 on socket 0
EAL: Detected lcore 50 as core 72 on socket 0
EAL: Detected lcore 51 as core 72 on socket 0
EAL: Detected lcore 52 as core 76 on socket 0
EAL: Detected lcore 53 as core 76 on socket 0
EAL: Detected lcore 54 as core 76 on socket 0
EAL: Detected lcore 55 as core 76 on socket 0
EAL: Detected lcore 56 as core 80 on socket 0
EAL: Detected lcore 57 as core 80 on socket 0
EAL: Detected lcore 58 as core 80 on socket 0
EAL: Detected lcore 59 as core 80 on socket 0
EAL: Detected lcore 60 as core 84 on socket 0
EAL: Detected lcore 61 as core 84 on socket 0
EAL: Detected lcore 62 as core 84 on socket 0
EAL: Detected lcore 63 as core 84 on socket 0
EAL: Detected lcore 64 as core 2048 on socket 8
EAL: Detected lcore 65 as core 2048 on socket 8
EAL: Detected lcore 66 as core 2048 on socket 8
EAL: Detected lcore 67 as core 2048 on socket 8
EAL: Detected lcore 68 as core 2052 on socket 8
EAL: Detected lcore 69 as core 2052 on socket 8
EAL: Detected lcore 70 as core 2052 on socket 8
EAL: Detected lcore 71 as core 2052 on socket 8
EAL: Detected lcore 72 as core 2056 on socket 8
EAL: Detected lcore 73 as core 2056 on socket 8
EAL: Detected lcore 74 as core 2056 on socket 8
EAL: Detected lcore 75 as core 2056 on socket 8
EAL: Detected lcore 76 as core 2060 on socket 8
EAL: Detected lcore 77 as core 2060 on socket 8
EAL: Detected lcore 78 as core 2060 on socket 8
EAL: Detected lcore 79 as core 2060 on socket 8
EAL: Detected lcore 80 as core 2072 on socket 8
EAL: Detected lcore 81 as core 2072 on socket 8
EAL: Detected lcore 82 as core 2072 on socket 8
EAL: Detected lcore 83 as core 2072 on socket 8
EAL: Detected lcore 84 as core 2076 on socket 8
EAL: Detected lcore 85 as core 2076 on socket 8
EAL: Detected lcore 86 as core 2076 on socket 8
EAL: Detected lcore 87 as core 2076 on socket 8
EAL: Detected lcore 88 as core 2080 on socket 8
EAL: Detected lcore 89 as core 2080 on socket 8
EAL: Detected lcore 90 as core 2080 on socket 8
EAL: Detected lcore 91 as core 2080 on socket 8
EAL: Detected lcore 92 as core 2084 on socket 8
EAL: Detected lcore 93 as core 2084 on socket 8
EAL: Detected lcore 94 as core 2084 on socket 8
EAL: Detected lcore 95 as core 2084 on socket 8
EAL: Detected lcore 96 as core 2088 on socket 8
EAL: Detected lcore 97 as core 2088 on socket 8
EAL: Detected lcore 98 as core 2088 on socket 8
EAL: Detected lcore 99 as core 2088 on socket 8
EAL: Detected lcore 100 as core 2092 on socket 8
EAL: Detected lcore 101 as core 2092 on socket 8
EAL: Detected lcore 102 as core 2092 on socket 8
EAL: Detected lcore 103 as core 2092 on socket 8
EAL: Detected lcore 104 as core 2096 on socket 8
EAL: Detected lcore 105 as core 2096 on socket 8
EAL: Detected lcore 106 as core 2096 on socket 8
EAL: Detected lcore 107 as core 2096 on socket 8
EAL: Detected lcore 108 as core 2100 on socket 8
EAL: Detected lcore 109 as core 2100 on socket 8
EAL: Detected lcore 110 as core 2100 on socket 8
EAL: Detected lcore 111 as core 2100 on socket 8
EAL: Detected lcore 112 as core 2120 on socket 8
EAL: Detected lcore 113 as core 2120 on socket 8
EAL: Detected lcore 114 as core 2120 on socket 8
EAL: Detected lcore 115 as core 2120 on socket 8
EAL: Detected lcore 116 as core 2124 on socket 8
EAL: Detected lcore 117 as core 2124 on socket 8
EAL: Detected lcore 118 as core 2124 on socket 8
EAL: Detected lcore 119 as core 2124 on socket 8
EAL: Detected lcore 120 as core 2136 on socket 8
EAL: Detected lcore 121 as core 2136 on socket 8
EAL: Detected lcore 122 as core 2136 on socket 8
EAL: Detected lcore 123 as core 2136 on socket 8
EAL: Detected lcore 124 as core 2140 on socket 8
EAL: Detected lcore 125 as core 2140 on socket 8
EAL: Detected lcore 126 as core 2140 on socket 8
EAL: Detected lcore 127 as core 2140 on socket 8
EAL: Support maximum 1536 logical core(s) by configuration.
EAL: Detected 128 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x10000)
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: DPAA Bus not present. Skipping.
EAL: VFIO PCI modules not loaded
EAL: Selected IOVA mode 'VA'
EAL: 2 hugepages of size 2097152 reserved, but no mounted hugetlbfs
found for that size
EAL: Probing VFIO support...
EAL: IOMMU type 1 (Type 1) is not supported
EAL: IOMMU type 7 (sPAPR) is supported
EAL: IOMMU type 8 (No-IOMMU) is not supported
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x30000 bytes
EAL: Virtual area found at 0x100010000 (size = 0x30000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 32768
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Detected memory type: socket_id:8 hugepage_sz:1073741824
EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x100040000 (size = 0x10000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x140000000, size 800000000
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x10000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x980000000, size 800000000
EAL: Creating 2 segment lists: n_segs:32 socket_id:8 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x1180000000 (size = 0x10000)
EAL: Memseg list allocated at socket 8, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x11c0000000, size 800000000
EAL: Ask a virtual area of 0x10000 bytes
EAL: Virtual area found at 0x19c0000000 (size = 0x10000)
EAL: Memseg list allocated at socket 8, page size 0x100000kB
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
EAL: VA reserved for memseg list at 0x1a00000000, size 800000000
EAL: TSC frequency is ~510000 KHz
EAL: Main lcore 64 is ready (tid=7fffb8018890;cpuset=[64])
EAL: lcore 65 is ready (tid=7fffb64ad090;cpuset=[65])
EAL: lcore 66 is ready (tid=7fffb5c9d090;cpuset=[66])
EAL: lcore 67 is ready (tid=7fffb548d090;cpuset=[67])
EAL: lcore 68 is ready (tid=7fffb4c7d090;cpuset=[68])
EAL: lcore 69 is ready (tid=7fffa7ffd090;cpuset=[69])
EAL: lcore 70 is ready (tid=7fffa77ed090;cpuset=[70])
EAL: lcore 71 is ready (tid=7fffa6fdd090;cpuset=[71])
EAL: lcore 72 is ready (tid=7fffa67cd090;cpuset=[72])
EAL: lcore 73 is ready (tid=7fffa5fbd090;cpuset=[73])
EAL: lcore 74 is ready (tid=7fffa57ad090;cpuset=[74])
EAL: lcore 75 is ready (tid=7fffa4f9d090;cpuset=[75])
EAL: lcore 76 is ready (tid=7fff8fffd090;cpuset=[76])
EAL: lcore 77 is ready (tid=7fff8f7ed090;cpuset=[77])
EAL: lcore 78 is ready (tid=7fff8efdd090;cpuset=[78])
EAL: lcore 79 is ready (tid=7fff8e7cd090;cpuset=[79])
EAL: lcore 80 is ready (tid=7fff8dfbd090;cpuset=[80])
EAL: lcore 81 is ready (tid=7fff8d7ad090;cpuset=[81])
EAL: lcore 82 is ready (tid=7fff8cf9d090;cpuset=[82])
EAL: lcore 83 is ready (tid=7fff6bffd090;cpuset=[83])
EAL: lcore 84 is ready (tid=7fff6b7ed090;cpuset=[84])
EAL: lcore 85 is ready (tid=7fff6afdd090;cpuset=[85])
EAL: lcore 86 is ready (tid=7fff6a7cd090;cpuset=[86])
EAL: lcore 87 is ready (tid=7fff69fbd090;cpuset=[87])
EAL: lcore 88 is ready (tid=7fff697ad090;cpuset=[88])
EAL: lcore 89 is ready (tid=7fff68f9d090;cpuset=[89])
EAL: lcore 90 is ready (tid=7fff4bffd090;cpuset=[90])
EAL: lcore 91 is ready (tid=7fff4b7ed090;cpuset=[91])
EAL: lcore 92 is ready (tid=7fff4afdd090;cpuset=[92])
EAL: lcore 93 is ready (tid=7fff4a7cd090;cpuset=[93])
EAL: lcore 94 is ready (tid=7fff49fbd090;cpuset=[94])
EAL: lcore 95 is ready (tid=7fff497ad090;cpuset=[95])
EAL: lcore 96 is ready (tid=7fff48f9d090;cpuset=[96])
EAL: lcore 97 is ready (tid=7fff2bffd090;cpuset=[97])
EAL: lcore 98 is ready (tid=7fff2b7ed090;cpuset=[98])
EAL: lcore 99 is ready (tid=7fff2afdd090;cpuset=[99])
EAL: lcore 100 is ready (tid=7fff2a7cd090;cpuset=[100])
EAL: lcore 101 is ready (tid=7fff29fbd090;cpuset=[101])
EAL: lcore 102 is ready (tid=7fff297ad090;cpuset=[102])
EAL: lcore 103 is ready (tid=7fff28f9d090;cpuset=[103])
EAL: lcore 104 is ready (tid=7fff07ffd090;cpuset=[104])
EAL: lcore 105 is ready (tid=7ffeff7ed090;cpuset=[105])
EAL: lcore 106 is ready (tid=7fff077ed090;cpuset=[106])
EAL: lcore 107 is ready (tid=7fff06fdd090;cpuset=[107])
EAL: lcore 108 is ready (tid=7fff067cd090;cpuset=[108])
EAL: lcore 109 is ready (tid=7fff05fbd090;cpuset=[109])
EAL: lcore 110 is ready (tid=7fff057ad090;cpuset=[110])
EAL: lcore 111 is ready (tid=7fff04f9d090;cpuset=[111])
EAL: lcore 112 is ready (tid=7ffeffffd090;cpuset=[112])
EAL: lcore 113 is ready (tid=7ffefefdd090;cpuset=[113])
EAL: lcore 114 is ready (tid=7ffefe7cd090;cpuset=[114])
EAL: lcore 115 is ready (tid=7ffefdfbd090;cpuset=[115])
EAL: lcore 116 is ready (tid=7ffefd7ad090;cpuset=[116])
EAL: lcore 117 is ready (tid=7ffefcf9d090;cpuset=[117])
EAL: lcore 118 is ready (tid=7ffecfffd090;cpuset=[118])
EAL: lcore 119 is ready (tid=7ffecf7ed090;cpuset=[119])
EAL: lcore 120 is ready (tid=7ffecefdd090;cpuset=[120])
EAL: lcore 121 is ready (tid=7ffece7cd090;cpuset=[121])
EAL: lcore 122 is ready (tid=7ffecdfbd090;cpuset=[122])
EAL: lcore 123 is ready (tid=7ffecd7ad090;cpuset=[123])
EAL: lcore 124 is ready (tid=7ffeccf9d090;cpuset=[124])
EAL: lcore 125 is ready (tid=7ffe9bffd090;cpuset=[125])
EAL: lcore 126 is ready (tid=7ffe9b7ed090;cpuset=[126])
EAL: lcore 127 is ready (tid=7ffe9afdd090;cpuset=[127])
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 8
EAL: Restoring previous memory policy: 0
EAL: request: mp_malloc_sync
EAL: Heap on socket 8 was expanded by 1024MB
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 15b3:1019 mlx5_pci
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.0 (socket 0)
EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)'
EAL: request: mp_malloc_sync
EAL: Heap on socket 0 was expanded by 1024MB
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
EAL: PCI device 0000:01:00.1 on NUMA socket 0
EAL: probe driver: 15b3:1019 mlx5_pci
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.1 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
EAL: PCI device 0003:01:00.0 on NUMA socket 0
EAL: probe driver: 14e4:168a net_bnx2x
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0003:01:00.1 on NUMA socket 0
EAL: probe driver: 14e4:168a net_bnx2x
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0003:01:00.2 on NUMA socket 0
EAL: probe driver: 14e4:168a net_bnx2x
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0003:01:00.3 on NUMA socket 0
EAL: probe driver: 14e4:168a net_bnx2x
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0030:01:00.0 on NUMA socket 8
EAL: probe driver: 15b3:1019 mlx5_pci
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.0 (socket 8)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
EAL: PCI device 0030:01:00.1 on NUMA socket 8
EAL: probe driver: 15b3:1019 mlx5_pci
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.1 (socket 8)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Default miss action is not supported.
EAL: PCI device 0034:01:00.0 on NUMA socket 8
EAL: probe driver: 8086:1583 net_i40e
EAL: set IOMMU type 1 (Type 1) failed, error 19 (No such device)
EAL: using IOMMU type 7 (sPAPR)
EAL: Highest VA address in memseg list is 0x2200000000
EAL: Setting DMA window size to 0x4000000000
EAL: Mem event callback 'vfio_mem_event_clb:(nil)' registered
EAL: Installed memory event callback for VFIO
EAL: VFIO reports MSI-X BAR as mappable
EAL: PCI memory mapped at 0x2200000000
EAL: PCI memory mapped at 0x2200800000
EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.0 (socket 8)
EAL: PCI device 0034:01:00.1 on NUMA socket 8
EAL: probe driver: 8086:1583 net_i40e
EAL: VFIO reports MSI-X BAR as mappable
EAL: PCI memory mapped at 0x2200810000
EAL: PCI memory mapped at 0x2201010000
EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.1 (socket 8)
APP: HPET is not enabled, using TSC as default timer
RTE>>vfio_autotest
DRC: sz = 0x20000
DRC: mem = 0x0x7fffb8680000
EAL: dma map attempt outside DMA window
EAL: Failed to map DMA
EAL: Couldn't map new region for DMA
Failed to dma map whole region, ret=-1(No such file or directory)
Error vfio dma map/unmap, ret=-1
Test Failed
RTE>>
^ permalink raw reply [relevance 6%]
* Re: [dpdk-dev] [PATCH v4 3/4] test: add test case to validate VFIO DMA map/unmap
2020-12-02 19:23 6% ` David Christensen
@ 2020-12-03 7:14 0% ` Nithin Dabilpuram
2020-12-14 8:24 0% ` Nithin Dabilpuram
0 siblings, 1 reply; 15+ results
From: Nithin Dabilpuram @ 2020-12-03 7:14 UTC (permalink / raw)
To: David Christensen; +Cc: anatoly.burakov, david.marchand, jerinj, dev
On Wed, Dec 02, 2020 at 11:23:09AM -0800, David Christensen wrote:
>
>
> On 12/1/20 9:46 PM, Nithin Dabilpuram wrote:
> > Test case mmap's system pages and tries to performs a user
> > DMA map and unmap both partially and fully.
> >
> > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > ---
> > app/test/meson.build | 1 +
> > app/test/test_vfio.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 104 insertions(+)
> > create mode 100644 app/test/test_vfio.c
> >
> > diff --git a/app/test/meson.build b/app/test/meson.build
> > index 94fd39f..d9eedb6 100644
> > --- a/app/test/meson.build
> > +++ b/app/test/meson.build
> > @@ -139,6 +139,7 @@ test_sources = files('commands.c',
> > 'test_trace_register.c',
> > 'test_trace_perf.c',
> > 'test_version.c',
> > + 'test_vfio.c',
> > 'virtual_pmd.c'
> > )
> >
> > diff --git a/app/test/test_vfio.c b/app/test/test_vfio.c
> > new file mode 100644
> > index 0000000..00626d4
> > --- /dev/null
> > +++ b/app/test/test_vfio.c
> > @@ -0,0 +1,103 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(C) 2020 Marvell.
> > + */
> > +
> > +#include <stdio.h>
> > +#include <stdint.h>
> > +#include <string.h>
> > +#include <sys/mman.h>
> > +#include <unistd.h>
> > +
> > +#include <rte_common.h>
> > +#include <rte_eal.h>
> > +#include <rte_eal_paging.h>
> > +#include <rte_errno.h>
> > +#include <rte_memory.h>
> > +#include <rte_vfio.h>
> > +
> > +#include "test.h"
> > +
> > +static int
> > +test_memory_vfio_dma_map(void)
> > +{
> > + uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
> > + uint64_t unmap1, unmap2;
> > + uint8_t *mem;
> > + int ret;
> > +
> > + /* Allocate twice size of page */
> > + mem = mmap(NULL, sz, PROT_READ | PROT_WRITE,
> > + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> > + if (mem == MAP_FAILED) {
> > + printf("Failed to allocate memory for external heap\n");
> > + return -1;
> > + }
> > +
> > + /* Force page allocation */
> > + memset(mem, 0, sz);
> > +
> > + /* map the whole region */
> > + ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > + (uintptr_t)mem, (rte_iova_t)mem, sz);
> > + if (ret) {
> > + /* Check if VFIO is not available or no device is probed */
> > + if (rte_errno == ENOTSUP || rte_errno == ENODEV) {
> > + ret = 1;
> > + goto fail;
> > + }
> > + printf("Failed to dma map whole region, ret=%d(%s)\n",
> > + ret, rte_strerror(rte_errno));
> > + goto fail;
> > + }
> > +
> > + unmap1 = (uint64_t)mem + (sz / 2);
> > + sz1 = sz / 2;
> > + unmap2 = (uint64_t)mem;
> > + sz2 = sz / 2;
> > + /* unmap the partial region */
> > + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > + unmap1, (rte_iova_t)unmap1, sz1);
> > + if (ret) {
> > + if (rte_errno == ENOTSUP) {
> > + printf("Partial dma unmap not supported\n");
> > + unmap2 = (uint64_t)mem;
> > + sz2 = sz;
> > + } else {
> > + printf("Failed to unmap second half region, ret=%d(%s)\n",
> > + ret, rte_strerror(rte_errno));
> > + goto fail;
> > + }
> > + }
> > +
> > + /* unmap the remaining region */
> > + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > + unmap2, (rte_iova_t)unmap2, sz2);
> > + if (ret) {
> > + printf("Failed to unmap remaining region, ret=%d(%s)\n", ret,
> > + rte_strerror(rte_errno));
> > + goto fail;
> > + }
> > +
> > +fail:
> > + munmap(mem, sz);
> > + return ret;
> > +}
> > +
> > +static int
> > +test_vfio(void)
> > +{
> > + int ret;
> > +
> > + /* test for vfio dma map/unmap */
> > + ret = test_memory_vfio_dma_map();
> > + if (ret == 1) {
> > + printf("VFIO dma map/unmap unsupported\n");
> > + } else if (ret < 0) {
> > + printf("Error vfio dma map/unmap, ret=%d\n", ret);
> > + return -1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +REGISTER_TEST_COMMAND(vfio_autotest, test_vfio);
> >
>
> The test as written fails on a POWER9 system (see below for debug output).
>
> IOMMU on POWER systems requires that a DMA window be defined and that all
> DMA mappings reside within that window. In this test, the DMA window is
> defined as 0x0 to 0x4000000000, but the VA allocated in your test is
> 0x7fffb8680000, well outside that range.
>
> I recently submitted a change in the 20.11 release which scans the memseg
> list in order to set the DMA window. That test can be seen here:
>
> EAL: Highest VA address in memseg list is 0x2200000000
> EAL: Setting DMA window size to 0x4000000000
>
I missed that thread. So basically external memory with IOVA as VA mode
is not supported in POWER9 systems as it's memseg lists can be created later
after DMA window size is fixed. Correct ?
> Can we modify the test to allocate memory out of the exsitng memseg
> allocations?
Since I'm mmapin'g normal pages for this test outside EAL, I cannot use memseg list
range as VA as it is already reserved by memseg list's.
I can see only three options left.
#1 Use initial process VA range by using heap memory instead of mmap
which is falling below eal's base_virtaddr for both freebsd and windows and also I think
your DMA window will include that.
#2 Use PA from real PA range or like externel_mem_autotest(). The test currently
is acting like IOVA as VA test but this change will make it as IOVA as PA.
#3 Disable this test for VFIO SPAPR or remove it completely.
Will #1 work in your case ?
>
> Dave
>
> $ sudo ~/src/dpdk/build/app/test/dpdk-test --log="eal,debug" --iova-mode=va
> -l 64-127
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 0 on socket 0
> EAL: Detected lcore 2 as core 0 on socket 0
> EAL: Detected lcore 3 as core 0 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 4 on socket 0
> EAL: Detected lcore 6 as core 4 on socket 0
> EAL: Detected lcore 7 as core 4 on socket 0
> EAL: Detected lcore 8 as core 8 on socket 0
> EAL: Detected lcore 9 as core 8 on socket 0
> EAL: Detected lcore 10 as core 8 on socket 0
> EAL: Detected lcore 11 as core 8 on socket 0
> EAL: Detected lcore 12 as core 12 on socket 0
> EAL: Detected lcore 13 as core 12 on socket 0
> EAL: Detected lcore 14 as core 12 on socket 0
> EAL: Detected lcore 15 as core 12 on socket 0
> EAL: Detected lcore 16 as core 16 on socket 0
> EAL: Detected lcore 17 as core 16 on socket 0
> EAL: Detected lcore 18 as core 16 on socket 0
> EAL: Detected lcore 19 as core 16 on socket 0
> EAL: Detected lcore 20 as core 20 on socket 0
> EAL: Detected lcore 21 as core 20 on socket 0
> EAL: Detected lcore 22 as core 20 on socket 0
> EAL: Detected lcore 23 as core 20 on socket 0
> EAL: Detected lcore 24 as core 24 on socket 0
> EAL: Detected lcore 25 as core 24 on socket 0
> EAL: Detected lcore 26 as core 24 on socket 0
> EAL: Detected lcore 27 as core 24 on socket 0
> EAL: Detected lcore 28 as core 28 on socket 0
> EAL: Detected lcore 29 as core 28 on socket 0
> EAL: Detected lcore 30 as core 28 on socket 0
> EAL: Detected lcore 31 as core 28 on socket 0
> EAL: Detected lcore 32 as core 32 on socket 0
> EAL: Detected lcore 33 as core 32 on socket 0
> EAL: Detected lcore 34 as core 32 on socket 0
> EAL: Detected lcore 35 as core 32 on socket 0
> EAL: Detected lcore 36 as core 36 on socket 0
> EAL: Detected lcore 37 as core 36 on socket 0
> EAL: Detected lcore 38 as core 36 on socket 0
> EAL: Detected lcore 39 as core 36 on socket 0
> EAL: Detected lcore 40 as core 48 on socket 0
> EAL: Detected lcore 41 as core 48 on socket 0
> EAL: Detected lcore 42 as core 48 on socket 0
> EAL: Detected lcore 43 as core 48 on socket 0
> EAL: Detected lcore 44 as core 52 on socket 0
> EAL: Detected lcore 45 as core 52 on socket 0
> EAL: Detected lcore 46 as core 52 on socket 0
> EAL: Detected lcore 47 as core 52 on socket 0
> EAL: Detected lcore 48 as core 72 on socket 0
> EAL: Detected lcore 49 as core 72 on socket 0
> EAL: Detected lcore 50 as core 72 on socket 0
> EAL: Detected lcore 51 as core 72 on socket 0
> EAL: Detected lcore 52 as core 76 on socket 0
> EAL: Detected lcore 53 as core 76 on socket 0
> EAL: Detected lcore 54 as core 76 on socket 0
> EAL: Detected lcore 55 as core 76 on socket 0
> EAL: Detected lcore 56 as core 80 on socket 0
> EAL: Detected lcore 57 as core 80 on socket 0
> EAL: Detected lcore 58 as core 80 on socket 0
> EAL: Detected lcore 59 as core 80 on socket 0
> EAL: Detected lcore 60 as core 84 on socket 0
> EAL: Detected lcore 61 as core 84 on socket 0
> EAL: Detected lcore 62 as core 84 on socket 0
> EAL: Detected lcore 63 as core 84 on socket 0
> EAL: Detected lcore 64 as core 2048 on socket 8
> EAL: Detected lcore 65 as core 2048 on socket 8
> EAL: Detected lcore 66 as core 2048 on socket 8
> EAL: Detected lcore 67 as core 2048 on socket 8
> EAL: Detected lcore 68 as core 2052 on socket 8
> EAL: Detected lcore 69 as core 2052 on socket 8
> EAL: Detected lcore 70 as core 2052 on socket 8
> EAL: Detected lcore 71 as core 2052 on socket 8
> EAL: Detected lcore 72 as core 2056 on socket 8
> EAL: Detected lcore 73 as core 2056 on socket 8
> EAL: Detected lcore 74 as core 2056 on socket 8
> EAL: Detected lcore 75 as core 2056 on socket 8
> EAL: Detected lcore 76 as core 2060 on socket 8
> EAL: Detected lcore 77 as core 2060 on socket 8
> EAL: Detected lcore 78 as core 2060 on socket 8
> EAL: Detected lcore 79 as core 2060 on socket 8
> EAL: Detected lcore 80 as core 2072 on socket 8
> EAL: Detected lcore 81 as core 2072 on socket 8
> EAL: Detected lcore 82 as core 2072 on socket 8
> EAL: Detected lcore 83 as core 2072 on socket 8
> EAL: Detected lcore 84 as core 2076 on socket 8
> EAL: Detected lcore 85 as core 2076 on socket 8
> EAL: Detected lcore 86 as core 2076 on socket 8
> EAL: Detected lcore 87 as core 2076 on socket 8
> EAL: Detected lcore 88 as core 2080 on socket 8
> EAL: Detected lcore 89 as core 2080 on socket 8
> EAL: Detected lcore 90 as core 2080 on socket 8
> EAL: Detected lcore 91 as core 2080 on socket 8
> EAL: Detected lcore 92 as core 2084 on socket 8
> EAL: Detected lcore 93 as core 2084 on socket 8
> EAL: Detected lcore 94 as core 2084 on socket 8
> EAL: Detected lcore 95 as core 2084 on socket 8
> EAL: Detected lcore 96 as core 2088 on socket 8
> EAL: Detected lcore 97 as core 2088 on socket 8
> EAL: Detected lcore 98 as core 2088 on socket 8
> EAL: Detected lcore 99 as core 2088 on socket 8
> EAL: Detected lcore 100 as core 2092 on socket 8
> EAL: Detected lcore 101 as core 2092 on socket 8
> EAL: Detected lcore 102 as core 2092 on socket 8
> EAL: Detected lcore 103 as core 2092 on socket 8
> EAL: Detected lcore 104 as core 2096 on socket 8
> EAL: Detected lcore 105 as core 2096 on socket 8
> EAL: Detected lcore 106 as core 2096 on socket 8
> EAL: Detected lcore 107 as core 2096 on socket 8
> EAL: Detected lcore 108 as core 2100 on socket 8
> EAL: Detected lcore 109 as core 2100 on socket 8
> EAL: Detected lcore 110 as core 2100 on socket 8
> EAL: Detected lcore 111 as core 2100 on socket 8
> EAL: Detected lcore 112 as core 2120 on socket 8
> EAL: Detected lcore 113 as core 2120 on socket 8
> EAL: Detected lcore 114 as core 2120 on socket 8
> EAL: Detected lcore 115 as core 2120 on socket 8
> EAL: Detected lcore 116 as core 2124 on socket 8
> EAL: Detected lcore 117 as core 2124 on socket 8
> EAL: Detected lcore 118 as core 2124 on socket 8
> EAL: Detected lcore 119 as core 2124 on socket 8
> EAL: Detected lcore 120 as core 2136 on socket 8
> EAL: Detected lcore 121 as core 2136 on socket 8
> EAL: Detected lcore 122 as core 2136 on socket 8
> EAL: Detected lcore 123 as core 2136 on socket 8
> EAL: Detected lcore 124 as core 2140 on socket 8
> EAL: Detected lcore 125 as core 2140 on socket 8
> EAL: Detected lcore 126 as core 2140 on socket 8
> EAL: Detected lcore 127 as core 2140 on socket 8
> EAL: Support maximum 1536 logical core(s) by configuration.
> EAL: Detected 128 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Ask a virtual area of 0x10000 bytes
> EAL: Virtual area found at 0x100000000 (size = 0x10000)
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: DPAA Bus not present. Skipping.
> EAL: VFIO PCI modules not loaded
> EAL: Selected IOVA mode 'VA'
> EAL: 2 hugepages of size 2097152 reserved, but no mounted hugetlbfs found
> for that size
> EAL: Probing VFIO support...
> EAL: IOMMU type 1 (Type 1) is not supported
> EAL: IOMMU type 7 (sPAPR) is supported
> EAL: IOMMU type 8 (No-IOMMU) is not supported
> EAL: VFIO support initialized
> EAL: Ask a virtual area of 0x30000 bytes
> EAL: Virtual area found at 0x100010000 (size = 0x30000)
> EAL: Setting up physically contiguous memory...
> EAL: Setting maximum number of open files to 32768
> EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> EAL: Detected memory type: socket_id:8 hugepage_sz:1073741824
> EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
> EAL: Ask a virtual area of 0x10000 bytes
> EAL: Virtual area found at 0x100040000 (size = 0x10000)
> EAL: Memseg list allocated at socket 0, page size 0x100000kB
> EAL: Ask a virtual area of 0x800000000 bytes
> EAL: Virtual area found at 0x140000000 (size = 0x800000000)
> EAL: VA reserved for memseg list at 0x140000000, size 800000000
> EAL: Ask a virtual area of 0x10000 bytes
> EAL: Virtual area found at 0x940000000 (size = 0x10000)
> EAL: Memseg list allocated at socket 0, page size 0x100000kB
> EAL: Ask a virtual area of 0x800000000 bytes
> EAL: Virtual area found at 0x980000000 (size = 0x800000000)
> EAL: VA reserved for memseg list at 0x980000000, size 800000000
> EAL: Creating 2 segment lists: n_segs:32 socket_id:8 hugepage_sz:1073741824
> EAL: Ask a virtual area of 0x10000 bytes
> EAL: Virtual area found at 0x1180000000 (size = 0x10000)
> EAL: Memseg list allocated at socket 8, page size 0x100000kB
> EAL: Ask a virtual area of 0x800000000 bytes
> EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
> EAL: VA reserved for memseg list at 0x11c0000000, size 800000000
> EAL: Ask a virtual area of 0x10000 bytes
> EAL: Virtual area found at 0x19c0000000 (size = 0x10000)
> EAL: Memseg list allocated at socket 8, page size 0x100000kB
> EAL: Ask a virtual area of 0x800000000 bytes
> EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
> EAL: VA reserved for memseg list at 0x1a00000000, size 800000000
> EAL: TSC frequency is ~510000 KHz
> EAL: Main lcore 64 is ready (tid=7fffb8018890;cpuset=[64])
> EAL: lcore 65 is ready (tid=7fffb64ad090;cpuset=[65])
> EAL: lcore 66 is ready (tid=7fffb5c9d090;cpuset=[66])
> EAL: lcore 67 is ready (tid=7fffb548d090;cpuset=[67])
> EAL: lcore 68 is ready (tid=7fffb4c7d090;cpuset=[68])
> EAL: lcore 69 is ready (tid=7fffa7ffd090;cpuset=[69])
> EAL: lcore 70 is ready (tid=7fffa77ed090;cpuset=[70])
> EAL: lcore 71 is ready (tid=7fffa6fdd090;cpuset=[71])
> EAL: lcore 72 is ready (tid=7fffa67cd090;cpuset=[72])
> EAL: lcore 73 is ready (tid=7fffa5fbd090;cpuset=[73])
> EAL: lcore 74 is ready (tid=7fffa57ad090;cpuset=[74])
> EAL: lcore 75 is ready (tid=7fffa4f9d090;cpuset=[75])
> EAL: lcore 76 is ready (tid=7fff8fffd090;cpuset=[76])
> EAL: lcore 77 is ready (tid=7fff8f7ed090;cpuset=[77])
> EAL: lcore 78 is ready (tid=7fff8efdd090;cpuset=[78])
> EAL: lcore 79 is ready (tid=7fff8e7cd090;cpuset=[79])
> EAL: lcore 80 is ready (tid=7fff8dfbd090;cpuset=[80])
> EAL: lcore 81 is ready (tid=7fff8d7ad090;cpuset=[81])
> EAL: lcore 82 is ready (tid=7fff8cf9d090;cpuset=[82])
> EAL: lcore 83 is ready (tid=7fff6bffd090;cpuset=[83])
> EAL: lcore 84 is ready (tid=7fff6b7ed090;cpuset=[84])
> EAL: lcore 85 is ready (tid=7fff6afdd090;cpuset=[85])
> EAL: lcore 86 is ready (tid=7fff6a7cd090;cpuset=[86])
> EAL: lcore 87 is ready (tid=7fff69fbd090;cpuset=[87])
> EAL: lcore 88 is ready (tid=7fff697ad090;cpuset=[88])
> EAL: lcore 89 is ready (tid=7fff68f9d090;cpuset=[89])
> EAL: lcore 90 is ready (tid=7fff4bffd090;cpuset=[90])
> EAL: lcore 91 is ready (tid=7fff4b7ed090;cpuset=[91])
> EAL: lcore 92 is ready (tid=7fff4afdd090;cpuset=[92])
> EAL: lcore 93 is ready (tid=7fff4a7cd090;cpuset=[93])
> EAL: lcore 94 is ready (tid=7fff49fbd090;cpuset=[94])
> EAL: lcore 95 is ready (tid=7fff497ad090;cpuset=[95])
> EAL: lcore 96 is ready (tid=7fff48f9d090;cpuset=[96])
> EAL: lcore 97 is ready (tid=7fff2bffd090;cpuset=[97])
> EAL: lcore 98 is ready (tid=7fff2b7ed090;cpuset=[98])
> EAL: lcore 99 is ready (tid=7fff2afdd090;cpuset=[99])
> EAL: lcore 100 is ready (tid=7fff2a7cd090;cpuset=[100])
> EAL: lcore 101 is ready (tid=7fff29fbd090;cpuset=[101])
> EAL: lcore 102 is ready (tid=7fff297ad090;cpuset=[102])
> EAL: lcore 103 is ready (tid=7fff28f9d090;cpuset=[103])
> EAL: lcore 104 is ready (tid=7fff07ffd090;cpuset=[104])
> EAL: lcore 105 is ready (tid=7ffeff7ed090;cpuset=[105])
> EAL: lcore 106 is ready (tid=7fff077ed090;cpuset=[106])
> EAL: lcore 107 is ready (tid=7fff06fdd090;cpuset=[107])
> EAL: lcore 108 is ready (tid=7fff067cd090;cpuset=[108])
> EAL: lcore 109 is ready (tid=7fff05fbd090;cpuset=[109])
> EAL: lcore 110 is ready (tid=7fff057ad090;cpuset=[110])
> EAL: lcore 111 is ready (tid=7fff04f9d090;cpuset=[111])
> EAL: lcore 112 is ready (tid=7ffeffffd090;cpuset=[112])
> EAL: lcore 113 is ready (tid=7ffefefdd090;cpuset=[113])
> EAL: lcore 114 is ready (tid=7ffefe7cd090;cpuset=[114])
> EAL: lcore 115 is ready (tid=7ffefdfbd090;cpuset=[115])
> EAL: lcore 116 is ready (tid=7ffefd7ad090;cpuset=[116])
> EAL: lcore 117 is ready (tid=7ffefcf9d090;cpuset=[117])
> EAL: lcore 118 is ready (tid=7ffecfffd090;cpuset=[118])
> EAL: lcore 119 is ready (tid=7ffecf7ed090;cpuset=[119])
> EAL: lcore 120 is ready (tid=7ffecefdd090;cpuset=[120])
> EAL: lcore 121 is ready (tid=7ffece7cd090;cpuset=[121])
> EAL: lcore 122 is ready (tid=7ffecdfbd090;cpuset=[122])
> EAL: lcore 123 is ready (tid=7ffecd7ad090;cpuset=[123])
> EAL: lcore 124 is ready (tid=7ffeccf9d090;cpuset=[124])
> EAL: lcore 125 is ready (tid=7ffe9bffd090;cpuset=[125])
> EAL: lcore 126 is ready (tid=7ffe9b7ed090;cpuset=[126])
> EAL: lcore 127 is ready (tid=7ffe9afdd090;cpuset=[127])
> EAL: Trying to obtain current memory policy.
> EAL: Setting policy MPOL_PREFERRED for socket 8
> EAL: Restoring previous memory policy: 0
> EAL: request: mp_malloc_sync
> EAL: Heap on socket 8 was expanded by 1024MB
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 15b3:1019 mlx5_pci
> EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.0 (socket 0)
> EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered
> EAL: Trying to obtain current memory policy.
> EAL: Setting policy MPOL_PREFERRED for socket 0
> EAL: Restoring previous memory policy: 0
> EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)'
> EAL: request: mp_malloc_sync
> EAL: Heap on socket 0 was expanded by 1024MB
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: Default miss action is not supported.
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> EAL: probe driver: 15b3:1019 mlx5_pci
> EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.1 (socket 0)
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: Default miss action is not supported.
> EAL: PCI device 0003:01:00.0 on NUMA socket 0
> EAL: probe driver: 14e4:168a net_bnx2x
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0003:01:00.1 on NUMA socket 0
> EAL: probe driver: 14e4:168a net_bnx2x
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0003:01:00.2 on NUMA socket 0
> EAL: probe driver: 14e4:168a net_bnx2x
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0003:01:00.3 on NUMA socket 0
> EAL: probe driver: 14e4:168a net_bnx2x
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0030:01:00.0 on NUMA socket 8
> EAL: probe driver: 15b3:1019 mlx5_pci
> EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.0 (socket 8)
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: Default miss action is not supported.
> EAL: PCI device 0030:01:00.1 on NUMA socket 8
> EAL: probe driver: 15b3:1019 mlx5_pci
> EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.1 (socket 8)
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: Default miss action is not supported.
> EAL: PCI device 0034:01:00.0 on NUMA socket 8
> EAL: probe driver: 8086:1583 net_i40e
> EAL: set IOMMU type 1 (Type 1) failed, error 19 (No such device)
> EAL: using IOMMU type 7 (sPAPR)
> EAL: Highest VA address in memseg list is 0x2200000000
> EAL: Setting DMA window size to 0x4000000000
> EAL: Mem event callback 'vfio_mem_event_clb:(nil)' registered
> EAL: Installed memory event callback for VFIO
> EAL: VFIO reports MSI-X BAR as mappable
> EAL: PCI memory mapped at 0x2200000000
> EAL: PCI memory mapped at 0x2200800000
> EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.0 (socket 8)
> EAL: PCI device 0034:01:00.1 on NUMA socket 8
> EAL: probe driver: 8086:1583 net_i40e
> EAL: VFIO reports MSI-X BAR as mappable
> EAL: PCI memory mapped at 0x2200810000
> EAL: PCI memory mapped at 0x2201010000
> EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.1 (socket 8)
> APP: HPET is not enabled, using TSC as default timer
> RTE>>vfio_autotest
> DRC: sz = 0x20000
> DRC: mem = 0x0x7fffb8680000
> EAL: dma map attempt outside DMA window
> EAL: Failed to map DMA
> EAL: Couldn't map new region for DMA
> Failed to dma map whole region, ret=-1(No such file or directory)
> Error vfio dma map/unmap, ret=-1
> Test Failed
> RTE>>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] [PATCH v4 3/4] test: add test case to validate VFIO DMA map/unmap
2020-12-03 7:14 0% ` Nithin Dabilpuram
@ 2020-12-14 8:24 0% ` Nithin Dabilpuram
0 siblings, 0 replies; 15+ results
From: Nithin Dabilpuram @ 2020-12-14 8:24 UTC (permalink / raw)
To: David Christensen; +Cc: anatoly.burakov, david.marchand, jerinj, dev
Hi David,
As mentioned below in #1, I sent v5 with memory allocated from Heap which I think
falls in initially configured DMA window 0x0 .... 0x4000000000 atleast in
Linux as DPDK memory starts after HEAP.
Let me know if it is ok for POWER9 system.
On Thu, Dec 03, 2020 at 12:44:06PM +0530, Nithin Dabilpuram wrote:
> On Wed, Dec 02, 2020 at 11:23:09AM -0800, David Christensen wrote:
> >
> >
> > On 12/1/20 9:46 PM, Nithin Dabilpuram wrote:
> > > Test case mmap's system pages and tries to performs a user
> > > DMA map and unmap both partially and fully.
> > >
> > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > > Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > > ---
> > > app/test/meson.build | 1 +
> > > app/test/test_vfio.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++
> > > 2 files changed, 104 insertions(+)
> > > create mode 100644 app/test/test_vfio.c
> > >
> > > diff --git a/app/test/meson.build b/app/test/meson.build
> > > index 94fd39f..d9eedb6 100644
> > > --- a/app/test/meson.build
> > > +++ b/app/test/meson.build
> > > @@ -139,6 +139,7 @@ test_sources = files('commands.c',
> > > 'test_trace_register.c',
> > > 'test_trace_perf.c',
> > > 'test_version.c',
> > > + 'test_vfio.c',
> > > 'virtual_pmd.c'
> > > )
> > >
> > > diff --git a/app/test/test_vfio.c b/app/test/test_vfio.c
> > > new file mode 100644
> > > index 0000000..00626d4
> > > --- /dev/null
> > > +++ b/app/test/test_vfio.c
> > > @@ -0,0 +1,103 @@
> > > +/* SPDX-License-Identifier: BSD-3-Clause
> > > + * Copyright(C) 2020 Marvell.
> > > + */
> > > +
> > > +#include <stdio.h>
> > > +#include <stdint.h>
> > > +#include <string.h>
> > > +#include <sys/mman.h>
> > > +#include <unistd.h>
> > > +
> > > +#include <rte_common.h>
> > > +#include <rte_eal.h>
> > > +#include <rte_eal_paging.h>
> > > +#include <rte_errno.h>
> > > +#include <rte_memory.h>
> > > +#include <rte_vfio.h>
> > > +
> > > +#include "test.h"
> > > +
> > > +static int
> > > +test_memory_vfio_dma_map(void)
> > > +{
> > > + uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
> > > + uint64_t unmap1, unmap2;
> > > + uint8_t *mem;
> > > + int ret;
> > > +
> > > + /* Allocate twice size of page */
> > > + mem = mmap(NULL, sz, PROT_READ | PROT_WRITE,
> > > + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> > > + if (mem == MAP_FAILED) {
> > > + printf("Failed to allocate memory for external heap\n");
> > > + return -1;
> > > + }
> > > +
> > > + /* Force page allocation */
> > > + memset(mem, 0, sz);
> > > +
> > > + /* map the whole region */
> > > + ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > > + (uintptr_t)mem, (rte_iova_t)mem, sz);
> > > + if (ret) {
> > > + /* Check if VFIO is not available or no device is probed */
> > > + if (rte_errno == ENOTSUP || rte_errno == ENODEV) {
> > > + ret = 1;
> > > + goto fail;
> > > + }
> > > + printf("Failed to dma map whole region, ret=%d(%s)\n",
> > > + ret, rte_strerror(rte_errno));
> > > + goto fail;
> > > + }
> > > +
> > > + unmap1 = (uint64_t)mem + (sz / 2);
> > > + sz1 = sz / 2;
> > > + unmap2 = (uint64_t)mem;
> > > + sz2 = sz / 2;
> > > + /* unmap the partial region */
> > > + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > > + unmap1, (rte_iova_t)unmap1, sz1);
> > > + if (ret) {
> > > + if (rte_errno == ENOTSUP) {
> > > + printf("Partial dma unmap not supported\n");
> > > + unmap2 = (uint64_t)mem;
> > > + sz2 = sz;
> > > + } else {
> > > + printf("Failed to unmap second half region, ret=%d(%s)\n",
> > > + ret, rte_strerror(rte_errno));
> > > + goto fail;
> > > + }
> > > + }
> > > +
> > > + /* unmap the remaining region */
> > > + ret = rte_vfio_container_dma_unmap(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > > + unmap2, (rte_iova_t)unmap2, sz2);
> > > + if (ret) {
> > > + printf("Failed to unmap remaining region, ret=%d(%s)\n", ret,
> > > + rte_strerror(rte_errno));
> > > + goto fail;
> > > + }
> > > +
> > > +fail:
> > > + munmap(mem, sz);
> > > + return ret;
> > > +}
> > > +
> > > +static int
> > > +test_vfio(void)
> > > +{
> > > + int ret;
> > > +
> > > + /* test for vfio dma map/unmap */
> > > + ret = test_memory_vfio_dma_map();
> > > + if (ret == 1) {
> > > + printf("VFIO dma map/unmap unsupported\n");
> > > + } else if (ret < 0) {
> > > + printf("Error vfio dma map/unmap, ret=%d\n", ret);
> > > + return -1;
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +REGISTER_TEST_COMMAND(vfio_autotest, test_vfio);
> > >
> >
> > The test as written fails on a POWER9 system (see below for debug output).
> >
> > IOMMU on POWER systems requires that a DMA window be defined and that all
> > DMA mappings reside within that window. In this test, the DMA window is
> > defined as 0x0 to 0x4000000000, but the VA allocated in your test is
> > 0x7fffb8680000, well outside that range.
> >
> > I recently submitted a change in the 20.11 release which scans the memseg
> > list in order to set the DMA window. That test can be seen here:
> >
> > EAL: Highest VA address in memseg list is 0x2200000000
> > EAL: Setting DMA window size to 0x4000000000
> >
>
> I missed that thread. So basically external memory with IOVA as VA mode
> is not supported in POWER9 systems as it's memseg lists can be created later
> after DMA window size is fixed. Correct ?
>
> > Can we modify the test to allocate memory out of the exsitng memseg
> > allocations?
>
> Since I'm mmapin'g normal pages for this test outside EAL, I cannot use memseg list
> range as VA as it is already reserved by memseg list's.
>
> I can see only three options left.
>
> #1 Use initial process VA range by using heap memory instead of mmap
> which is falling below eal's base_virtaddr for both freebsd and windows and also I think
> your DMA window will include that.
>
> #2 Use PA from real PA range or like externel_mem_autotest(). The test currently
> is acting like IOVA as VA test but this change will make it as IOVA as PA.
>
> #3 Disable this test for VFIO SPAPR or remove it completely.
>
> Will #1 work in your case ?
> >
> > Dave
> >
> > $ sudo ~/src/dpdk/build/app/test/dpdk-test --log="eal,debug" --iova-mode=va
> > -l 64-127
> > EAL: Detected lcore 0 as core 0 on socket 0
> > EAL: Detected lcore 1 as core 0 on socket 0
> > EAL: Detected lcore 2 as core 0 on socket 0
> > EAL: Detected lcore 3 as core 0 on socket 0
> > EAL: Detected lcore 4 as core 4 on socket 0
> > EAL: Detected lcore 5 as core 4 on socket 0
> > EAL: Detected lcore 6 as core 4 on socket 0
> > EAL: Detected lcore 7 as core 4 on socket 0
> > EAL: Detected lcore 8 as core 8 on socket 0
> > EAL: Detected lcore 9 as core 8 on socket 0
> > EAL: Detected lcore 10 as core 8 on socket 0
> > EAL: Detected lcore 11 as core 8 on socket 0
> > EAL: Detected lcore 12 as core 12 on socket 0
> > EAL: Detected lcore 13 as core 12 on socket 0
> > EAL: Detected lcore 14 as core 12 on socket 0
> > EAL: Detected lcore 15 as core 12 on socket 0
> > EAL: Detected lcore 16 as core 16 on socket 0
> > EAL: Detected lcore 17 as core 16 on socket 0
> > EAL: Detected lcore 18 as core 16 on socket 0
> > EAL: Detected lcore 19 as core 16 on socket 0
> > EAL: Detected lcore 20 as core 20 on socket 0
> > EAL: Detected lcore 21 as core 20 on socket 0
> > EAL: Detected lcore 22 as core 20 on socket 0
> > EAL: Detected lcore 23 as core 20 on socket 0
> > EAL: Detected lcore 24 as core 24 on socket 0
> > EAL: Detected lcore 25 as core 24 on socket 0
> > EAL: Detected lcore 26 as core 24 on socket 0
> > EAL: Detected lcore 27 as core 24 on socket 0
> > EAL: Detected lcore 28 as core 28 on socket 0
> > EAL: Detected lcore 29 as core 28 on socket 0
> > EAL: Detected lcore 30 as core 28 on socket 0
> > EAL: Detected lcore 31 as core 28 on socket 0
> > EAL: Detected lcore 32 as core 32 on socket 0
> > EAL: Detected lcore 33 as core 32 on socket 0
> > EAL: Detected lcore 34 as core 32 on socket 0
> > EAL: Detected lcore 35 as core 32 on socket 0
> > EAL: Detected lcore 36 as core 36 on socket 0
> > EAL: Detected lcore 37 as core 36 on socket 0
> > EAL: Detected lcore 38 as core 36 on socket 0
> > EAL: Detected lcore 39 as core 36 on socket 0
> > EAL: Detected lcore 40 as core 48 on socket 0
> > EAL: Detected lcore 41 as core 48 on socket 0
> > EAL: Detected lcore 42 as core 48 on socket 0
> > EAL: Detected lcore 43 as core 48 on socket 0
> > EAL: Detected lcore 44 as core 52 on socket 0
> > EAL: Detected lcore 45 as core 52 on socket 0
> > EAL: Detected lcore 46 as core 52 on socket 0
> > EAL: Detected lcore 47 as core 52 on socket 0
> > EAL: Detected lcore 48 as core 72 on socket 0
> > EAL: Detected lcore 49 as core 72 on socket 0
> > EAL: Detected lcore 50 as core 72 on socket 0
> > EAL: Detected lcore 51 as core 72 on socket 0
> > EAL: Detected lcore 52 as core 76 on socket 0
> > EAL: Detected lcore 53 as core 76 on socket 0
> > EAL: Detected lcore 54 as core 76 on socket 0
> > EAL: Detected lcore 55 as core 76 on socket 0
> > EAL: Detected lcore 56 as core 80 on socket 0
> > EAL: Detected lcore 57 as core 80 on socket 0
> > EAL: Detected lcore 58 as core 80 on socket 0
> > EAL: Detected lcore 59 as core 80 on socket 0
> > EAL: Detected lcore 60 as core 84 on socket 0
> > EAL: Detected lcore 61 as core 84 on socket 0
> > EAL: Detected lcore 62 as core 84 on socket 0
> > EAL: Detected lcore 63 as core 84 on socket 0
> > EAL: Detected lcore 64 as core 2048 on socket 8
> > EAL: Detected lcore 65 as core 2048 on socket 8
> > EAL: Detected lcore 66 as core 2048 on socket 8
> > EAL: Detected lcore 67 as core 2048 on socket 8
> > EAL: Detected lcore 68 as core 2052 on socket 8
> > EAL: Detected lcore 69 as core 2052 on socket 8
> > EAL: Detected lcore 70 as core 2052 on socket 8
> > EAL: Detected lcore 71 as core 2052 on socket 8
> > EAL: Detected lcore 72 as core 2056 on socket 8
> > EAL: Detected lcore 73 as core 2056 on socket 8
> > EAL: Detected lcore 74 as core 2056 on socket 8
> > EAL: Detected lcore 75 as core 2056 on socket 8
> > EAL: Detected lcore 76 as core 2060 on socket 8
> > EAL: Detected lcore 77 as core 2060 on socket 8
> > EAL: Detected lcore 78 as core 2060 on socket 8
> > EAL: Detected lcore 79 as core 2060 on socket 8
> > EAL: Detected lcore 80 as core 2072 on socket 8
> > EAL: Detected lcore 81 as core 2072 on socket 8
> > EAL: Detected lcore 82 as core 2072 on socket 8
> > EAL: Detected lcore 83 as core 2072 on socket 8
> > EAL: Detected lcore 84 as core 2076 on socket 8
> > EAL: Detected lcore 85 as core 2076 on socket 8
> > EAL: Detected lcore 86 as core 2076 on socket 8
> > EAL: Detected lcore 87 as core 2076 on socket 8
> > EAL: Detected lcore 88 as core 2080 on socket 8
> > EAL: Detected lcore 89 as core 2080 on socket 8
> > EAL: Detected lcore 90 as core 2080 on socket 8
> > EAL: Detected lcore 91 as core 2080 on socket 8
> > EAL: Detected lcore 92 as core 2084 on socket 8
> > EAL: Detected lcore 93 as core 2084 on socket 8
> > EAL: Detected lcore 94 as core 2084 on socket 8
> > EAL: Detected lcore 95 as core 2084 on socket 8
> > EAL: Detected lcore 96 as core 2088 on socket 8
> > EAL: Detected lcore 97 as core 2088 on socket 8
> > EAL: Detected lcore 98 as core 2088 on socket 8
> > EAL: Detected lcore 99 as core 2088 on socket 8
> > EAL: Detected lcore 100 as core 2092 on socket 8
> > EAL: Detected lcore 101 as core 2092 on socket 8
> > EAL: Detected lcore 102 as core 2092 on socket 8
> > EAL: Detected lcore 103 as core 2092 on socket 8
> > EAL: Detected lcore 104 as core 2096 on socket 8
> > EAL: Detected lcore 105 as core 2096 on socket 8
> > EAL: Detected lcore 106 as core 2096 on socket 8
> > EAL: Detected lcore 107 as core 2096 on socket 8
> > EAL: Detected lcore 108 as core 2100 on socket 8
> > EAL: Detected lcore 109 as core 2100 on socket 8
> > EAL: Detected lcore 110 as core 2100 on socket 8
> > EAL: Detected lcore 111 as core 2100 on socket 8
> > EAL: Detected lcore 112 as core 2120 on socket 8
> > EAL: Detected lcore 113 as core 2120 on socket 8
> > EAL: Detected lcore 114 as core 2120 on socket 8
> > EAL: Detected lcore 115 as core 2120 on socket 8
> > EAL: Detected lcore 116 as core 2124 on socket 8
> > EAL: Detected lcore 117 as core 2124 on socket 8
> > EAL: Detected lcore 118 as core 2124 on socket 8
> > EAL: Detected lcore 119 as core 2124 on socket 8
> > EAL: Detected lcore 120 as core 2136 on socket 8
> > EAL: Detected lcore 121 as core 2136 on socket 8
> > EAL: Detected lcore 122 as core 2136 on socket 8
> > EAL: Detected lcore 123 as core 2136 on socket 8
> > EAL: Detected lcore 124 as core 2140 on socket 8
> > EAL: Detected lcore 125 as core 2140 on socket 8
> > EAL: Detected lcore 126 as core 2140 on socket 8
> > EAL: Detected lcore 127 as core 2140 on socket 8
> > EAL: Support maximum 1536 logical core(s) by configuration.
> > EAL: Detected 128 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Ask a virtual area of 0x10000 bytes
> > EAL: Virtual area found at 0x100000000 (size = 0x10000)
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: DPAA Bus not present. Skipping.
> > EAL: VFIO PCI modules not loaded
> > EAL: Selected IOVA mode 'VA'
> > EAL: 2 hugepages of size 2097152 reserved, but no mounted hugetlbfs found
> > for that size
> > EAL: Probing VFIO support...
> > EAL: IOMMU type 1 (Type 1) is not supported
> > EAL: IOMMU type 7 (sPAPR) is supported
> > EAL: IOMMU type 8 (No-IOMMU) is not supported
> > EAL: VFIO support initialized
> > EAL: Ask a virtual area of 0x30000 bytes
> > EAL: Virtual area found at 0x100010000 (size = 0x30000)
> > EAL: Setting up physically contiguous memory...
> > EAL: Setting maximum number of open files to 32768
> > EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > EAL: Detected memory type: socket_id:8 hugepage_sz:1073741824
> > EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
> > EAL: Ask a virtual area of 0x10000 bytes
> > EAL: Virtual area found at 0x100040000 (size = 0x10000)
> > EAL: Memseg list allocated at socket 0, page size 0x100000kB
> > EAL: Ask a virtual area of 0x800000000 bytes
> > EAL: Virtual area found at 0x140000000 (size = 0x800000000)
> > EAL: VA reserved for memseg list at 0x140000000, size 800000000
> > EAL: Ask a virtual area of 0x10000 bytes
> > EAL: Virtual area found at 0x940000000 (size = 0x10000)
> > EAL: Memseg list allocated at socket 0, page size 0x100000kB
> > EAL: Ask a virtual area of 0x800000000 bytes
> > EAL: Virtual area found at 0x980000000 (size = 0x800000000)
> > EAL: VA reserved for memseg list at 0x980000000, size 800000000
> > EAL: Creating 2 segment lists: n_segs:32 socket_id:8 hugepage_sz:1073741824
> > EAL: Ask a virtual area of 0x10000 bytes
> > EAL: Virtual area found at 0x1180000000 (size = 0x10000)
> > EAL: Memseg list allocated at socket 8, page size 0x100000kB
> > EAL: Ask a virtual area of 0x800000000 bytes
> > EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
> > EAL: VA reserved for memseg list at 0x11c0000000, size 800000000
> > EAL: Ask a virtual area of 0x10000 bytes
> > EAL: Virtual area found at 0x19c0000000 (size = 0x10000)
> > EAL: Memseg list allocated at socket 8, page size 0x100000kB
> > EAL: Ask a virtual area of 0x800000000 bytes
> > EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
> > EAL: VA reserved for memseg list at 0x1a00000000, size 800000000
> > EAL: TSC frequency is ~510000 KHz
> > EAL: Main lcore 64 is ready (tid=7fffb8018890;cpuset=[64])
> > EAL: lcore 65 is ready (tid=7fffb64ad090;cpuset=[65])
> > EAL: lcore 66 is ready (tid=7fffb5c9d090;cpuset=[66])
> > EAL: lcore 67 is ready (tid=7fffb548d090;cpuset=[67])
> > EAL: lcore 68 is ready (tid=7fffb4c7d090;cpuset=[68])
> > EAL: lcore 69 is ready (tid=7fffa7ffd090;cpuset=[69])
> > EAL: lcore 70 is ready (tid=7fffa77ed090;cpuset=[70])
> > EAL: lcore 71 is ready (tid=7fffa6fdd090;cpuset=[71])
> > EAL: lcore 72 is ready (tid=7fffa67cd090;cpuset=[72])
> > EAL: lcore 73 is ready (tid=7fffa5fbd090;cpuset=[73])
> > EAL: lcore 74 is ready (tid=7fffa57ad090;cpuset=[74])
> > EAL: lcore 75 is ready (tid=7fffa4f9d090;cpuset=[75])
> > EAL: lcore 76 is ready (tid=7fff8fffd090;cpuset=[76])
> > EAL: lcore 77 is ready (tid=7fff8f7ed090;cpuset=[77])
> > EAL: lcore 78 is ready (tid=7fff8efdd090;cpuset=[78])
> > EAL: lcore 79 is ready (tid=7fff8e7cd090;cpuset=[79])
> > EAL: lcore 80 is ready (tid=7fff8dfbd090;cpuset=[80])
> > EAL: lcore 81 is ready (tid=7fff8d7ad090;cpuset=[81])
> > EAL: lcore 82 is ready (tid=7fff8cf9d090;cpuset=[82])
> > EAL: lcore 83 is ready (tid=7fff6bffd090;cpuset=[83])
> > EAL: lcore 84 is ready (tid=7fff6b7ed090;cpuset=[84])
> > EAL: lcore 85 is ready (tid=7fff6afdd090;cpuset=[85])
> > EAL: lcore 86 is ready (tid=7fff6a7cd090;cpuset=[86])
> > EAL: lcore 87 is ready (tid=7fff69fbd090;cpuset=[87])
> > EAL: lcore 88 is ready (tid=7fff697ad090;cpuset=[88])
> > EAL: lcore 89 is ready (tid=7fff68f9d090;cpuset=[89])
> > EAL: lcore 90 is ready (tid=7fff4bffd090;cpuset=[90])
> > EAL: lcore 91 is ready (tid=7fff4b7ed090;cpuset=[91])
> > EAL: lcore 92 is ready (tid=7fff4afdd090;cpuset=[92])
> > EAL: lcore 93 is ready (tid=7fff4a7cd090;cpuset=[93])
> > EAL: lcore 94 is ready (tid=7fff49fbd090;cpuset=[94])
> > EAL: lcore 95 is ready (tid=7fff497ad090;cpuset=[95])
> > EAL: lcore 96 is ready (tid=7fff48f9d090;cpuset=[96])
> > EAL: lcore 97 is ready (tid=7fff2bffd090;cpuset=[97])
> > EAL: lcore 98 is ready (tid=7fff2b7ed090;cpuset=[98])
> > EAL: lcore 99 is ready (tid=7fff2afdd090;cpuset=[99])
> > EAL: lcore 100 is ready (tid=7fff2a7cd090;cpuset=[100])
> > EAL: lcore 101 is ready (tid=7fff29fbd090;cpuset=[101])
> > EAL: lcore 102 is ready (tid=7fff297ad090;cpuset=[102])
> > EAL: lcore 103 is ready (tid=7fff28f9d090;cpuset=[103])
> > EAL: lcore 104 is ready (tid=7fff07ffd090;cpuset=[104])
> > EAL: lcore 105 is ready (tid=7ffeff7ed090;cpuset=[105])
> > EAL: lcore 106 is ready (tid=7fff077ed090;cpuset=[106])
> > EAL: lcore 107 is ready (tid=7fff06fdd090;cpuset=[107])
> > EAL: lcore 108 is ready (tid=7fff067cd090;cpuset=[108])
> > EAL: lcore 109 is ready (tid=7fff05fbd090;cpuset=[109])
> > EAL: lcore 110 is ready (tid=7fff057ad090;cpuset=[110])
> > EAL: lcore 111 is ready (tid=7fff04f9d090;cpuset=[111])
> > EAL: lcore 112 is ready (tid=7ffeffffd090;cpuset=[112])
> > EAL: lcore 113 is ready (tid=7ffefefdd090;cpuset=[113])
> > EAL: lcore 114 is ready (tid=7ffefe7cd090;cpuset=[114])
> > EAL: lcore 115 is ready (tid=7ffefdfbd090;cpuset=[115])
> > EAL: lcore 116 is ready (tid=7ffefd7ad090;cpuset=[116])
> > EAL: lcore 117 is ready (tid=7ffefcf9d090;cpuset=[117])
> > EAL: lcore 118 is ready (tid=7ffecfffd090;cpuset=[118])
> > EAL: lcore 119 is ready (tid=7ffecf7ed090;cpuset=[119])
> > EAL: lcore 120 is ready (tid=7ffecefdd090;cpuset=[120])
> > EAL: lcore 121 is ready (tid=7ffece7cd090;cpuset=[121])
> > EAL: lcore 122 is ready (tid=7ffecdfbd090;cpuset=[122])
> > EAL: lcore 123 is ready (tid=7ffecd7ad090;cpuset=[123])
> > EAL: lcore 124 is ready (tid=7ffeccf9d090;cpuset=[124])
> > EAL: lcore 125 is ready (tid=7ffe9bffd090;cpuset=[125])
> > EAL: lcore 126 is ready (tid=7ffe9b7ed090;cpuset=[126])
> > EAL: lcore 127 is ready (tid=7ffe9afdd090;cpuset=[127])
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 8
> > EAL: Restoring previous memory policy: 0
> > EAL: request: mp_malloc_sync
> > EAL: Heap on socket 8 was expanded by 1024MB
> > EAL: PCI device 0000:01:00.0 on NUMA socket 0
> > EAL: probe driver: 15b3:1019 mlx5_pci
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.0 (socket 0)
> > EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered
> > EAL: Trying to obtain current memory policy.
> > EAL: Setting policy MPOL_PREFERRED for socket 0
> > EAL: Restoring previous memory policy: 0
> > EAL: Calling mem event callback 'MLX5_MEM_EVENT_CB:(nil)'
> > EAL: request: mp_malloc_sync
> > EAL: Heap on socket 0 was expanded by 1024MB
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > mlx5_pci: Default miss action is not supported.
> > EAL: PCI device 0000:01:00.1 on NUMA socket 0
> > EAL: probe driver: 15b3:1019 mlx5_pci
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0000:01:00.1 (socket 0)
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > mlx5_pci: Default miss action is not supported.
> > EAL: PCI device 0003:01:00.0 on NUMA socket 0
> > EAL: probe driver: 14e4:168a net_bnx2x
> > EAL: Not managed by a supported kernel driver, skipped
> > EAL: PCI device 0003:01:00.1 on NUMA socket 0
> > EAL: probe driver: 14e4:168a net_bnx2x
> > EAL: Not managed by a supported kernel driver, skipped
> > EAL: PCI device 0003:01:00.2 on NUMA socket 0
> > EAL: probe driver: 14e4:168a net_bnx2x
> > EAL: Not managed by a supported kernel driver, skipped
> > EAL: PCI device 0003:01:00.3 on NUMA socket 0
> > EAL: probe driver: 14e4:168a net_bnx2x
> > EAL: Not managed by a supported kernel driver, skipped
> > EAL: PCI device 0030:01:00.0 on NUMA socket 8
> > EAL: probe driver: 15b3:1019 mlx5_pci
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.0 (socket 8)
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > mlx5_pci: Default miss action is not supported.
> > EAL: PCI device 0030:01:00.1 on NUMA socket 8
> > EAL: probe driver: 15b3:1019 mlx5_pci
> > EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0030:01:00.1 (socket 8)
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > mlx5_pci: Default miss action is not supported.
> > EAL: PCI device 0034:01:00.0 on NUMA socket 8
> > EAL: probe driver: 8086:1583 net_i40e
> > EAL: set IOMMU type 1 (Type 1) failed, error 19 (No such device)
> > EAL: using IOMMU type 7 (sPAPR)
> > EAL: Highest VA address in memseg list is 0x2200000000
> > EAL: Setting DMA window size to 0x4000000000
> > EAL: Mem event callback 'vfio_mem_event_clb:(nil)' registered
> > EAL: Installed memory event callback for VFIO
> > EAL: VFIO reports MSI-X BAR as mappable
> > EAL: PCI memory mapped at 0x2200000000
> > EAL: PCI memory mapped at 0x2200800000
> > EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.0 (socket 8)
> > EAL: PCI device 0034:01:00.1 on NUMA socket 8
> > EAL: probe driver: 8086:1583 net_i40e
> > EAL: VFIO reports MSI-X BAR as mappable
> > EAL: PCI memory mapped at 0x2200810000
> > EAL: PCI memory mapped at 0x2201010000
> > EAL: Probe PCI driver: net_i40e (8086:1583) device: 0034:01:00.1 (socket 8)
> > APP: HPET is not enabled, using TSC as default timer
> > RTE>>vfio_autotest
> > DRC: sz = 0x20000
> > DRC: mem = 0x0x7fffb8680000
> > EAL: dma map attempt outside DMA window
> > EAL: Failed to map DMA
> > EAL: Couldn't map new region for DMA
> > Failed to dma map whole region, ret=-1(No such file or directory)
> > Error vfio dma map/unmap, ret=-1
> > Test Failed
> > RTE>>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] Duplicating traffic with RTE Flow
@ 2021-03-11 16:32 7% ` Jan Viktorin
2021-03-12 9:32 0% ` Jiawei(Jonny) Wang
0 siblings, 1 reply; 15+ results
From: Jan Viktorin @ 2021-03-11 16:32 UTC (permalink / raw)
To: Jiawei(Jonny) Wang; +Cc: Slava Ovsiienko, Asaf Penso, dev, Ori Kam
On Thu, 11 Mar 2021 02:11:07 +0000
"Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
> Hi Jan,
>
> Sorry for late response,
>
> First rule is invalid, port only works on FDB domain so need 'transfer' here;
> Second rule should be ok, could you please check if the port 1 was enabled on you dpdk application?
I assume that it is enabled, see full transcript:
$ ofed_info
MLNX_OFED_LINUX-5.2-1.0.4.0 (OFED-5.2-1.0.4):
...
$ sudo dpdk-testpmd -v -- -i
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: RTE Version: 'DPDK 20.11.0'
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.0 (socket 0)
mlx5_pci: No available register for Sampler.
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.1 (socket 0)
mlx5_pci: No available register for Sampler.
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
Port 0: B8:59:9F:E2:09:F6
Configuring Port 1 (socket 0)
Port 1: B8:59:9F:E2:09:F7
Checking link statuses...
Done
testpmd> port start 1
Port 1 is now not stopped
Please stop the ports first
Done
testpmd> set sample_actions 0 port_id id 1 / end
testpmd> flow validate 0 ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no stated reason): Operation not supported
testpmd> flow create 0 ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no stated reason): Operation not supported
testpmd>
Stopping port 0...
Stopping ports...
Done
Stopping port 1...
Stopping ports...
Done
Shutting down port 0...
Closing ports...
Port 0 is closed
Done
Shutting down port 1...
Closing ports...
Port 1 is closed
Done
Bye...
Jan
>
> Thanks.
> Jonny
>
> > -----Original Message-----
> > From: Jan Viktorin <viktorin@cesnet.cz>
> > Sent: Monday, March 1, 2021 10:43 PM
> > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Ori Kam
> > <orika@nvidia.com>; Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> >
> > On Mon, 1 Mar 2021 14:34:07 +0000
> > Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
> >
> > > Hi, Jan
> > >
> > > To use port action (I see it is in your sample action list) the flow
> > > should be applied to the FDB domain, ie "transfer" attribute should be
> > specified:
> > >
> > > flow validate 0 ingress transfer...
> >
> > As you can see (however, it's a bit messy in the response below, in [1], it is
> > better), I tried both. First without transfer and second with. The first gives
> > hint "action is valid in transfer mode only" but the second try with transfer
> > gives "Operation not supported".
> >
> > Jan
> >
> > [1] http://mails.dpdk.org/archives/dev/2021-March/200475.html
> >
> > >
> > > With best regards, Slava
> > >
> > > > -----Original Message-----
> > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > Sent: Monday, March 1, 2021 14:21
> > > > To: Asaf Penso <asafp@nvidia.com>
> > > > Cc: dev@dpdk.org; Ori Kam <orika@nvidia.com>; Jiawei(Jonny) Wang
> > > > <jiaweiw@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > >
> > > > Hello Asaf,
> > > >
> > > > it is a while we were in touch regarding this topic. Finally, I am
> > > > again trying to get work this feature. I've seen that sampling is
> > > > already upstreamed which is great. However, I am not very successful
> > > > with that. There is nearly no documentation, just [1], I found no examples,
> > just commit logs...
> > > >
> > > > I tried:
> > > >
> > > > > set sample_actions 0 port_id id 1 / end > flow validate 0
> > > > ingress pattern end actions sample ratio 1 index 0 / drop / end
> > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > port id action is valid in transfer mode only: Operation not
> > > > supported > flow validate
> > > > 0 ingress transfer pattern end actions sample ratio 1 index 0 / drop
> > > > / end
> > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > (no stated reason): Operation not supported
> > > >
> > > > Using CentOS 7, DPDK 20.11.0, OFED-5.2-1.0.4.
> > > > NICs: MT2892 Family [ConnectX-6 Dx] 101d (fw 22.28.1002), MT27800
> > > > Family [ConnectX-5] 1017 (fw 16.27.2008).
> > > >
> > > > My primary goal is to be able to deliver exactly the same packets
> > > > both to DPDK and to the Linux kernel. Doing this at RTE Flow level
> > > > would be great due to performance and transparency.
> > > >
> > > > Jan
> > > >
> > > > [1]
> > > > https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-sample
> > > >
> > > > On Fri, 18 Sep 2020 14:23:42 +0000
> > > > Asaf Penso <asafp@nvidia.com> wrote:
> > > >
> > > > > Hello Jan,
> > > > >
> > > > > You can have a look in series [1] where we propose to add APIs to
> > > > DPDK20.11 for both mirroring and sampling for packets, with
> > > > additional actions of the different traffic.
> > > > >
> > > > > [1]
> > > > > http://patches.dpdk.org/project/dpdk/list/?series=12045
> > > > >
> > > > > Regards,
> > > > > Asaf Penso
> > > > >
> > > > > >-----Original Message-----
> > > > > >From: dev <dev-bounces@dpdk.org> On Behalf Of Jan Viktorin
> > > > > >Sent: Friday, September 18, 2020 3:56 PM
> > > > > >To: dev@dpdk.org
> > > > > >Subject: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > >
> > > > > >Hello all,
> > > > > >
> > > > > >we are looking for a way to duplicate ingress traffic in hardware.
> > > > > >
> > > > > >There is an example in [1] suggesting to insert two fate actions
> > > > > >into the RTE Flow actions array like:
> > > > > >
> > > > > > flow create 0 ingress pattern end \
> > > > > > actions queue index 0 / void / queue index 1 / end
> > > > > >
> > > > > >But our experience is that PMDs reject two fate actions (tried
> > > > > >with mlx5). Another similar approach would be to deliver every
> > > > > >single packet into two virtual
> > > > > >functions:
> > > > > >
> > > > > > flow create 0 ingress pattern end \
> > > > > > actions vf index 0 / vf index 1 / end
> > > > > >
> > > > > >Third possibility was to use passthru:
> > > > > >
> > > > > > flow create 0 ingress pattern end \
> > > > > > actions passthru / vf index 0 / end flow create 0 ingress
> > > > > > pattern end \
> > > > > > actions vf index 1 / end
> > > > > >
> > > > > >Again, tried on mlx5 and it does not support the passthru.
> > > > > >
> > > > > >Last idea was to use isolate with passthru (to deliver both to
> > > > > >DPDK application and to the kernel) but again there was no
> > > > > >support on mlx5 for
> > > > passthru...
> > > > > >
> > > > > > flow isolate 0 true
> > > > > > flow create 0 ingress pattern end actions passthru / rss end /
> > > > > > end
> > > > > >
> > > > > >Is there any other possibility or PMD+NIC that is known to solve
> > > > > >such
> > > > issue?
> > > > > >
> > > > > >Thanks
> > > > > >Jan Viktorin
> > > > > >
> > > > > >[1]
> > > > > >https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%
> > 2
> > > > > >Fdoc
> > > > > >.dpdk
> > > > > >.org%2Fguides%2Fprog_guide%2Frte_flow.html%23table-rte-flow-
> > redir
> > > > > >ect-
> > > > > >queue-5-
> > > > >
> > > > >3&data=02%7C01%7Casafp%40nvidia.com%7C1a46005bec5245e72
> > 9e70
> > > > 8d
> > > > >
> > > > >85bd24caf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C6373
> > 6030
> > > > 60
> > > > >
> > > > >73519816&sdata=EOF%2Fz62crvBZK8rwzwKIWxj5cVlfPVnU3FLmcL9
> > X2w0
> > > > %3
> > > > > >D&reserved=0
> > >
>
^ permalink raw reply [relevance 7%]
* Re: [dpdk-dev] Duplicating traffic with RTE Flow
2021-03-11 16:32 7% ` Jan Viktorin
@ 2021-03-12 9:32 0% ` Jiawei(Jonny) Wang
2021-03-15 13:22 5% ` Jan Viktorin
0 siblings, 1 reply; 15+ results
From: Jiawei(Jonny) Wang @ 2021-03-12 9:32 UTC (permalink / raw)
To: Jan Viktorin; +Cc: Slava Ovsiienko, Asaf Penso, dev, Ori Kam
Hi Jan,
> -----Original Message-----
> From: Jan Viktorin <viktorin@cesnet.cz>
> Sent: Friday, March 12, 2021 12:33 AM
> To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> Cc: Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>; dev@dpdk.org; Ori Kam <orika@nvidia.com>
> Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
>
> On Thu, 11 Mar 2021 02:11:07 +0000
> "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
>
> > Hi Jan,
> >
> > Sorry for late response,
> >
> > First rule is invalid, port only works on FDB domain so need
> > 'transfer' here; Second rule should be ok, could you please check if the
> port 1 was enabled on you dpdk application?
>
> I assume that it is enabled, see full transcript:
>
> $ ofed_info
> MLNX_OFED_LINUX-5.2-1.0.4.0 (OFED-5.2-1.0.4):
> ...
> $ sudo dpdk-testpmd -v -- -i
> EAL: Detected 24 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: RTE Version: 'DPDK 20.11.0'
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.0 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.1 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> EAL: No legacy callbacks, legacy socket not created Interactive-mode
> selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176,
> socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0
> (socket 0) Port 0: B8:59:9F:E2:09:F6 Configuring Port 1 (socket 0) Port 1:
> B8:59:9F:E2:09:F7 Checking link statuses...
> Done
Seems that you start two PF port here, Port 1 is not VF port;
FDB rule can steering the packet form PF to its VFs and vice versa, Could you please try to open the
VF ports and start the testpmd with representor=.
Thanks.
> testpmd> port start 1
> Port 1 is now not stopped
> Please stop the ports first
> Done
> testpmd> set sample_actions 0 port_id id 1 / end testpmd> flow validate 0
> ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
> port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> stated reason): Operation not supported testpmd> flow create 0 ingress
> transfer pattern end actions sample ratio 1 index 0 / drop / end
> port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> stated reason): Operation not supported testpmd> Stopping port 0...
> Stopping ports...
> Done
>
> Stopping port 1...
> Stopping ports...
> Done
>
> Shutting down port 0...
> Closing ports...
> Port 0 is closed
> Done
>
> Shutting down port 1...
> Closing ports...
> Port 1 is closed
> Done
>
> Bye...
>
> Jan
>
> >
> > Thanks.
> > Jonny
> >
> > > -----Original Message-----
> > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > Sent: Monday, March 1, 2021 10:43 PM
> > > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > > Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Ori Kam
> > > <orika@nvidia.com>; Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > >
> > > On Mon, 1 Mar 2021 14:34:07 +0000
> > > Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
> > >
> > > > Hi, Jan
> > > >
> > > > To use port action (I see it is in your sample action list) the
> > > > flow should be applied to the FDB domain, ie "transfer" attribute
> > > > should be
> > > specified:
> > > >
> > > > flow validate 0 ingress transfer...
> > >
> > > As you can see (however, it's a bit messy in the response below, in
> > > [1], it is better), I tried both. First without transfer and second
> > > with. The first gives hint "action is valid in transfer mode only"
> > > but the second try with transfer gives "Operation not supported".
> > >
> > > Jan
> > >
> > > [1] http://mails.dpdk.org/archives/dev/2021-March/200475.html
> > >
> > > >
> > > > With best regards, Slava
> > > >
> > > > > -----Original Message-----
> > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > Sent: Monday, March 1, 2021 14:21
> > > > > To: Asaf Penso <asafp@nvidia.com>
> > > > > Cc: dev@dpdk.org; Ori Kam <orika@nvidia.com>; Jiawei(Jonny) Wang
> > > > > <jiaweiw@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > >
> > > > > Hello Asaf,
> > > > >
> > > > > it is a while we were in touch regarding this topic. Finally, I
> > > > > am again trying to get work this feature. I've seen that
> > > > > sampling is already upstreamed which is great. However, I am not
> > > > > very successful with that. There is nearly no documentation,
> > > > > just [1], I found no examples,
> > > just commit logs...
> > > > >
> > > > > I tried:
> > > > >
> > > > > > set sample_actions 0 port_id id 1 / end > flow validate 0
> > > > > ingress pattern end actions sample ratio 1 index 0 / drop / end
> > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > port id action is valid in transfer mode only: Operation not
> > > > > supported > flow validate
> > > > > 0 ingress transfer pattern end actions sample ratio 1 index 0 /
> > > > > drop / end
> > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > (no stated reason): Operation not supported
> > > > >
> > > > > Using CentOS 7, DPDK 20.11.0, OFED-5.2-1.0.4.
> > > > > NICs: MT2892 Family [ConnectX-6 Dx] 101d (fw 22.28.1002),
> > > > > MT27800 Family [ConnectX-5] 1017 (fw 16.27.2008).
> > > > >
> > > > > My primary goal is to be able to deliver exactly the same
> > > > > packets both to DPDK and to the Linux kernel. Doing this at RTE
> > > > > Flow level would be great due to performance and transparency.
> > > > >
> > > > > Jan
> > > > >
> > > > > [1]
> > > > > https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-samp
> > > > > le
> > > > >
> > > > > On Fri, 18 Sep 2020 14:23:42 +0000 Asaf Penso <asafp@nvidia.com>
> > > > > wrote:
> > > > >
> > > > > > Hello Jan,
> > > > > >
> > > > > > You can have a look in series [1] where we propose to add APIs
> > > > > > to
> > > > > DPDK20.11 for both mirroring and sampling for packets, with
> > > > > additional actions of the different traffic.
> > > > > >
> > > > > > [1]
> > > > > > http://patches.dpdk.org/project/dpdk/list/?series=12045
> > > > > >
> > > > > > Regards,
> > > > > > Asaf Penso
> > > > > >
> > > > > > >-----Original Message-----
> > > > > > >From: dev <dev-bounces@dpdk.org> On Behalf Of Jan Viktorin
> > > > > > >Sent: Friday, September 18, 2020 3:56 PM
> > > > > > >To: dev@dpdk.org
> > > > > > >Subject: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > >
> > > > > > >Hello all,
> > > > > > >
> > > > > > >we are looking for a way to duplicate ingress traffic in hardware.
> > > > > > >
> > > > > > >There is an example in [1] suggesting to insert two fate
> > > > > > >actions into the RTE Flow actions array like:
> > > > > > >
> > > > > > > flow create 0 ingress pattern end \
> > > > > > > actions queue index 0 / void / queue index 1 / end
> > > > > > >
> > > > > > >But our experience is that PMDs reject two fate actions
> > > > > > >(tried with mlx5). Another similar approach would be to
> > > > > > >deliver every single packet into two virtual
> > > > > > >functions:
> > > > > > >
> > > > > > > flow create 0 ingress pattern end \
> > > > > > > actions vf index 0 / vf index 1 / end
> > > > > > >
> > > > > > >Third possibility was to use passthru:
> > > > > > >
> > > > > > > flow create 0 ingress pattern end \
> > > > > > > actions passthru / vf index 0 / end flow create 0
> > > > > > > ingress pattern end \
> > > > > > > actions vf index 1 / end
> > > > > > >
> > > > > > >Again, tried on mlx5 and it does not support the passthru.
> > > > > > >
> > > > > > >Last idea was to use isolate with passthru (to deliver both
> > > > > > >to DPDK application and to the kernel) but again there was no
> > > > > > >support on mlx5 for
> > > > > passthru...
> > > > > > >
> > > > > > > flow isolate 0 true
> > > > > > > flow create 0 ingress pattern end actions passthru / rss
> > > > > > > end / end
> > > > > > >
> > > > > > >Is there any other possibility or PMD+NIC that is known to
> > > > > > >solve such
> > > > > issue?
> > > > > > >
> > > > > > >Thanks
> > > > > > >Jan Viktorin
> > > > > > >
> > > > > > >[1]
> > > > > > >https://nam11.safelinks.protection.outlook.com/?url=https%3A%
> > > > > > >2F%
> > > 2
> > > > > > >Fdoc
> > > > > > >.dpdk
> > > > > > >.org%2Fguides%2Fprog_guide%2Frte_flow.html%23table-rte-flow-
> > > redir
> > > > > > >ect-
> > > > > > >queue-5-
> > > > > >
> > > > > >3&data=02%7C01%7Casafp%40nvidia.com%7C1a46005bec5245
> e72
> > > 9e70
> > > > > 8d
> > > > > >
> > > > > >85bd24caf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C6
> 373
> > > 6030
> > > > > 60
> > > > > >
> > > > > >73519816&sdata=EOF%2Fz62crvBZK8rwzwKIWxj5cVlfPVnU3FLm
> cL9
> > > X2w0
> > > > > %3
> > > > > > >D&reserved=0
> > > >
> >
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] Duplicating traffic with RTE Flow
2021-03-12 9:32 0% ` Jiawei(Jonny) Wang
@ 2021-03-15 13:22 5% ` Jan Viktorin
2021-03-15 17:57 0% ` Jan Viktorin
2021-03-16 4:00 0% ` Jiawei(Jonny) Wang
0 siblings, 2 replies; 15+ results
From: Jan Viktorin @ 2021-03-15 13:22 UTC (permalink / raw)
To: Jiawei(Jonny) Wang; +Cc: Slava Ovsiienko, Asaf Penso, dev, Ori Kam
Hello Jiawei,
On Fri, 12 Mar 2021 09:32:44 +0000
"Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
> Hi Jan,
>
> > -----Original Message-----
> > From: Jan Viktorin <viktorin@cesnet.cz>
> > Sent: Friday, March 12, 2021 12:33 AM
> > To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > Cc: Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> > <asafp@nvidia.com>; dev@dpdk.org; Ori Kam <orika@nvidia.com>
> > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> >
> > On Thu, 11 Mar 2021 02:11:07 +0000
> > "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
> >
> > > Hi Jan,
> > >
> > > Sorry for late response,
> > >
> > > First rule is invalid, port only works on FDB domain so need
> > > 'transfer' here; Second rule should be ok, could you please check if the
> > port 1 was enabled on you dpdk application?
> >
> > I assume that it is enabled, see full transcript:
> >
> > $ ofed_info
> > MLNX_OFED_LINUX-5.2-1.0.4.0 (OFED-5.2-1.0.4):
> > ...
> > $ sudo dpdk-testpmd -v -- -i
> > EAL: Detected 24 lcore(s)
> > EAL: Detected 1 NUMA nodes
> > EAL: RTE Version: 'DPDK 20.11.0'
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'PA'
> > EAL: No available hugepages reported in hugepages-1048576kB
> > EAL: Probing VFIO support...
> > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.0 (socket 0)
> > mlx5_pci: No available register for Sampler.
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.1 (socket 0)
> > mlx5_pci: No available register for Sampler.
> > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > EAL: No legacy callbacks, legacy socket not created Interactive-mode
> > selected
> > testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176,
> > socket=0
> > testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0
> > (socket 0) Port 0: B8:59:9F:E2:09:F6 Configuring Port 1 (socket 0) Port 1:
> > B8:59:9F:E2:09:F7 Checking link statuses...
> > Done
>
> Seems that you start two PF port here, Port 1 is not VF port;
> FDB rule can steering the packet form PF to its VFs and vice versa, Could you please try to open the
> VF ports and start the testpmd with representor=.
I did not know this, so I tried with VFs:
# echo 2 > /sys/class/net/hge1/device/sriov_numvfs
# echo switchdev > /sys/class/net/hge1/compat/devlink/mode
# dpdk-testpmd -v -a '0000:05:00.1,representor=[0-1]' -- -i
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: RTE Version: 'DPDK 20.11.0'
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:05:00.1 (socket 0)
mlx5_pci: No available register for Sampler.
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: No available register for Sampler.
mlx5_pci: No available register for Sampler.
EAL: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.
Configuring Port 0 (socket 0)
Port 0: B8:59:9F:E2:09:F7
Configuring Port 1 (socket 0)
Port 1: B2:57:D6:72:F3:31
Configuring Port 2 (socket 0)
Port 2: 9E:CB:D0:73:59:CE
Checking link statuses...
Done
testpmd> show port summary all
Number of available ports: 3
Port MAC Address Name Driver Status Link
0 B8:59:9F:E2:09:F7 0000:05:00.1 mlx5_pci up 100 Gbps
1 B2:57:D6:72:F3:31 0000:05:00.1_representor_0 mlx5_pci up 100 Gbps
2 9E:CB:D0:73:59:CE 0000:05:00.1_representor_1 mlx5_pci up 100 Gbps
testpmd> set sample_actions 0 port_id id 1 / end
testpmd> flow validate 0 ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
port_flow_complain(): Caught PMD error type 1 (cause unspecified): sample action not supported: Operation not supported
Still no luck. However, there is this message 3-times in the log:
mlx5_pci: No available register for Sampler.
It looks like it might be related. What does it mean?
Jan
>
> Thanks.
>
> > testpmd> port start 1
> > Port 1 is now not stopped
> > Please stop the ports first
> > Done
> > testpmd> set sample_actions 0 port_id id 1 / end testpmd> flow validate 0
> > ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
> > port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> > stated reason): Operation not supported testpmd> flow create 0 ingress
> > transfer pattern end actions sample ratio 1 index 0 / drop / end
> > port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> > stated reason): Operation not supported testpmd> Stopping port 0...
> > Stopping ports...
> > Done
> >
> > Stopping port 1...
> > Stopping ports...
> > Done
> >
> > Shutting down port 0...
> > Closing ports...
> > Port 0 is closed
> > Done
> >
> > Shutting down port 1...
> > Closing ports...
> > Port 1 is closed
> > Done
> >
> > Bye...
> >
> > Jan
> >
> > >
> > > Thanks.
> > > Jonny
> > >
> > > > -----Original Message-----
> > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > Sent: Monday, March 1, 2021 10:43 PM
> > > > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Ori Kam
> > > > <orika@nvidia.com>; Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > >
> > > > On Mon, 1 Mar 2021 14:34:07 +0000
> > > > Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
> > > >
> > > > > Hi, Jan
> > > > >
> > > > > To use port action (I see it is in your sample action list) the
> > > > > flow should be applied to the FDB domain, ie "transfer" attribute
> > > > > should be
> > > > specified:
> > > > >
> > > > > flow validate 0 ingress transfer...
> > > >
> > > > As you can see (however, it's a bit messy in the response below, in
> > > > [1], it is better), I tried both. First without transfer and second
> > > > with. The first gives hint "action is valid in transfer mode only"
> > > > but the second try with transfer gives "Operation not supported".
> > > >
> > > > Jan
> > > >
> > > > [1] http://mails.dpdk.org/archives/dev/2021-March/200475.html
> > > >
> > > > >
> > > > > With best regards, Slava
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > > Sent: Monday, March 1, 2021 14:21
> > > > > > To: Asaf Penso <asafp@nvidia.com>
> > > > > > Cc: dev@dpdk.org; Ori Kam <orika@nvidia.com>; Jiawei(Jonny) Wang
> > > > > > <jiaweiw@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > >
> > > > > > Hello Asaf,
> > > > > >
> > > > > > it is a while we were in touch regarding this topic. Finally, I
> > > > > > am again trying to get work this feature. I've seen that
> > > > > > sampling is already upstreamed which is great. However, I am not
> > > > > > very successful with that. There is nearly no documentation,
> > > > > > just [1], I found no examples,
> > > > just commit logs...
> > > > > >
> > > > > > I tried:
> > > > > >
> > > > > > > set sample_actions 0 port_id id 1 / end > flow validate 0
> > > > > > ingress pattern end actions sample ratio 1 index 0 / drop / end
> > > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > > port id action is valid in transfer mode only: Operation not
> > > > > > supported > flow validate
> > > > > > 0 ingress transfer pattern end actions sample ratio 1 index 0 /
> > > > > > drop / end
> > > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > > (no stated reason): Operation not supported
> > > > > >
> > > > > > Using CentOS 7, DPDK 20.11.0, OFED-5.2-1.0.4.
> > > > > > NICs: MT2892 Family [ConnectX-6 Dx] 101d (fw 22.28.1002),
> > > > > > MT27800 Family [ConnectX-5] 1017 (fw 16.27.2008).
> > > > > >
> > > > > > My primary goal is to be able to deliver exactly the same
> > > > > > packets both to DPDK and to the Linux kernel. Doing this at RTE
> > > > > > Flow level would be great due to performance and transparency.
> > > > > >
> > > > > > Jan
> > > > > >
> > > > > > [1]
> > > > > > https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-samp
> > > > > > le
> > > > > >
> > > > > > On Fri, 18 Sep 2020 14:23:42 +0000 Asaf Penso <asafp@nvidia.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Jan,
> > > > > > >
> > > > > > > You can have a look in series [1] where we propose to add APIs
> > > > > > > to
> > > > > > DPDK20.11 for both mirroring and sampling for packets, with
> > > > > > additional actions of the different traffic.
> > > > > > >
> > > > > > > [1]
> > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=12045
> > > > > > >
> > > > > > > Regards,
> > > > > > > Asaf Penso
> > > > > > >
> > > > > > > >-----Original Message-----
> > > > > > > >From: dev <dev-bounces@dpdk.org> On Behalf Of Jan Viktorin
> > > > > > > >Sent: Friday, September 18, 2020 3:56 PM
> > > > > > > >To: dev@dpdk.org
> > > > > > > >Subject: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > > >
> > > > > > > >Hello all,
> > > > > > > >
> > > > > > > >we are looking for a way to duplicate ingress traffic in hardware.
> > > > > > > >
> > > > > > > >There is an example in [1] suggesting to insert two fate
> > > > > > > >actions into the RTE Flow actions array like:
> > > > > > > >
> > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > actions queue index 0 / void / queue index 1 / end
> > > > > > > >
> > > > > > > >But our experience is that PMDs reject two fate actions
> > > > > > > >(tried with mlx5). Another similar approach would be to
> > > > > > > >deliver every single packet into two virtual
> > > > > > > >functions:
> > > > > > > >
> > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > actions vf index 0 / vf index 1 / end
> > > > > > > >
> > > > > > > >Third possibility was to use passthru:
> > > > > > > >
> > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > actions passthru / vf index 0 / end flow create 0
> > > > > > > > ingress pattern end \
> > > > > > > > actions vf index 1 / end
> > > > > > > >
> > > > > > > >Again, tried on mlx5 and it does not support the passthru.
> > > > > > > >
> > > > > > > >Last idea was to use isolate with passthru (to deliver both
> > > > > > > >to DPDK application and to the kernel) but again there was no
> > > > > > > >support on mlx5 for
> > > > > > passthru...
> > > > > > > >
> > > > > > > > flow isolate 0 true
> > > > > > > > flow create 0 ingress pattern end actions passthru / rss
> > > > > > > > end / end
> > > > > > > >
> > > > > > > >Is there any other possibility or PMD+NIC that is known to
> > > > > > > >solve such
> > > > > > issue?
> > > > > > > >
> > > > > > > >Thanks
> > > > > > > >Jan Viktorin
> > > > > > > >
> > > > > > > >[1]
> > > > > > > >https://nam11.safelinks.protection.outlook.com/?url=https%3A%
> > > > > > > >2F%
> > > > 2
> > > > > > > >Fdoc
> > > > > > > >.dpdk
> > > > > > > >.org%2Fguides%2Fprog_guide%2Frte_flow.html%23table-rte-flow-
> > > > redir
> > > > > > > >ect-
> > > > > > > >queue-5-
> > > > > > >
> > > > > > >3&data=02%7C01%7Casafp%40nvidia.com%7C1a46005bec5245
> > e72
> > > > 9e70
> > > > > > 8d
> > > > > > >
> > > > > > >85bd24caf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C6
> > 373
> > > > 6030
> > > > > > 60
> > > > > > >
> > > > > > >73519816&sdata=EOF%2Fz62crvBZK8rwzwKIWxj5cVlfPVnU3FLm
> > cL9
> > > > X2w0
> > > > > > %3
> > > > > > > >D&reserved=0
> > > > >
> > >
>
^ permalink raw reply [relevance 5%]
* Re: [dpdk-dev] Duplicating traffic with RTE Flow
2021-03-15 13:22 5% ` Jan Viktorin
@ 2021-03-15 17:57 0% ` Jan Viktorin
2021-03-16 4:00 0% ` Jiawei(Jonny) Wang
1 sibling, 0 replies; 15+ results
From: Jan Viktorin @ 2021-03-15 17:57 UTC (permalink / raw)
To: Jiawei(Jonny) Wang; +Cc: Slava Ovsiienko, Asaf Penso, dev, Ori Kam
OK, after firmware update, it seems that it works now.
Thank you for help!
Jan
On Mon, 15 Mar 2021 14:22:16 +0100
Jan Viktorin <viktorin@cesnet.cz> wrote:
> Hello Jiawei,
>
> On Fri, 12 Mar 2021 09:32:44 +0000
> "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
>
> > Hi Jan,
> >
> > > -----Original Message-----
> > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > Sent: Friday, March 12, 2021 12:33 AM
> > > To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > Cc: Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> > > <asafp@nvidia.com>; dev@dpdk.org; Ori Kam <orika@nvidia.com>
> > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > >
> > > On Thu, 11 Mar 2021 02:11:07 +0000
> > > "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
> > >
> > > > Hi Jan,
> > > >
> > > > Sorry for late response,
> > > >
> > > > First rule is invalid, port only works on FDB domain so need
> > > > 'transfer' here; Second rule should be ok, could you please check if the
> > > port 1 was enabled on you dpdk application?
> > >
> > > I assume that it is enabled, see full transcript:
> > >
> > > $ ofed_info
> > > MLNX_OFED_LINUX-5.2-1.0.4.0 (OFED-5.2-1.0.4):
> > > ...
> > > $ sudo dpdk-testpmd -v -- -i
> > > EAL: Detected 24 lcore(s)
> > > EAL: Detected 1 NUMA nodes
> > > EAL: RTE Version: 'DPDK 20.11.0'
> > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > EAL: Selected IOVA mode 'PA'
> > > EAL: No available hugepages reported in hugepages-1048576kB
> > > EAL: Probing VFIO support...
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.0 (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.1 (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > > EAL: No legacy callbacks, legacy socket not created Interactive-mode
> > > selected
> > > testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176,
> > > socket=0
> > > testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0
> > > (socket 0) Port 0: B8:59:9F:E2:09:F6 Configuring Port 1 (socket 0) Port 1:
> > > B8:59:9F:E2:09:F7 Checking link statuses...
> > > Done
> >
> > Seems that you start two PF port here, Port 1 is not VF port;
> > FDB rule can steering the packet form PF to its VFs and vice versa, Could you please try to open the
> > VF ports and start the testpmd with representor=.
>
> I did not know this, so I tried with VFs:
>
> # echo 2 > /sys/class/net/hge1/device/sriov_numvfs
> # echo switchdev > /sys/class/net/hge1/compat/devlink/mode
>
> # dpdk-testpmd -v -a '0000:05:00.1,representor=[0-1]' -- -i
> EAL: Detected 24 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: RTE Version: 'DPDK 20.11.0'
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:05:00.1 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: No available register for Sampler.
> mlx5_pci: No available register for Sampler.
> EAL: No legacy callbacks, legacy socket not created
> Interactive-mode selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
>
> Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.
>
> Configuring Port 0 (socket 0)
> Port 0: B8:59:9F:E2:09:F7
> Configuring Port 1 (socket 0)
> Port 1: B2:57:D6:72:F3:31
> Configuring Port 2 (socket 0)
> Port 2: 9E:CB:D0:73:59:CE
> Checking link statuses...
> Done
> testpmd> show port summary all
> Number of available ports: 3
> Port MAC Address Name Driver Status Link
> 0 B8:59:9F:E2:09:F7 0000:05:00.1 mlx5_pci up 100 Gbps
> 1 B2:57:D6:72:F3:31 0000:05:00.1_representor_0 mlx5_pci up 100 Gbps
> 2 9E:CB:D0:73:59:CE 0000:05:00.1_representor_1 mlx5_pci up 100 Gbps
> testpmd> set sample_actions 0 port_id id 1 / end
> testpmd> flow validate 0 ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
> port_flow_complain(): Caught PMD error type 1 (cause unspecified): sample action not supported: Operation not supported
>
> Still no luck. However, there is this message 3-times in the log:
>
> mlx5_pci: No available register for Sampler.
>
> It looks like it might be related. What does it mean?
>
> Jan
>
> >
> > Thanks.
> >
> > > testpmd> port start 1
> > > Port 1 is now not stopped
> > > Please stop the ports first
> > > Done
> > > testpmd> set sample_actions 0 port_id id 1 / end testpmd> flow validate 0
> > > ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
> > > port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> > > stated reason): Operation not supported testpmd> flow create 0 ingress
> > > transfer pattern end actions sample ratio 1 index 0 / drop / end
> > > port_flow_complain(): Caught PMD error type 1 (cause unspecified): (no
> > > stated reason): Operation not supported testpmd> Stopping port 0...
> > > Stopping ports...
> > > Done
> > >
> > > Stopping port 1...
> > > Stopping ports...
> > > Done
> > >
> > > Shutting down port 0...
> > > Closing ports...
> > > Port 0 is closed
> > > Done
> > >
> > > Shutting down port 1...
> > > Closing ports...
> > > Port 1 is closed
> > > Done
> > >
> > > Bye...
> > >
> > > Jan
> > >
> > > >
> > > > Thanks.
> > > > Jonny
> > > >
> > > > > -----Original Message-----
> > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > Sent: Monday, March 1, 2021 10:43 PM
> > > > > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > > Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Ori Kam
> > > > > <orika@nvidia.com>; Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > >
> > > > > On Mon, 1 Mar 2021 14:34:07 +0000
> > > > > Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
> > > > >
> > > > > > Hi, Jan
> > > > > >
> > > > > > To use port action (I see it is in your sample action list) the
> > > > > > flow should be applied to the FDB domain, ie "transfer" attribute
> > > > > > should be
> > > > > specified:
> > > > > >
> > > > > > flow validate 0 ingress transfer...
> > > > >
> > > > > As you can see (however, it's a bit messy in the response below, in
> > > > > [1], it is better), I tried both. First without transfer and second
> > > > > with. The first gives hint "action is valid in transfer mode only"
> > > > > but the second try with transfer gives "Operation not supported".
> > > > >
> > > > > Jan
> > > > >
> > > > > [1] http://mails.dpdk.org/archives/dev/2021-March/200475.html
> > > > >
> > > > > >
> > > > > > With best regards, Slava
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > > > Sent: Monday, March 1, 2021 14:21
> > > > > > > To: Asaf Penso <asafp@nvidia.com>
> > > > > > > Cc: dev@dpdk.org; Ori Kam <orika@nvidia.com>; Jiawei(Jonny) Wang
> > > > > > > <jiaweiw@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > >
> > > > > > > Hello Asaf,
> > > > > > >
> > > > > > > it is a while we were in touch regarding this topic. Finally, I
> > > > > > > am again trying to get work this feature. I've seen that
> > > > > > > sampling is already upstreamed which is great. However, I am not
> > > > > > > very successful with that. There is nearly no documentation,
> > > > > > > just [1], I found no examples,
> > > > > just commit logs...
> > > > > > >
> > > > > > > I tried:
> > > > > > >
> > > > > > > > set sample_actions 0 port_id id 1 / end > flow validate 0
> > > > > > > ingress pattern end actions sample ratio 1 index 0 / drop / end
> > > > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > > > port id action is valid in transfer mode only: Operation not
> > > > > > > supported > flow validate
> > > > > > > 0 ingress transfer pattern end actions sample ratio 1 index 0 /
> > > > > > > drop / end
> > > > > > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > > > > > (no stated reason): Operation not supported
> > > > > > >
> > > > > > > Using CentOS 7, DPDK 20.11.0, OFED-5.2-1.0.4.
> > > > > > > NICs: MT2892 Family [ConnectX-6 Dx] 101d (fw 22.28.1002),
> > > > > > > MT27800 Family [ConnectX-5] 1017 (fw 16.27.2008).
> > > > > > >
> > > > > > > My primary goal is to be able to deliver exactly the same
> > > > > > > packets both to DPDK and to the Linux kernel. Doing this at RTE
> > > > > > > Flow level would be great due to performance and transparency.
> > > > > > >
> > > > > > > Jan
> > > > > > >
> > > > > > > [1]
> > > > > > > https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-samp
> > > > > > > le
> > > > > > >
> > > > > > > On Fri, 18 Sep 2020 14:23:42 +0000 Asaf Penso <asafp@nvidia.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Jan,
> > > > > > > >
> > > > > > > > You can have a look in series [1] where we propose to add APIs
> > > > > > > > to
> > > > > > > DPDK20.11 for both mirroring and sampling for packets, with
> > > > > > > additional actions of the different traffic.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=12045
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Asaf Penso
> > > > > > > >
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: dev <dev-bounces@dpdk.org> On Behalf Of Jan Viktorin
> > > > > > > > >Sent: Friday, September 18, 2020 3:56 PM
> > > > > > > > >To: dev@dpdk.org
> > > > > > > > >Subject: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > > > >
> > > > > > > > >Hello all,
> > > > > > > > >
> > > > > > > > >we are looking for a way to duplicate ingress traffic in hardware.
> > > > > > > > >
> > > > > > > > >There is an example in [1] suggesting to insert two fate
> > > > > > > > >actions into the RTE Flow actions array like:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions queue index 0 / void / queue index 1 / end
> > > > > > > > >
> > > > > > > > >But our experience is that PMDs reject two fate actions
> > > > > > > > >(tried with mlx5). Another similar approach would be to
> > > > > > > > >deliver every single packet into two virtual
> > > > > > > > >functions:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions vf index 0 / vf index 1 / end
> > > > > > > > >
> > > > > > > > >Third possibility was to use passthru:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions passthru / vf index 0 / end flow create 0
> > > > > > > > > ingress pattern end \
> > > > > > > > > actions vf index 1 / end
> > > > > > > > >
> > > > > > > > >Again, tried on mlx5 and it does not support the passthru.
> > > > > > > > >
> > > > > > > > >Last idea was to use isolate with passthru (to deliver both
> > > > > > > > >to DPDK application and to the kernel) but again there was no
> > > > > > > > >support on mlx5 for
> > > > > > > passthru...
> > > > > > > > >
> > > > > > > > > flow isolate 0 true
> > > > > > > > > flow create 0 ingress pattern end actions passthru / rss
> > > > > > > > > end / end
> > > > > > > > >
> > > > > > > > >Is there any other possibility or PMD+NIC that is known to
> > > > > > > > >solve such
> > > > > > > issue?
> > > > > > > > >
> > > > > > > > >Thanks
> > > > > > > > >Jan Viktorin
> > > > > > > > >
> > > > > > > > >[1]
> > > > > > > > >https://nam11.safelinks.protection.outlook.com/?url=https%3A%
> > > > > > > > >2F%
> > > > > 2
> > > > > > > > >Fdoc
> > > > > > > > >.dpdk
> > > > > > > > >.org%2Fguides%2Fprog_guide%2Frte_flow.html%23table-rte-flow-
> > > > > redir
> > > > > > > > >ect-
> > > > > > > > >queue-5-
> > > > > > > >
> > > > > > > >3&data=02%7C01%7Casafp%40nvidia.com%7C1a46005bec5245
> > > e72
> > > > > 9e70
> > > > > > > 8d
> > > > > > > >
> > > > > > > >85bd24caf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C6
> > > 373
> > > > > 6030
> > > > > > > 60
> > > > > > > >
> > > > > > > >73519816&sdata=EOF%2Fz62crvBZK8rwzwKIWxj5cVlfPVnU3FLm
> > > cL9
> > > > > X2w0
> > > > > > > %3
> > > > > > > > >D&reserved=0
> > > > > >
> > > >
> >
>
^ permalink raw reply [relevance 0%]
* Re: [dpdk-dev] Duplicating traffic with RTE Flow
2021-03-15 13:22 5% ` Jan Viktorin
2021-03-15 17:57 0% ` Jan Viktorin
@ 2021-03-16 4:00 0% ` Jiawei(Jonny) Wang
1 sibling, 0 replies; 15+ results
From: Jiawei(Jonny) Wang @ 2021-03-16 4:00 UTC (permalink / raw)
To: Jan Viktorin; +Cc: Slava Ovsiienko, Asaf Penso, dev, Ori Kam
Hi Jan,
> -----Original Message-----
> From: Jan Viktorin <viktorin@cesnet.cz>
> Sent: Monday, March 15, 2021 9:22 PM
> To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> Cc: Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>; dev@dpdk.org; Ori Kam <orika@nvidia.com>
> Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
>
> Hello Jiawei,
>
> On Fri, 12 Mar 2021 09:32:44 +0000
> "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
>
> > Hi Jan,
> >
> > > -----Original Message-----
> > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > Sent: Friday, March 12, 2021 12:33 AM
> > > To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > Cc: Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> > > <asafp@nvidia.com>; dev@dpdk.org; Ori Kam <orika@nvidia.com>
> > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > >
> > > On Thu, 11 Mar 2021 02:11:07 +0000
> > > "Jiawei(Jonny) Wang" <jiaweiw@nvidia.com> wrote:
> > >
> > > > Hi Jan,
> > > >
> > > > Sorry for late response,
> > > >
> > > > First rule is invalid, port only works on FDB domain so need
> > > > 'transfer' here; Second rule should be ok, could you please check
> > > > if the
> > > port 1 was enabled on you dpdk application?
> > >
> > > I assume that it is enabled, see full transcript:
> > >
> > > $ ofed_info
> > > MLNX_OFED_LINUX-5.2-1.0.4.0 (OFED-5.2-1.0.4):
> > > ...
> > > $ sudo dpdk-testpmd -v -- -i
> > > EAL: Detected 24 lcore(s)
> > > EAL: Detected 1 NUMA nodes
> > > EAL: RTE Version: 'DPDK 20.11.0'
> > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > > EAL: Selected IOVA mode 'PA'
> > > EAL: No available hugepages reported in hugepages-1048576kB
> > > EAL: Probing VFIO support...
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.0
> > > (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:04:00.1
> > > (socket 0)
> > > mlx5_pci: No available register for Sampler.
> > > mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> > > EAL: No legacy callbacks, legacy socket not created
> > > Interactive-mode selected
> > > testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176,
> > > socket=0
> > > testpmd: preferred mempool ops selected: ring_mp_mc Configuring
> > > Port 0 (socket 0) Port 0: B8:59:9F:E2:09:F6 Configuring Port 1 (socket 0)
> Port 1:
> > > B8:59:9F:E2:09:F7 Checking link statuses...
> > > Done
> >
> > Seems that you start two PF port here, Port 1 is not VF port; FDB
> > rule can steering the packet form PF to its VFs and vice versa, Could
> > you please try to open the VF ports and start the testpmd with
> representor=.
>
> I did not know this, so I tried with VFs:
>
> # echo 2 > /sys/class/net/hge1/device/sriov_numvfs
> # echo switchdev > /sys/class/net/hge1/compat/devlink/mode
>
> # dpdk-testpmd -v -a '0000:05:00.1,representor=[0-1]' -- -i
> EAL: Detected 24 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: RTE Version: 'DPDK 20.11.0'
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:05:00.1 (socket 0)
> mlx5_pci: No available register for Sampler.
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: No available register for Sampler.
> mlx5_pci: No available register for Sampler.
> EAL: No legacy callbacks, legacy socket not created Interactive-mode
> selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=331456, size=2176,
> socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
>
> Warning! port-topology=paired and odd forward ports number, the last port
> will pair with itself.
>
> Configuring Port 0 (socket 0)
> Port 0: B8:59:9F:E2:09:F7
> Configuring Port 1 (socket 0)
> Port 1: B2:57:D6:72:F3:31
> Configuring Port 2 (socket 0)
> Port 2: 9E:CB:D0:73:59:CE
> Checking link statuses...
> Done
> testpmd> show port summary all
> Number of available ports: 3
> Port MAC Address Name Driver Status Link
> 0 B8:59:9F:E2:09:F7 0000:05:00.1 mlx5_pci up 100 Gbps
> 1 B2:57:D6:72:F3:31 0000:05:00.1_representor_0 mlx5_pci up 100
> Gbps
> 2 9E:CB:D0:73:59:CE 0000:05:00.1_representor_1 mlx5_pci up 100
> Gbps
> testpmd> set sample_actions 0 port_id id 1 / end testpmd> flow validate 0
> ingress transfer pattern end actions sample ratio 1 index 0 / drop / end
> port_flow_complain(): Caught PMD error type 1 (cause unspecified): sample
> action not supported: Operation not supported
>
> Still no luck. However, there is this message 3-times in the log:
>
> mlx5_pci: No available register for Sampler.
>
> It looks like it might be related. What does it mean?
>
Yes, the error message means that can't find available sampler register, and this value is fetched from the FW;
Seems that you use the older FW, the OFED version 5.2-1.0.4 is ok, but FW isn't match with
this OFED version.
Thanks.
Jonny
> Jan
>
> >
> > Thanks.
> >
> > > testpmd> port start 1
> > > Port 1 is now not stopped
> > > Please stop the ports first
> > > Done
> > > testpmd> set sample_actions 0 port_id id 1 / end testpmd> flow
> > > validate 0 ingress transfer pattern end actions sample ratio 1 index
> > > 0 / drop / end
> > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > (no stated reason): Operation not supported testpmd> flow create 0
> > > ingress transfer pattern end actions sample ratio 1 index 0 / drop /
> > > end
> > > port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > (no stated reason): Operation not supported testpmd> Stopping port 0...
> > > Stopping ports...
> > > Done
> > >
> > > Stopping port 1...
> > > Stopping ports...
> > > Done
> > >
> > > Shutting down port 0...
> > > Closing ports...
> > > Port 0 is closed
> > > Done
> > >
> > > Shutting down port 1...
> > > Closing ports...
> > > Port 1 is closed
> > > Done
> > >
> > > Bye...
> > >
> > > Jan
> > >
> > > >
> > > > Thanks.
> > > > Jonny
> > > >
> > > > > -----Original Message-----
> > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > Sent: Monday, March 1, 2021 10:43 PM
> > > > > To: Slava Ovsiienko <viacheslavo@nvidia.com>
> > > > > Cc: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org; Ori Kam
> > > > > <orika@nvidia.com>; Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > >
> > > > > On Mon, 1 Mar 2021 14:34:07 +0000 Slava Ovsiienko
> > > > > <viacheslavo@nvidia.com> wrote:
> > > > >
> > > > > > Hi, Jan
> > > > > >
> > > > > > To use port action (I see it is in your sample action list)
> > > > > > the flow should be applied to the FDB domain, ie "transfer"
> > > > > > attribute should be
> > > > > specified:
> > > > > >
> > > > > > flow validate 0 ingress transfer...
> > > > >
> > > > > As you can see (however, it's a bit messy in the response below,
> > > > > in [1], it is better), I tried both. First without transfer and
> > > > > second with. The first gives hint "action is valid in transfer mode only"
> > > > > but the second try with transfer gives "Operation not supported".
> > > > >
> > > > > Jan
> > > > >
> > > > > [1] http://mails.dpdk.org/archives/dev/2021-March/200475.html
> > > > >
> > > > > >
> > > > > > With best regards, Slava
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Jan Viktorin <viktorin@cesnet.cz>
> > > > > > > Sent: Monday, March 1, 2021 14:21
> > > > > > > To: Asaf Penso <asafp@nvidia.com>
> > > > > > > Cc: dev@dpdk.org; Ori Kam <orika@nvidia.com>; Jiawei(Jonny)
> > > > > > > Wang <jiaweiw@nvidia.com>; Slava Ovsiienko
> > > > > > > <viacheslavo@nvidia.com>
> > > > > > > Subject: Re: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > >
> > > > > > > Hello Asaf,
> > > > > > >
> > > > > > > it is a while we were in touch regarding this topic.
> > > > > > > Finally, I am again trying to get work this feature. I've
> > > > > > > seen that sampling is already upstreamed which is great.
> > > > > > > However, I am not very successful with that. There is nearly
> > > > > > > no documentation, just [1], I found no examples,
> > > > > just commit logs...
> > > > > > >
> > > > > > > I tried:
> > > > > > >
> > > > > > > > set sample_actions 0 port_id id 1 / end > flow validate
> > > > > > > 0 ingress pattern end actions sample ratio 1 index 0 / drop
> > > > > > > / end
> > > > > > > port_flow_complain(): Caught PMD error type 1 (cause
> unspecified):
> > > > > > > port id action is valid in transfer mode only: Operation not
> > > > > > > supported > flow validate
> > > > > > > 0 ingress transfer pattern end actions sample ratio 1 index
> > > > > > > 0 / drop / end
> > > > > > > port_flow_complain(): Caught PMD error type 1 (cause
> unspecified):
> > > > > > > (no stated reason): Operation not supported
> > > > > > >
> > > > > > > Using CentOS 7, DPDK 20.11.0, OFED-5.2-1.0.4.
> > > > > > > NICs: MT2892 Family [ConnectX-6 Dx] 101d (fw 22.28.1002),
> > > > > > > MT27800 Family [ConnectX-5] 1017 (fw 16.27.2008).
> > > > > > >
> > > > > > > My primary goal is to be able to deliver exactly the same
> > > > > > > packets both to DPDK and to the Linux kernel. Doing this at
> > > > > > > RTE Flow level would be great due to performance and
> transparency.
> > > > > > >
> > > > > > > Jan
> > > > > > >
> > > > > > > [1]
> > > > > > > https://doc.dpdk.org/guides/prog_guide/rte_flow.html#action-
> > > > > > > samp
> > > > > > > le
> > > > > > >
> > > > > > > On Fri, 18 Sep 2020 14:23:42 +0000 Asaf Penso
> > > > > > > <asafp@nvidia.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Jan,
> > > > > > > >
> > > > > > > > You can have a look in series [1] where we propose to add
> > > > > > > > APIs to
> > > > > > > DPDK20.11 for both mirroring and sampling for packets, with
> > > > > > > additional actions of the different traffic.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=12045
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Asaf Penso
> > > > > > > >
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: dev <dev-bounces@dpdk.org> On Behalf Of Jan
> > > > > > > > >Viktorin
> > > > > > > > >Sent: Friday, September 18, 2020 3:56 PM
> > > > > > > > >To: dev@dpdk.org
> > > > > > > > >Subject: [dpdk-dev] Duplicating traffic with RTE Flow
> > > > > > > > >
> > > > > > > > >Hello all,
> > > > > > > > >
> > > > > > > > >we are looking for a way to duplicate ingress traffic in hardware.
> > > > > > > > >
> > > > > > > > >There is an example in [1] suggesting to insert two fate
> > > > > > > > >actions into the RTE Flow actions array like:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions queue index 0 / void / queue index 1 / end
> > > > > > > > >
> > > > > > > > >But our experience is that PMDs reject two fate actions
> > > > > > > > >(tried with mlx5). Another similar approach would be to
> > > > > > > > >deliver every single packet into two virtual
> > > > > > > > >functions:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions vf index 0 / vf index 1 / end
> > > > > > > > >
> > > > > > > > >Third possibility was to use passthru:
> > > > > > > > >
> > > > > > > > > flow create 0 ingress pattern end \
> > > > > > > > > actions passthru / vf index 0 / end flow create 0
> > > > > > > > > ingress pattern end \
> > > > > > > > > actions vf index 1 / end
> > > > > > > > >
> > > > > > > > >Again, tried on mlx5 and it does not support the passthru.
> > > > > > > > >
> > > > > > > > >Last idea was to use isolate with passthru (to deliver
> > > > > > > > >both to DPDK application and to the kernel) but again
> > > > > > > > >there was no support on mlx5 for
> > > > > > > passthru...
> > > > > > > > >
> > > > > > > > > flow isolate 0 true
> > > > > > > > > flow create 0 ingress pattern end actions passthru /
> > > > > > > > > rss end / end
> > > > > > > > >
> > > > > > > > >Is there any other possibility or PMD+NIC that is known
> > > > > > > > >to solve such
> > > > > > > issue?
> > > > > > > > >
> > > > > > > > >Thanks
> > > > > > > > >Jan Viktorin
> > > > > > > > >
> > > > > > > > >[1]
> > > > > > > > >https://nam11.safelinks.protection.outlook.com/?url=https
> > > > > > > > >%3A%
> > > > > > > > >2F%
> > > > > 2
> > > > > > > > >Fdoc
> > > > > > > > >.dpdk
> > > > > > > > >.org%2Fguides%2Fprog_guide%2Frte_flow.html%23table-rte-
> fl
> > > > > > > > >ow-
> > > > > redir
> > > > > > > > >ect-
> > > > > > > > >queue-5-
> > > > > > > >
> > > > > > > >3&data=02%7C01%7Casafp%40nvidia.com%7C1a46005bec5
> 245
> > > e72
> > > > > 9e70
> > > > > > > 8d
> > > > > > > >
> > > > > > > >85bd24caf%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7
> C6
> > > 373
> > > > > 6030
> > > > > > > 60
> > > > > > > >
> > > > > > > >73519816&sdata=EOF%2Fz62crvBZK8rwzwKIWxj5cVlfPVnU3
> FLm
> > > cL9
> > > > > X2w0
> > > > > > > %3
> > > > > > > > >D&reserved=0
> > > > > >
> > > >
> >
^ permalink raw reply [relevance 0%]
* [PATCH] common/mlx5: decrease log level for hlist creation
@ 2021-11-16 20:33 5% David Marchand
2021-11-17 12:14 0% ` Slava Ovsiienko
0 siblings, 1 reply; 15+ results
From: David Marchand @ 2021-11-16 20:33 UTC (permalink / raw)
To: dev; +Cc: thomas, ferruh.yigit, Matan Azrad, Viacheslav Ovsiienko, Suanming Mou
Initialising mlx5 devices in OVS, I get the following logs:
2021-11-16T20:08:37Z|00021|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci
(15b3:101d) device: 0000:3b:00.0 (socket 0)
2021-11-16T20:08:37Z|00022|dpdk|INFO|common_mlx5: RTE_MEM is selected.
2021-11-16T20:08:38Z|00023|dpdk|WARN|mlx5_pci: Size 0xFFFF is not power
of 2, will be aligned to 0x10000.
Those logs just make no sense for a final user and were raised to WARN
level recently.
Lower them back to DEBUG.
Fixes: 961b6774c451 ("common/mlx5: add per-lcore cache to hash list utility")
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
drivers/common/mlx5/mlx5_common_utils.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/mlx5_common_utils.c b/drivers/common/mlx5/mlx5_common_utils.c
index 775fabd478..c83333b4b4 100644
--- a/drivers/common/mlx5/mlx5_common_utils.c
+++ b/drivers/common/mlx5/mlx5_common_utils.c
@@ -406,7 +406,7 @@ mlx5_hlist_create(const char *name, uint32_t size, bool direct_key,
/* Align to the next power of 2, 32bits integer is enough now. */
if (!rte_is_power_of_2(size)) {
act_size = rte_align32pow2(size);
- DRV_LOG(WARNING, "Size 0x%" PRIX32 " is not power of 2, will "
+ DRV_LOG(DEBUG, "Size 0x%" PRIX32 " is not power of 2, will "
"be aligned to 0x%" PRIX32 ".", size, act_size);
} else {
act_size = size;
--
2.23.0
^ permalink raw reply [relevance 5%]
* RE: [PATCH] common/mlx5: decrease log level for hlist creation
2021-11-16 20:33 5% [PATCH] common/mlx5: decrease log level for hlist creation David Marchand
@ 2021-11-17 12:14 0% ` Slava Ovsiienko
2021-11-17 13:02 0% ` David Marchand
0 siblings, 1 reply; 15+ results
From: Slava Ovsiienko @ 2021-11-17 12:14 UTC (permalink / raw)
To: David Marchand, dev
Cc: NBU-Contact-Thomas Monjalon, ferruh.yigit, Matan Azrad, Suanming Mou
Hi, David
In general, I'm OK with lowering the log level.
But we do not expect the warning " is not power of 2".
What mlx5_hlist_create() call caused this?
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, November 16, 2021 22:34
> To: dev@dpdk.org
> Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> ferruh.yigit@intel.com; Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Suanming Mou <suanmingm@nvidia.com>
> Subject: [PATCH] common/mlx5: decrease log level for hlist creation
>
> Initialising mlx5 devices in OVS, I get the following logs:
> 2021-11-16T20:08:37Z|00021|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci
> (15b3:101d) device: 0000:3b:00.0 (socket 0)
> 2021-11-16T20:08:37Z|00022|dpdk|INFO|common_mlx5: RTE_MEM is
> selected.
> 2021-11-16T20:08:38Z|00023|dpdk|WARN|mlx5_pci: Size 0xFFFF is not
> power of 2, will be aligned to 0x10000.
Can we remove not too meaningful details from commit message?
Date/time in messages: 2021-11-16T20:08:38Z|00023| ?
With best regards,
Slava
>
> Those logs just make no sense for a final user and were raised to WARN level
> recently.
> Lower them back to DEBUG.
>
> Fixes: 961b6774c451 ("common/mlx5: add per-lcore cache to hash list
> utility")
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> drivers/common/mlx5/mlx5_common_utils.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/common/mlx5/mlx5_common_utils.c
> b/drivers/common/mlx5/mlx5_common_utils.c
> index 775fabd478..c83333b4b4 100644
> --- a/drivers/common/mlx5/mlx5_common_utils.c
> +++ b/drivers/common/mlx5/mlx5_common_utils.c
> @@ -406,7 +406,7 @@ mlx5_hlist_create(const char *name, uint32_t size,
> bool direct_key,
> /* Align to the next power of 2, 32bits integer is enough now. */
> if (!rte_is_power_of_2(size)) {
> act_size = rte_align32pow2(size);
> - DRV_LOG(WARNING, "Size 0x%" PRIX32 " is not power of 2,
> will "
> + DRV_LOG(DEBUG, "Size 0x%" PRIX32 " is not power of 2, will
> "
> "be aligned to 0x%" PRIX32 ".", size, act_size);
> } else {
> act_size = size;
> --
> 2.23.0
^ permalink raw reply [relevance 0%]
* Re: [PATCH] common/mlx5: decrease log level for hlist creation
2021-11-17 12:14 0% ` Slava Ovsiienko
@ 2021-11-17 13:02 0% ` David Marchand
2021-11-17 13:28 0% ` Slava Ovsiienko
0 siblings, 1 reply; 15+ results
From: David Marchand @ 2021-11-17 13:02 UTC (permalink / raw)
To: Slava Ovsiienko
Cc: dev, NBU-Contact-Thomas Monjalon, ferruh.yigit, Matan Azrad,
Suanming Mou, Maxime Coquelin
On Wed, Nov 17, 2021 at 1:14 PM Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
>
> Hi, David
>
> In general, I'm OK with lowering the log level.
> But we do not expect the warning " is not power of 2".
> What mlx5_hlist_create() call caused this?
That's something Maxime can see on RHEL8 with CX6 devices.
I remember seeing it too in the (recent) past, probably when working
with our QE.
So far, I did not associate this log presence to functional issues.
Looking at DPDK mailing lists:
http://inbox.dpdk.org/dev/?q=%220xFFFF+is+not+power+of+2%22
http://inbox.dpdk.org/users/?q=%220xFFFF+is+not+power+of+2%22
So we are not the only ones seeing this.
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Tuesday, November 16, 2021 22:34
> > To: dev@dpdk.org
> > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> > ferruh.yigit@intel.com; Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> > <viacheslavo@nvidia.com>; Suanming Mou <suanmingm@nvidia.com>
> > Subject: [PATCH] common/mlx5: decrease log level for hlist creation
> >
> > Initialising mlx5 devices in OVS, I get the following logs:
> > 2021-11-16T20:08:37Z|00021|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci
> > (15b3:101d) device: 0000:3b:00.0 (socket 0)
> > 2021-11-16T20:08:37Z|00022|dpdk|INFO|common_mlx5: RTE_MEM is
> > selected.
> > 2021-11-16T20:08:38Z|00023|dpdk|WARN|mlx5_pci: Size 0xFFFF is not
> > power of 2, will be aligned to 0x10000.
>
> Can we remove not too meaningful details from commit message?
> Date/time in messages: 2021-11-16T20:08:38Z|00023| ?
I don't mind.
Logs were coming from OVS unit tests but the same can most likely be
reproduced with testpmd.
--
David Marchand
^ permalink raw reply [relevance 0%]
* RE: [PATCH] common/mlx5: decrease log level for hlist creation
2021-11-17 13:02 0% ` David Marchand
@ 2021-11-17 13:28 0% ` Slava Ovsiienko
0 siblings, 1 reply; 15+ results
From: Slava Ovsiienko @ 2021-11-17 13:28 UTC (permalink / raw)
To: David Marchand
Cc: dev, NBU-Contact-Thomas Monjalon, ferruh.yigit, Matan Azrad,
Suanming Mou, Maxime Coquelin
Hi, David
I've re-checked the mlx5_hlist_create() call tree.
And it seems all the calls are done with hardcoded const values for "size" argument,
and all these values are powers-of-2.
We had an issue in the past, but then I was not seeing this warning for a long time
on my setup.
With best regards,
Slava
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, November 17, 2021 15:02
> To: Slava Ovsiienko <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> ferruh.yigit@intel.com; Matan Azrad <matan@nvidia.com>; Suanming Mou
> <suanmingm@nvidia.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>
> Subject: Re: [PATCH] common/mlx5: decrease log level for hlist creation
>
> On Wed, Nov 17, 2021 at 1:14 PM Slava Ovsiienko <viacheslavo@nvidia.com>
> wrote:
> >
> > Hi, David
> >
> > In general, I'm OK with lowering the log level.
> > But we do not expect the warning " is not power of 2".
> > What mlx5_hlist_create() call caused this?
>
> That's something Maxime can see on RHEL8 with CX6 devices.
> I remember seeing it too in the (recent) past, probably when working with our
> QE.
> So far, I did not associate this log presence to functional issues.
>
>
> Looking at DPDK mailing lists:
> http://inbox.dpdk.org/dev/?q=%220xFFFF+is+not+power+of+2%22
> http://inbox.dpdk.org/users/?q=%220xFFFF+is+not+power+of+2%22
>
> So we are not the only ones seeing this.
>
> >
> > > -----Original Message-----
> > > From: David Marchand <david.marchand@redhat.com>
> > > Sent: Tuesday, November 16, 2021 22:34
> > > To: dev@dpdk.org
> > > Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> > > ferruh.yigit@intel.com; Matan Azrad <matan@nvidia.com>; Slava
> > > Ovsiienko <viacheslavo@nvidia.com>; Suanming Mou
> > > <suanmingm@nvidia.com>
> > > Subject: [PATCH] common/mlx5: decrease log level for hlist creation
> > >
> > > Initialising mlx5 devices in OVS, I get the following logs:
> > > 2021-11-16T20:08:37Z|00021|dpdk|INFO|EAL: Probe PCI driver:
> mlx5_pci
> > > (15b3:101d) device: 0000:3b:00.0 (socket 0)
> > > 2021-11-16T20:08:37Z|00022|dpdk|INFO|common_mlx5: RTE_MEM is
> > > selected.
> > > 2021-11-16T20:08:38Z|00023|dpdk|WARN|mlx5_pci: Size 0xFFFF is not
> > > power of 2, will be aligned to 0x10000.
> >
> > Can we remove not too meaningful details from commit message?
> > Date/time in messages: 2021-11-16T20:08:38Z|00023| ?
>
> I don't mind.
> Logs were coming from OVS unit tests but the same can most likely be
> reproduced with testpmd.
>
>
> --
> David Marchand
^ permalink raw reply [relevance 0%]
* Re: [PATCH] common/mlx5: decrease log level for hlist creation
@ 2021-11-18 14:22 5% ` David Marchand
2021-11-19 8:36 0% ` Slava Ovsiienko
0 siblings, 1 reply; 15+ results
From: David Marchand @ 2021-11-18 14:22 UTC (permalink / raw)
To: Slava Ovsiienko
Cc: dev, NBU-Contact-Thomas Monjalon, ferruh.yigit, Matan Azrad,
Suanming Mou, Maxime Coquelin
Hi Slava,
On Wed, Nov 17, 2021 at 3:46 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Wed, Nov 17, 2021 at 2:28 PM Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
> >
> > I've re-checked the mlx5_hlist_create() call tree.
> >
> > And it seems all the calls are done with hardcoded const values for "size" argument,
> > and all these values are powers-of-2.
> >
> > We had an issue in the past, but then I was not seeing this warning for a long time
> > on my setup.
>
> I'll double check with Maxime.
> There might be a misunderstanding between us.
Maxime passed me his setup with a CX6.
I confirm there is no warning in main and the problem has been fixed
in v20.11 LTS.
Sorry for the noise, I'll withdraw this patch.
Thanks.
For the record:
- v20.11 and v20.11.1 has logs about:
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
- v20.11.2 has only:
mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
- v20.11.3 has no warning
--
David Marchand
^ permalink raw reply [relevance 5%]
* RE: [PATCH] common/mlx5: decrease log level for hlist creation
2021-11-18 14:22 5% ` David Marchand
@ 2021-11-19 8:36 0% ` Slava Ovsiienko
0 siblings, 0 replies; 15+ results
From: Slava Ovsiienko @ 2021-11-19 8:36 UTC (permalink / raw)
To: David Marchand
Cc: dev, NBU-Contact-Thomas Monjalon, ferruh.yigit, Matan Azrad,
Suanming Mou, Maxime Coquelin
Hi, David
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Thursday, November 18, 2021 16:23
> To: Slava Ovsiienko <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> ferruh.yigit@intel.com; Matan Azrad <matan@nvidia.com>; Suanming Mou
> <suanmingm@nvidia.com>; Maxime Coquelin
> <maxime.coquelin@redhat.com>
> Subject: Re: [PATCH] common/mlx5: decrease log level for hlist creation
>
> Hi Slava,
>
> On Wed, Nov 17, 2021 at 3:46 PM David Marchand
> <david.marchand@redhat.com> wrote:
> >
> > On Wed, Nov 17, 2021 at 2:28 PM Slava Ovsiienko
> <viacheslavo@nvidia.com> wrote:
> > >
> > > I've re-checked the mlx5_hlist_create() call tree.
> > >
> > > And it seems all the calls are done with hardcoded const values for
> > > "size" argument, and all these values are powers-of-2.
> > >
> > > We had an issue in the past, but then I was not seeing this warning
> > > for a long time on my setup.
> >
> > I'll double check with Maxime.
> > There might be a misunderstanding between us.
>
> Maxime passed me his setup with a CX6.
> I confirm there is no warning in main and the problem has been fixed in
> v20.11 LTS.
> Sorry for the noise, I'll withdraw this patch.
>
> Thanks.
>
>
> For the record:
> - v20.11 and v20.11.1 has logs about:
> mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
>
> - v20.11.2 has only:
> mlx5_pci: Failed to init cache list FDB_ingress_0_matcher_cache entry (nil).
>
> - v20.11.3 has no warning
Thanks a lot for checking and clear confirmation.
With best regards,
Slava
^ permalink raw reply [relevance 0%]
Results 1-15 of 15 | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2020-09-18 12:56 [dpdk-dev] Duplicating traffic with RTE Flow Jan Viktorin
2020-09-18 14:23 ` Asaf Penso
2021-03-01 12:21 ` Jan Viktorin
2021-03-01 14:34 ` Slava Ovsiienko
2021-03-01 14:43 ` Jan Viktorin
2021-03-11 2:11 ` Jiawei(Jonny) Wang
2021-03-11 16:32 7% ` Jan Viktorin
2021-03-12 9:32 0% ` Jiawei(Jonny) Wang
2021-03-15 13:22 5% ` Jan Viktorin
2021-03-15 17:57 0% ` Jan Viktorin
2021-03-16 4:00 0% ` Jiawei(Jonny) Wang
2020-10-12 8:11 [dpdk-dev] [PATCH 0/2] fix issue with partial DMA unmap Nithin Dabilpuram
2020-12-02 5:46 ` [dpdk-dev] [PATCH v4 0/4] " Nithin Dabilpuram
2020-12-02 5:46 ` [dpdk-dev] [PATCH v4 3/4] test: add test case to validate VFIO DMA map/unmap Nithin Dabilpuram
2020-12-02 19:23 6% ` David Christensen
2020-12-03 7:14 0% ` Nithin Dabilpuram
2020-12-14 8:24 0% ` Nithin Dabilpuram
2020-11-19 17:45 6% [dpdk-dev] [Bug 580] [dpdk-20.11] testpmd: Ubuntu 16.04: Failed to start port as mlx5_pci can't allocate hash list bugzilla
2021-11-16 20:33 5% [PATCH] common/mlx5: decrease log level for hlist creation David Marchand
2021-11-17 12:14 0% ` Slava Ovsiienko
2021-11-17 13:02 0% ` David Marchand
2021-11-17 13:28 0% ` Slava Ovsiienko
2021-11-17 14:46 ` David Marchand
2021-11-18 14:22 5% ` David Marchand
2021-11-19 8:36 0% ` Slava Ovsiienko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).