Re: [dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD

[Aziz Hajee <aziz@saisei.com>]

Thanks Rashmin,
I am close, just getting this solib loading undefined symbol error: (I mod
code to see the error). Using dpdk-1.6.0r1_ss

EAL: Setting up memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fc0a8200000 (size = 0x200000)
EAL: Ask a virtual area of 0x5c00000 bytes
EAL: Virtual area found at 0x7fc07e200000 (size = 0x5c00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fc0a1c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fc0a1800000 (size = 0x200000)
EAL: Ask a virtual area of 0x19c00000 bytes
EAL: Virtual area found at 0x7fc064400000 (size = 0x19c00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fc0a1400000 (size = 0x200000)
EAL: Requesting 256 pages of size 2MB from socket 0
EAL: TSC frequency is ~2300000 KHz
Cannot load solib librte_pmd_vmxnet3.so DLERROR:./librte_pmd_vmxnet3.so:
undefined symbol: per_lcore__lcore_id
EAL: (null)

coming from eal.c /* Launch threads, called at application init(). */
int
rte_eal_init(int argc, char **argv)

       TAILQ_FOREACH(solib, &solib_list, next) {
                solib->lib_handle = dlopen(solib->name, RTLD_NOW);
                if ((solib->lib_handle == NULL) && (solib->name[0] != '/'))
{
                        /* relative path: try again with "./" prefix */
                        char sopath[PATH_MAX];
                        snprintf(sopath, sizeof(sopath), "./%s",
solib->name);
                        solib->lib_handle = dlopen(sopath, RTLD_NOW);
                }
                if (solib->lib_handle == NULL)
                        printf("Cannot load solib %s DLERROR:%s\n",
solib->name, dlerror());
//                      RTE_LOG(WARNING, EAL, "%s\n", dlerror());

        }


On Tue, Nov 11, 2014 at 9:19 AM, Patel, Rashmin N <rashmin.n.patel@intel.com
> wrote:

>  Please find comments in-lined.
>
>
>
> Thanks,
>
> RP
>
>
>
> *From:* Aziz Hajee [mailto:aziz@saisei.com]
> *Sent:* Monday, November 10, 2014 6:00 PM
> *To:* Patel, Rashmin N
> *Cc:* dev@dpdk.org
> *Subject:* Re: [dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD
>
>
>
> Rashmin,
> Since I do need the jumbo, I use the vmxnet3-plugin you described, i.e.
> (1)
> sudo insmod ./vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=2048
> num_txds=2048
>
> and (2) when running the application, use in the args list:
>
> "-d", "librte_pmd_vmxnet3.so"
>
> Does the above two piece mean vmxnet3-plugin
>
> [RP] that’s correct
>
> I do see my vmxnet3 device from the dump,    rte_eal_pci_dump();
> but the 'nb_ports' in DPDK never gets incremented rte_eth_dev_count()
> returns zero.
> so all the other api fails, if (port_id >= nb_ports) {
>                 PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>                 return;
>         }
>
>   0000:03:00.0 - vendor:15ad device:7b0
>    00000000d2404000 0000000000001000
>    00000000d2403000 0000000000001000
>    00000000d2400000 0000000000002000
>    0000000000000000 0000000000000000
>    0000000000000000 0000000000000000
>    0000000000000000 0000000000000000
>    00000000d4400000 0000000000010000
> 0000:0b:00.0 - vendor:15ad device:7b0
>    00000000d2504000 0000000000001000
>    00000000d2503000 0000000000001000
>    00000000d2500000 0000000000002000
>    0000000000000000 0000000000000000
>    0000000000000000 0000000000000000
>    0000000000000000 0000000000000000
>    00000000d4500000 0000000000010000
> DPDK: No Ethernet ports (rte_eth_dev_count() returns zero)
> PMD: rte_eth_dev_info_get: Invalid port_id=0
> PMD: rte_eth_dev_configure: Invalid port_id=0
> PMD: rte_eth_dev_info_get: Invalid port_id=0
> PMD: rte_eth_dev_configure: Invalid port_id=0
>
> So when using not using DPDK PMD for VMXNET3, what am i missing, for the
> the DPDK to know the nb_ports,
>
> How will the rte_eth_dev_start(portid) in DPDK library know  ?
>
> rte_pmd_init_all() will not have the init the Intel DPDK PMD ,
> RTE_LIBRTE_VMXNET3_PMD = n in config.
>
> [RP] If you’re using the vmxnet3-plugin, you should keep
> RTE_LIBRTE_VMXNET3_PMD = n in config, and link the shared library with its
> headers, it should work. I have tried it once a long back.
>
>
> #ifdef RTE_LIBRTE_VMXNET3_PMD
>         if ((ret = rte_vmxnet3_pmd_init()) != 0) {
>                 RTE_LOG(ERR, PMD, "Cannot init vmxnet3 PMD\n");
>                 return (ret);
>         }
>
> If I make RTE_LIBRTE_VMXNET3_PMD = y, then I am using the Intel DPDK PMD
> and no jumbo.
>
> [RP] Yes, I understood that part. We need to support jumbo frames in
> in-tree version of VMXNET3-PMD, we’ll merge all soon, we can discuss in the
> community conf. call so please do attend the next one on Nov 18 and we can
> raise concerns there
>
>
>
> Thanks,
>
> aziz
>
>
>
> On Fri, Nov 7, 2014 at 8:53 AM, Patel, Rashmin N <
> rashmin.n.patel@intel.com> wrote:
>
> Hi Aziz,
>
> Yes, you're right DPDK VMXNET3-PMD in /lib/librte_pmd_vmxnet3 does not
> support mbuf chaining today. But it's a standalone bsd driver just like any
> other pmd in that directory, it does not need vmxnet3-usermap.ko module.
>
> Now there is another vmxnet3 solution in a separate branch as a plugin,
> which must have vmxnet3-usermap.ko linux module(1), and a user space
> interface piece(2) to tie it to any DPDK application in the main branch.
> (1) and (2) makes the solution which is known as vmxnet3-plugin. It's been
> there for a long time just like virtio-plugin, I don't know who uses it,
> but community can *reply* here if there is still any need of a separate
> solution that way.
>
> I'm in favor of consolidating all those version into one elegant solution
> by grabbing best features from all of them and maintain one copy. I'm sure
> that developers contributing from VMware would also support that idea
> because then it makes easy to maintain and debug and bug fix and most
> importantly avoid such confusion in future.
>
> Thanks,
> Rashmin
>
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aziz Hajee
> Sent: Thursday, November 06, 2014 5:47 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD
>
> I am using the dpdk1.6.0r1
> I could not find a complete clarification, sorry if missed.
> VMXNET3 PMD
> ============
> I have enabled the VMXNET3 PMD  in the dpdk.
>  # Compile burst-oriented VMXNET3 PMD driver  #
>
> CONFIG_RTE_LIBRTE_VMXNET3_PMD=y
> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_RX=n
> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX=n
> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n
> The Intel DPDK VMXNET3 PMD driver does not support mbuf chaining, and I
> have to set NOMULTSEGS for the vmxnet3 interface init to succeed.
> tx_conf.txq_flags =  ETH_TXQ_FLAGS_NOMULTSEGS Is there a later version of
> DPDK that supports multiseg for the dpdk
> VMXNET3 PMD.
>
> vmware vmxnet3-usermap AND  DPDK VMXNET3 PMD
> =========================================
> Is the vmxnet3-usermap.ko module driver also needed ? (appears that I
> need, otherwise the eal initialise fails.
> sudo insmod ./vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=2048
> num_txds=2048
>
> I do not understand if VMXNET3 PMD is there, what is the purpose of
> /vmxnet3-usermap.ko/vmxnet3-usermap.ko ?
>
> From some responses i saw that the following ifdef RTE_EAL_UNBIND_PORTS is
> also need to be removed in lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c ?
>      {
>          .name = "rte_vmxnet3_pmd",
>          .id_table = pci_id_vmxnet3_map, -#ifdef RTE_EAL_UNBIND_PORTS
> +// #ifdef RTE_EAL_UNBIND_PORTS
>          .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO, -#endif
> +// #endif
>      },
>      .eth_dev_init = eth_vmxnet3_dev_init,
>      .dev_private_size = sizeof(struct vmxnet3_adapter),
>
> thanks,
> -aziz
>
>
>