Re: [dpdk-dev] [PATCH v2 46/46] doc: add doc for liquidio

[Shijith Thotton <shijith.thotton@caviumnetworks.com>]

On Tue, Mar 21, 2017 at 12:33:30PM +0000, Ferruh Yigit wrote:
> On 3/2/2017 11:32 AM, Shijith Thotton wrote:
> > Added doc/guides/nics/liquidio.rst and
> > doc/guides/nics/features/liquidio.ini. Updated release notes.
> > 
> > Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > Signed-off-by: Derek Chickles <derek.chickles@caviumnetworks.com>
> > Signed-off-by: Venkat Koppula <venkat.koppula@caviumnetworks.com>
> > Signed-off-by: Srisivasubramanian S <ssrinivasan@caviumnetworks.com>
> > Signed-off-by: Mallesham Jatharakonda <mjatharakonda@oneconvergence.com>
> 
> <...>
> 
> > --- /dev/null
> > +++ b/doc/guides/nics/features/liquidio.ini
> > @@ -0,0 +1,29 @@
> > +;
> > +; Supported features of the 'LiquidIO' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Link status          = Y
> > +Link status event    = Y
> > +MTU update           = Y
> > +Jumbo frame          = Y
> > +Scattered Rx         = Y
> > +Allmulticast mode    = Y
> > +RSS hash             = Y
> > +RSS key update       = Y
> > +RSS reta update      = Y
> > +SR-IOV               = Y
> 
> This has been discussed before, but I am not still clear what does
> setting this feature in VF driver means. What is the intention here?
> 

Will remove.

> > +VLAN filter          = Y
> > +CRC offload          = Y
> > +VLAN offload         = P
> > +L3 checksum offload  = Y
> > +L4 checksum offload  = Y
> > +Inner L3 checksum    = Y
> > +Inner L4 checksum    = Y
> > +Basic stats          = Y
> > +Extended stats       = Y
> > +Linux UIO            = Y
> > +Linux VFIO           = Y
> > +x86-64               = Y
> 
> Is other platforms not supported?
> If that is the case, PMD should not be enabled by default.
> 

Verified build on x86-32, x86-64, ARM and for BSD targets. Hope that
qualifies to be enabled by default.

> > +Usage doc            = Y
> 
> Instead of features file in one patch, can you please split some into
> other patches?
> Each patch that implements the feature can update the relevant item in
> the doc?

Will do.

> 
> <...>
> 
> > +.. _lio_driver-compilation:
> 
> Thank you for the documentation but below part seems generic to all
> PMDs, and already documented in other PMDs.
> I am not sure about this part, what do you think only keeping LiquidIO
> related part? And perhaps we can prepare a common document that you can
> reference here.
> 

I can submit a patchset aimed at this later, as this need to touch
documentation of all PMDs with common part.

> > +
> > +Driver Compilation
> > +------------------
> > +
> > +To compile LiquidIO PMD for Linux x86_64 gcc target, run the following "make"
> > +command:
> > +
> > +.. code-block:: console
> > +
> > +   cd <DPDK-source-directory>
> > +   make install T=x86_64-native-linuxapp-gcc
> > +
> > +
> > +Sample Application Notes
> > +------------------------
> > +
> > +This section demonstrates how to launch ``testpmd`` with LiquidIO® CN23XX
> > +device managed by ``librte_pmd_lio`` in Linux operating system.
> > +
> > +#. Mount huge pages:
> > +
> > +   .. code-block:: console
> > +
> > +      mkdir /mnt/huge
> > +      mount -t hugetlbfs nodev /mnt/huge
> > +
> > +#. Request huge pages:
> > +
> > +   .. code-block:: console
> > +
> > +      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
> > +
> > +#. Load ``vfio-pci`` driver:
> > +
> > +   .. code-block:: console
> > +
> > +      modprobe vfio-pci
> > +
> > +#. Bind the LiquidIO VFs to ``vfio-pci`` loaded in previous step:
> > +
> > +   Setup VFIO permissions for regular users and then bind to ``vfio-pci``:
> > +
> > +   .. code-block:: console
> > +
> > +      sudo chmod a+x /dev/vfio
> > +
> > +      sudo chmod 0666 /dev/vfio/*
> > +
> > +      ./usertools/dpdk-devbind.py --bind vfio-pci 0000:03:00.3 0000:03:08.3
> > +
> > +#. Start ``testpmd`` with basic parameters:
> > +
> > +   .. code-block:: console
> > +
> > +      ./build/app/testpmd -c 0xf -n 4 -- -i
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      [...]
> > +      EAL: PCI device 0000:03:00.3 on NUMA socket 0
> > +      EAL:   probe driver: 177d:9712 net_liovf
> > +      EAL:   using IOMMU type 1 (Type 1)
> > +      PMD: net_liovf[03:00.3]INFO: DEVICE : CN23XX VF
> > +      EAL: PCI device 0000:03:08.3 on NUMA socket 0
> > +      EAL:   probe driver: 177d:9712 net_liovf
> > +      PMD: net_liovf[03:08.3]INFO: DEVICE : CN23XX VF
> > +      Interactive-mode selected
> > +      USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
> > +      Configuring Port 0 (socket 0)
> > +      PMD: net_liovf[03:00.3]INFO: Starting port 0
> > +      Port 0: F2:A8:1B:5E:B4:66
> > +      Configuring Port 1 (socket 0)
> > +      PMD: net_liovf[03:08.3]INFO: Starting port 1
> > +      Port 1: 32:76:CC:EE:56:D7
> > +      Checking link statuses...
> > +      Port 0 Link Up - speed 10000 Mbps - full-duplex
> > +      Port 1 Link Up - speed 10000 Mbps - full-duplex
> > +      Done
> > +      testpmd>
> > +
> > +
> > +SR-IOV: Prerequisites and Sample Application Notes
> > +--------------------------------------------------
> > +
> > +This section provides instructions to configure SR-IOV with Linux OS.
> > +
> > +#. Verify SR-IOV and ARI capabilities are enabled on the adapter using ``lspci``:
> > +
> > +   .. code-block:: console
> > +
> > +      lspci -s <slot> -vvv
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      [...]
> > +      Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
> > +      [...]
> > +      Capabilities: [178 v1] Single Root I/O Virtualization (SR-IOV)
> > +      [...]
> > +      Kernel driver in use: LiquidIO
> > +
> > +#. Load the kernel module:
> > +
> > +   .. code-block:: console
> > +
> > +      modprobe liquidio
> > +
> > +#. Bring up the PF ports:
> > +
> > +   .. code-block:: console
> > +
> > +      ifconfig p4p1 up
> > +      ifconfig p4p2 up
> > +
> > +#. Change PF MTU if required:
> > +
> > +   .. code-block:: console
> > +
> > +      ifconfig p4p1 mtu 9000
> > +      ifconfig p4p2 mtu 9000
> > +
> > +#. Create VF device(s):
> > +
> > +   Echo number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
> > +   of the parent PF.
> > +
> > +   .. code-block:: console
> > +
> > +      echo 1 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs
> > +      echo 1 > /sys/bus/pci/devices/0000:03:00.1/sriov_numvfs
> > +
> > +
> > +#. Assign VF MAC address:
> > +
> > +   Assign MAC address to the VF using iproute2 utility. The syntax is::
> > +
> > +      ip link set <PF iface> vf <VF id> mac <macaddr>
> > +
> > +   Example output:
> > +
> > +   .. code-block:: console
> > +
> > +      ip link set p4p1 vf 0 mac F2:A8:1B:5E:B4:66
> > +
> > +
> > +#. Assign VF(s) to VM.
> > +
> > +   The VF devices may be passed through to the guest VM using qemu or
> > +   virt-manager or virsh etc.
> > +
> > +   Example qemu guest launch command:
> > +
> > +   .. code-block:: console
> > +
> > +      ./qemu-system-x86_64 -name lio-vm -machine accel=kvm \
> > +      -cpu host -m 4096 -smp 4 \
> > +      -drive file=<disk_file>,if=none,id=disk1,format=<type> \
> > +      -device virtio-blk-pci,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
> > +      -device vfio-pci,host=03:00.3 -device vfio-pci,host=03:08.3
> > +
> > +
> > +#. Running testpmd
> > +
> > +   Refer :ref:`notes above <lio_driver-compilation>`
> > +   to compile and run ``testpmd`` application.
> > +   Use ``igb_uio`` instead of ``vfio-pci`` in VM.
> > +
> > +
> > +Limitations
> > +-----------
> > +
> > +VF MTU
> > +~~~~~~
> > +
> > +VF MTU is limited by PF MTU. Raise PF value before configuring VF for larger packet size.
> > +
> > +VLAN offload
> > +~~~~~~~~~~~~
> > +
> > +Tx VLAN insertion is not supported and consequently VLAN offload feature is
> > +marked partial.
> > +
> > +Ring size
> > +~~~~~~~~~
> > +
> > +Number of descriptors for Rx/Tx ring should be in the range 128 to 512.
> > +
> > +CRC striping
> > +~~~~~~~~~~~~
> > +
> 
> <...>