[dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub

DPDK patches and discussions
 help / color / mirror / Atom feed

* [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub
@ 2017-03-21 21:43 Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3 Ed Czeck
                   ` (6 more replies)
  0 siblings, 7 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

Enable Arkville on supported configurations
Add overview documentation
Minimum driver support for valid compile


Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 MAINTAINERS                                 |   8 +
 config/common_base                          |  11 ++
 config/defconfig_arm-armv7a-linuxapp-gcc    |   1 +
 config/defconfig_ppc_64-power8-linuxapp-gcc |   1 +
 doc/guides/nics/ark.rst                     | 237 +++++++++++++++++++++++
 doc/guides/nics/features/ark.ini            |  15 ++
 doc/guides/nics/index.rst                   |   1 +
 drivers/net/Makefile                        |   1 +
 drivers/net/ark/Makefile                    |  63 +++++++
 drivers/net/ark/ark_debug.h                 |  74 ++++++++
 drivers/net/ark/ark_ethdev.c                | 281 ++++++++++++++++++++++++++++
 drivers/net/ark/ark_ethdev.h                |  39 ++++
 drivers/net/ark/ark_global.h                | 108 +++++++++++
 drivers/net/ark/rte_pmd_ark_version.map     |   4 +
 mk/rte.app.mk                               |   1 +
 15 files changed, 845 insertions(+)
 create mode 100644 doc/guides/nics/ark.rst
 create mode 100644 doc/guides/nics/features/ark.ini
 create mode 100644 drivers/net/ark/Makefile
 create mode 100644 drivers/net/ark/ark_debug.h
 create mode 100644 drivers/net/ark/ark_ethdev.c
 create mode 100644 drivers/net/ark/ark_ethdev.h
 create mode 100644 drivers/net/ark/ark_global.h
 create mode 100644 drivers/net/ark/rte_pmd_ark_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index 0c78b58..8043d75 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -278,6 +278,14 @@ M: Evgeny Schemeilin <evgenys@amazon.com>
 F: drivers/net/ena/
 F: doc/guides/nics/ena.rst
 
+Atomic Rules ark
+M: Shepard Siegel <shepard.siegel@atomicrules.com>
+M: Ed Czeck       <ed.czeck@atomicrules.com>
+M: John Miller    <john.miller@atomicrules.com>
+F: /drivers/net/ark/
+F: doc/guides/nics/ark.rst
+F: doc/guides/nics/features/ark.ini
+
 Broadcom bnxt
 M: Stephen Hurd <stephen.hurd@broadcom.com>
 M: Ajit Khaparde <ajit.khaparde@broadcom.com>
diff --git a/config/common_base b/config/common_base
index 37aa1e1..0916c44 100644
--- a/config/common_base
+++ b/config/common_base
@@ -353,6 +353,17 @@ CONFIG_RTE_LIBRTE_QEDE_FW=""
 CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n
 
 #
+# Compile ARK PMD
+#
+CONFIG_RTE_LIBRTE_ARK_PMD=y
+CONFIG_RTE_LIBRTE_ARK_PAD_TX=y
+CONFIG_RTE_LIBRTE_ARK_DEBUG_RX=n
+CONFIG_RTE_LIBRTE_ARK_DEBUG_TX=n
+CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS=n
+CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE=n
+
+
+#
 # Compile the TAP PMD
 # It is enabled by default for Linux only.
 #
diff --git a/config/defconfig_arm-armv7a-linuxapp-gcc b/config/defconfig_arm-armv7a-linuxapp-gcc
index d9bd2a8..6d2b5e0 100644
--- a/config/defconfig_arm-armv7a-linuxapp-gcc
+++ b/config/defconfig_arm-armv7a-linuxapp-gcc
@@ -61,6 +61,7 @@ CONFIG_RTE_SCHED_VECTOR=n
 
 # cannot use those on ARM
 CONFIG_RTE_KNI_KMOD=n
+CONFIG_RTE_LIBRTE_ARK_PMD=n
 CONFIG_RTE_LIBRTE_EM_PMD=n
 CONFIG_RTE_LIBRTE_IGB_PMD=n
 CONFIG_RTE_LIBRTE_CXGBE_PMD=n
diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc b/config/defconfig_ppc_64-power8-linuxapp-gcc
index 35f7fb6..89bc396 100644
--- a/config/defconfig_ppc_64-power8-linuxapp-gcc
+++ b/config/defconfig_ppc_64-power8-linuxapp-gcc
@@ -48,6 +48,7 @@ CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
 
 # Note: Initially, all of the PMD drivers compilation are turned off on Power
 # Will turn on them only after the successful testing on Power
+CONFIG_RTE_LIBRTE_ARK_PMD=n
 CONFIG_RTE_LIBRTE_IXGBE_PMD=n
 CONFIG_RTE_LIBRTE_I40E_PMD=n
 CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
diff --git a/doc/guides/nics/ark.rst b/doc/guides/nics/ark.rst
new file mode 100644
index 0000000..72fb8d6
--- /dev/null
+++ b/doc/guides/nics/ark.rst
@@ -0,0 +1,237 @@
+.. BSD LICENSE
+
+    Copyright (c) 2015-2017 Atomic Rules LLC
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Atomic Rules LLC nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ARK Poll Mode Driver
+====================
+
+The ARK PMD is a DPDK poll-mode driver for the Atomic Rules Arkville
+(ARK) family of devices.
+
+More information can be found at the `Atomic Rules website
+<http://atomicrules.com>`_.
+
+Overview
+--------
+
+The Atomic Rules Arkville product is DPDK and AXI compliant product
+that marshals packets across a PCIe conduit between host DPDK mbufs and
+FPGA AXI streams.
+
+The ARK PMD, and the spirit of the overall Arkville product,
+has been to take the DPDK API/ABI as a fixed specification;
+then implement much of the business logic in FPGA RTL circuits.
+The approach of *working backwards* from the DPDK API/ABI and having
+the GPP host software *dictate*, while the FPGA hardware *copes*,
+results in significant performance gains over a naive implementation.
+
+While this document describes the ARK PMD software, it is helpful to
+understand what the FPGA hardware is and is not. The Arkville RTL
+component provides a single PCIe Physical Function (PF) supporting
+some number of RX/Ingress and TX/Egress Queues. The ARK PMD controls
+the Arkville core through a dedicated opaque Core BAR (CBAR).
+To allow users full freedom for their own FPGA application IP,
+an independent FPGA Application BAR (ABAR) is provided.
+
+One popular way to imagine Arkville's FPGA hardware aspect is as the
+FPGA PCIe-facing side of a so-called Smart NIC. The Arkville core does
+not contain any MACs, and is link-speed independent, as well as
+agnostic to the number of physical ports the application chooses to
+use. The ARK driver exposes the familiar PMD interface to allow packet
+movement to and from mbufs across multiple queues.
+
+However FPGA RTL applications could contain a universe of added
+functionality that an Arkville RTL core does not provide or can
+not anticipate. To allow for this expectation of user-defined
+innovation, the ARK PMD provides a dynamic mechanism of adding
+capabilities without having to modify the ARK PMD.
+
+The ARK PMD is intended to support all instances of the Arkville
+RTL Core, regardless of configuration, FPGA vendor, or target
+board. While specific capabilities such as number of physical
+hardware queue-pairs are negotiated; the driver is designed to
+remain constant over a broad and extendable feature set.
+
+Intentionally, Arkville by itself DOES NOT provide common NIC
+capabilities such as offload or receive-side scaling (RSS).
+These capabilities would be viewed as a gate-level "tax" on
+Green-box FPGA applications that do not require such function.
+Instead, they can be added as needed with essentially no
+overhead to the FPGA Application.
+
+Data Path Interface
+-------------------
+
+Ingress RX and Egress TX operation is by the nominal DPDK API .
+The driver supports single-port, multi-queue for both RX and TX.
+
+Refer to ``ark_ethdev.h`` for the list of supported methods to
+act upon RX and TX Queues.
+
+Configuration Information
+-------------------------
+
+**DPDK Configuration Parameters**
+
+  The following configuration options are available for the ARK PMD:
+
+   * **CONFIG_RTE_LIBRTE_ARK_PMD** (default y): Enables or disables inclusion
+     of the ARK PMD driver in the DPDK compilation.
+
+   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_RX** (default n): Enables or disables debug
+     logging and internal checking of RX ingress logic within the ARK PMD driver.
+
+   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TX** (default n): Enables or disables debug
+     logging and internal checking of TX egress logic within the ARK PMD driver.
+
+   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS** (default n): Enables or disables debug
+     logging of detailed packet and performance statistics gathered in
+     the PMD and FPGA.
+
+   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE** (default n): Enables or disables debug
+     logging of detailed PMD events and status.
+
+
+Building DPDK
+-------------
+
+See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
+instructions on how to build DPDK.
+
+By default the ARK PMD library will be built into the DPDK library.
+
+For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
+documentation that comes with DPDK suite <linux_gsg>`.
+
+Supported ARK RTL PCIe Instances
+--------------------------------
+
+ARK PMD supports the following Arkville RTL PCIe instances including:
+
+* ``1d6c:100d`` - AR-ARKA-FX0 [Arkville 32B DPDK Data Mover]
+* ``1d6c:100e`` - AR-ARKA-FX1 [Arkville 64B DPDK Data Mover]
+
+Supported Operating Systems
+---------------------------
+
+Any Linux distribution fulfilling the conditions described in ``System Requirements``
+section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK Release Notes*.
+
+Supported Features
+------------------
+
+* Dynamic ARK PMD extensions
+* Multiple receive and transmit queues
+* Jumbo frames up to 9K
+* Hardware Statistics
+
+Unsupported Features
+--------------------
+
+Features that may be part of, or become part of, the Arkville RTL IP that are
+not currently supported or exposed by the ARK PMD include:
+
+* PCIe SR-IOV Virtual Functions (VFs)
+* Arkville's Packet Generator Control and Status
+* Arkville's Packet Director Control and Status
+* Arkville's Packet Checker Control and Status
+* Arkville's Timebase Management
+
+Pre-Requisites
+--------------
+
+#. Prepare the system as recommended by DPDK suite.  This includes environment
+   variables, hugepages configuration, tool-chains and configuration
+
+#. Insert igb_uio kernel module using the command 'modprobe igb_uio'
+
+#. Bind the intended ARK device to igb_uio module
+
+At this point the system should be ready to run DPDK applications. Once the
+application runs to completion, the ARK PMD can be detached from igb_uio if necessary.
+
+Usage Example
+-------------
+
+This section demonstrates how to launch **testpmd** with Atomic Rules ARK
+devices managed by librte_pmd_ark.
+
+#. Load the kernel modules:
+
+   .. code-block:: console
+
+      modprobe uio
+      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
+
+   .. note::
+
+      The ARK PMD driver depends upon the igb_uio user space I/O kernel module
+
+#. Mount and request huge pages:
+
+   .. code-block:: console
+
+      mount -t hugetlbfs nodev /mnt/huge
+      echo 256 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
+
+#. Bind UIO driver to ARK device at 0000:01:00.0 (using dpdk-devbind.py):
+
+   .. code-block:: console
+
+      ./usertools/dpdk-devbind.py --bind=igb_uio 0000:01:00.0
+
+   .. note::
+
+      The last argument to dpdk-devbind.py is the 4-tuple that indentifies a specific PCIe
+      device. You can use lspci -d 1d6c: to indentify all Atomic Rules devices in the system,
+      and thus determine the correct 4-tuple argument to dpdk-devbind.py
+
+#. Start testpmd with basic parameters:
+
+   .. code-block:: console
+
+      ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:01:00.0 on NUMA socket -1
+      EAL:   probe driver: 1d6c:100e rte_ark_pmd
+      EAL:   PCI memory mapped at 0x7f9b6c400000
+      PMD: eth_ark_dev_init(): Initializing 0:2:0.1
+      ARKP PMD CommitID: 378f3a67
+      Configuring Port 0 (socket 0)
+      Port 0: DC:3C:F6:00:00:01
+      Checking link statuses...
+      Port 0 Link Up - speed 100000 Mbps - full-duplex
+      Done
+      testpmd>
diff --git a/doc/guides/nics/features/ark.ini b/doc/guides/nics/features/ark.ini
new file mode 100644
index 0000000..dc8a0e2
--- /dev/null
+++ b/doc/guides/nics/features/ark.ini
@@ -0,0 +1,15 @@
+;
+; Supported features of the 'ark' poll mode driver.
+;
+; Refer to default.ini for the full list of available PMD features.
+;
+[Features]
+Queue start/stop     = Y
+Jumbo frame          = Y
+Scattered Rx         = Y
+Basic stats          = Y
+Stats per queue      = Y
+FW version           = Y
+Linux UIO            = Y
+x86-64               = Y
+Usage doc            = Y
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 87f9334..381d82c 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -36,6 +36,7 @@ Network Interface Controller Drivers
     :numbered:
 
     overview
+    ark
     bnx2x
     bnxt
     cxgbe
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index a16f25e..ea9868b 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -32,6 +32,7 @@
 include $(RTE_SDK)/mk/rte.vars.mk
 
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet
+DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark
 DIRS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD) += bnx2x
 DIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += bonding
 DIRS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += cxgbe
diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
new file mode 100644
index 0000000..217bd34
--- /dev/null
+++ b/drivers/net/ark/Makefile
@@ -0,0 +1,63 @@
+# BSD LICENSE
+#
+# Copyright (c) 2015-2017 Atomic Rules LLC
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+#   notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#   notice, this list of conditions and the following disclaimer in
+#   the documentation and/or other materials provided with the
+#   distribution.
+# * Neither the name of copyright holder nor the names of its
+#   contributors may be used to endorse or promote products derived
+#   from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_pmd_ark.a
+
+CFLAGS += -O3 -I./
+CFLAGS += $(WERROR_FLAGS)
+
+EXPORT_MAP := rte_pmd_ark_version.map
+
+LIBABIVER := 1
+
+#
+# all source are stored in SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD)
+#
+
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
+
+
+# this lib depends upon:
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_mbuf
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_ether
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_kvargs
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_eal
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_mempool
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/libpthread
+DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/libdl
+
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/ark/ark_debug.h b/drivers/net/ark/ark_debug.h
new file mode 100644
index 0000000..a108c28
--- /dev/null
+++ b/drivers/net/ark/ark_debug.h
@@ -0,0 +1,74 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_DEBUG_H_
+#define _ARK_DEBUG_H_
+
+#include <inttypes.h>
+#include <rte_log.h>
+
+/* Format specifiers for string data pairs */
+#define ARK_SU32  "\n\t%-20s    %'20" PRIu32
+#define ARK_SU64  "\n\t%-20s    %'20" PRIu64
+#define ARK_SU64X "\n\t%-20s    %#20" PRIx64
+#define ARK_SPTR  "\n\t%-20s    %20p"
+
+#define ARK_TRACE_ON(fmt, ...) \
+	PMD_DRV_LOG(ERR, fmt, ##__VA_ARGS__)
+
+#define ARK_TRACE_OFF(fmt, ...) \
+	do {if (0) PMD_DRV_LOG(ERR, fmt, ##__VA_ARGS__); } while (0)
+
+/* Debug macro for reporting Packet stats */
+#ifdef RTE_LIBRTE_ARK_DEBUG_STATS
+#define ARK_DEBUG_STATS(fmt, ...) ARK_TRACE_ON(fmt, ##__VA_ARGS__)
+#else
+#define ARK_DEBUG_STATS(fmt, ...)  ARK_TRACE_OFF(fmt, ##__VA_ARGS__)
+#endif
+
+/* Debug macro for tracing full behavior*/
+#ifdef RTE_LIBRTE_ARK_DEBUG_TRACE
+#define ARK_DEBUG_TRACE(fmt, ...)  ARK_TRACE_ON(fmt, ##__VA_ARGS__)
+#else
+#define ARK_DEBUG_TRACE(fmt, ...)  ARK_TRACE_OFF(fmt, ##__VA_ARGS__)
+#endif
+
+#ifdef ARK_STD_LOG
+#define PMD_DRV_LOG(level, fmt, args...) \
+	fprintf(stderr, fmt, args)
+#else
+#define PMD_DRV_LOG(level, fmt, args...) \
+	RTE_LOG(level, PMD, "%s(): " fmt, __func__, ## args)
+#endif
+
+#endif
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
new file mode 100644
index 0000000..124b73c
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev.c
@@ -0,0 +1,281 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+#include <sys/stat.h>
+#include <dlfcn.h>
+
+#include <rte_kvargs.h>
+#include <rte_vdev.h>
+
+#include "ark_global.h"
+#include "ark_debug.h"
+#include "ark_ethdev.h"
+
+/*  Internal prototypes */
+static int eth_ark_check_args(const char *params);
+static int eth_ark_dev_init(struct rte_eth_dev *dev);
+static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
+static int eth_ark_dev_configure(struct rte_eth_dev *dev);
+static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
+				 struct rte_eth_dev_info *dev_info);
+
+
+#define ARK_DEV_TO_PCI(eth_dev)			\
+	RTE_DEV_TO_PCI((eth_dev)->device)
+
+#define ARK_MAX_ARG_LEN 256
+static uint32_t pkt_dir_v;
+static char pkt_gen_args[ARK_MAX_ARG_LEN];
+static char pkt_chkr_args[ARK_MAX_ARG_LEN];
+
+#define ARK_PKTGEN_ARG "Pkt_gen"
+#define ARK_PKTCHKR_ARG "Pkt_chkr"
+#define ARK_PKTDIR_ARG "Pkt_dir"
+
+static const char * const valid_arguments[] = {
+	ARK_PKTGEN_ARG,
+	ARK_PKTCHKR_ARG,
+	ARK_PKTDIR_ARG,
+	"iface",
+	NULL
+};
+
+#define MAX_ARK_PHYS 16
+struct ark_adapter *gark[MAX_ARK_PHYS];
+
+static const struct rte_pci_id pci_id_ark_map[] = {
+	{RTE_PCI_DEVICE(0x1d6c, 0x100d)},
+	{RTE_PCI_DEVICE(0x1d6c, 0x100e)},
+	{.vendor_id = 0, /* sentinel */ },
+};
+
+static struct eth_driver rte_ark_pmd = {
+	.pci_drv = {
+		.probe = rte_eth_dev_pci_probe,
+		.remove = rte_eth_dev_pci_remove,
+		.id_table = pci_id_ark_map,
+		.drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC
+	},
+	.eth_dev_init = eth_ark_dev_init,
+	.eth_dev_uninit = eth_ark_dev_uninit,
+	.dev_private_size = sizeof(struct ark_adapter),
+};
+
+static const struct eth_dev_ops ark_eth_dev_ops = {
+	.dev_configure = eth_ark_dev_configure,
+	.dev_infos_get = eth_ark_dev_info_get,
+
+};
+
+
+static int
+eth_ark_dev_init(struct rte_eth_dev *dev __rte_unused)
+{
+	return -1;					/* STUB */
+}
+
+
+static int
+eth_ark_dev_uninit(struct rte_eth_dev *dev)
+{
+	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
+		return 0;
+
+	dev->dev_ops = NULL;
+	dev->rx_pkt_burst = NULL;
+	dev->tx_pkt_burst = NULL;
+	return 0;
+}
+
+static int
+eth_ark_dev_configure(struct rte_eth_dev *dev __rte_unused)
+{
+	ARK_DEBUG_TRACE("ARKP: In %s\n", __func__);
+	return 0;
+}
+
+static void
+eth_ark_dev_info_get(struct rte_eth_dev *dev,
+		     struct rte_eth_dev_info *dev_info)
+{
+	/* device specific configuration */
+	memset(dev_info, 0, sizeof(*dev_info));
+
+	dev_info->max_rx_pktlen = (16 * 1024) - 128;
+	dev_info->min_rx_bufsize = 1024;
+	dev_info->rx_offload_capa = 0;
+	dev_info->tx_offload_capa = 0;
+
+	dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+		.nb_max = 4096 * 4,
+		.nb_min = 512,	/* HW Q size for RX */
+		.nb_align = 2,};
+
+	dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
+		.nb_max = 4096 * 4,
+		.nb_min = 256,	/* HW Q size for TX */
+		.nb_align = 2,};
+
+	dev_info->rx_offload_capa = 0;
+	dev_info->tx_offload_capa = 0;
+
+	/* ARK PMD supports all line rates, how do we indicate that here ?? */
+	dev_info->speed_capa = (ETH_LINK_SPEED_1G |
+				ETH_LINK_SPEED_10G |
+				ETH_LINK_SPEED_25G |
+				ETH_LINK_SPEED_40G |
+				ETH_LINK_SPEED_50G |
+				ETH_LINK_SPEED_100G);
+	dev_info->pci_dev = ARK_DEV_TO_PCI(dev);
+	dev_info->driver_name = dev->data->drv_name;
+}
+
+
+static inline int
+process_pktdir_arg(const char *key, const char *value,
+		   void *extra_args __rte_unused)
+{
+	ARK_DEBUG_TRACE("In process_pktdir_arg, key = %s, value = %s\n",
+			key, value);
+	pkt_dir_v = strtol(value, NULL, 16);
+	ARK_DEBUG_TRACE("pkt_dir_v = 0x%x\n", pkt_dir_v);
+	return 0;
+}
+
+static inline int
+process_file_args(const char *key, const char *value, void *extra_args)
+{
+	ARK_DEBUG_TRACE("**** IN process_pktgen_arg, key = %s, value = %s\n",
+			key, value);
+	char *args = (char *)extra_args;
+
+	/* Open the configuration file */
+	FILE *file = fopen(value, "r");
+	char line[256];
+	int first = 1;
+
+	while (fgets(line, sizeof(line), file)) {
+		/* ARK_DEBUG_TRACE("%s\n", line); */
+		if (first) {
+			strncpy(args, line, ARK_MAX_ARG_LEN);
+			first = 0;
+		} else {
+			strncat(args, line, ARK_MAX_ARG_LEN);
+		}
+	}
+	ARK_DEBUG_TRACE("file = %s\n", args);
+	fclose(file);
+	return 0;
+}
+
+static int
+eth_ark_check_args(const char *params)
+{
+	struct rte_kvargs *kvlist;
+	unsigned int k_idx;
+	struct rte_kvargs_pair *pair = NULL;
+
+	kvlist = rte_kvargs_parse(params, valid_arguments);
+	if (kvlist == NULL)
+		return 0;
+
+	pkt_gen_args[0] = 0;
+	pkt_chkr_args[0] = 0;
+
+	for (k_idx = 0; k_idx < kvlist->count; k_idx++) {
+		pair = &kvlist->pairs[k_idx];
+		ARK_DEBUG_TRACE("**** Arg passed to PMD = %s:%s\n", pair->key,
+				pair->value);
+	}
+
+	if (rte_kvargs_process(kvlist,
+			       ARK_PKTDIR_ARG,
+			       &process_pktdir_arg,
+			       NULL) != 0) {
+		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTDIR_ARG);
+	}
+
+	if (rte_kvargs_process(kvlist,
+			       ARK_PKTGEN_ARG,
+			       &process_file_args,
+			       pkt_gen_args) != 0) {
+		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTGEN_ARG);
+	}
+
+	if (rte_kvargs_process(kvlist,
+			       ARK_PKTCHKR_ARG,
+			       &process_file_args,
+			       pkt_chkr_args) != 0) {
+		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTCHKR_ARG);
+	}
+
+	ARK_DEBUG_TRACE("INFO: packet director set to 0x%x\n", pkt_dir_v);
+
+	return 1;
+}
+
+
+/*
+ * PCIE
+ */
+static int
+pmd_ark_probe(const char *name, const char *params)
+{
+	RTE_LOG(INFO, PMD, "Initializing pmd_ark for %s params = %s\n", name,
+		params);
+
+	/* Parse off the v index */
+
+	eth_ark_check_args(params);
+	return 0;
+}
+
+static int
+pmd_ark_remove(const char *name)
+{
+	RTE_LOG(INFO, PMD, "Closing ark %s ethdev on numa socket %u\n", name,
+		rte_socket_id());
+	return 1;
+}
+
+static struct rte_vdev_driver pmd_ark_drv = {
+	.probe = pmd_ark_probe,
+	.remove = pmd_ark_remove,
+};
+
+RTE_PMD_REGISTER_VDEV(net_ark, pmd_ark_drv);
+RTE_PMD_REGISTER_ALIAS(net_ark, eth_ark);
+RTE_PMD_REGISTER_PCI(eth_ark, rte_ark_pmd.pci_drv);
+RTE_PMD_REGISTER_KMOD_DEP(net_ark, "* igb_uio | uio_pci_generic ");
+RTE_PMD_REGISTER_PCI_TABLE(eth_ark, pci_id_ark_map);
diff --git a/drivers/net/ark/ark_ethdev.h b/drivers/net/ark/ark_ethdev.h
new file mode 100644
index 0000000..08d7fb1
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev.h
@@ -0,0 +1,39 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_ETHDEV_H_
+#define _ARK_ETHDEV_H_
+
+/* STUB */
+
+#endif
diff --git a/drivers/net/ark/ark_global.h b/drivers/net/ark/ark_global.h
new file mode 100644
index 0000000..7cd62d5
--- /dev/null
+++ b/drivers/net/ark/ark_global.h
@@ -0,0 +1,108 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_GLOBAL_H_
+#define _ARK_GLOBAL_H_
+
+#include <time.h>
+#include <assert.h>
+
+#include <rte_mbuf.h>
+#include <rte_ethdev.h>
+#include <rte_malloc.h>
+#include <rte_memcpy.h>
+#include <rte_string_fns.h>
+#include <rte_cycles.h>
+#include <rte_kvargs.h>
+#include <rte_dev.h>
+#include <rte_version.h>
+
+#define ETH_ARK_ARG_MAXLEN	64
+#define ARK_SYSCTRL_BASE  0x0
+#define ARK_PKTGEN_BASE   0x10000
+#define ARK_MPU_RX_BASE   0x20000
+#define ARK_UDM_BASE      0x30000
+#define ARK_MPU_TX_BASE   0x40000
+#define ARK_DDM_BASE      0x60000
+#define ARK_CMAC_BASE     0x80000
+#define ARK_PKTDIR_BASE   0xa0000
+#define ARK_PKTCHKR_BASE  0x90000
+#define ARK_RCPACING_BASE 0xb0000
+#define ARK_EXTERNAL_BASE 0x100000
+#define ARK_MPU_QOFFSET   0x00100
+#define ARK_MAX_PORTS     8
+
+#define offset8(n)     n
+#define offset16(n)   ((n) / 2)
+#define offset32(n)   ((n) / 4)
+#define offset64(n)   ((n) / 8)
+
+/*
+ * Structure to store private data for each PF/VF instance.
+ */
+#define def_ptr(type, name) \
+	union type {		   \
+		uint64_t *t64;	   \
+		uint32_t *t32;	   \
+		uint16_t *t16;	   \
+		uint8_t  *t8;	   \
+		void     *v;	   \
+	} name
+
+struct ark_port {
+	struct rte_eth_dev *eth_dev;
+	int id;
+};
+
+struct ark_adapter {
+	/* User extension private data */
+	void *user_data;
+
+	struct ark_port port[ARK_MAX_PORTS];
+	int num_ports;
+
+	/* Common for both PF and VF */
+	struct rte_eth_dev *eth_dev;
+
+	void *d_handle;
+
+	/* Our Bar 0 */
+	uint8_t *bar0;
+
+	/* Application Bar */
+	uint8_t *a_bar;
+};
+
+typedef uint32_t *ark_t;
+
+#endif
diff --git a/drivers/net/ark/rte_pmd_ark_version.map b/drivers/net/ark/rte_pmd_ark_version.map
new file mode 100644
index 0000000..7f84780
--- /dev/null
+++ b/drivers/net/ark/rte_pmd_ark_version.map
@@ -0,0 +1,4 @@
+DPDK_2.0 {
+	 local: *;
+
+};
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 0e0b600..da23898 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -104,6 +104,7 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
+_LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD)        += -lrte_pmd_ark
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
 _LDLIBS-$(CONFIG_RTE_LIBRTE_BNXT_PMD)       += -lrte_pmd_bnxt
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-21 22:15   ` Stephen Hemminger
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 3/7] net/ark: HW API part 2 " Ed Czeck
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

Provide C-level interface for Arkville's internal HW resources
mpu, pktdir, and rqp modules

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/Makefile     |   3 +
 drivers/net/ark/ark_mpu.c    | 181 +++++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_mpu.h    | 143 ++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_pktdir.c |  83 ++++++++++++++++++++
 drivers/net/ark/ark_pktdir.h |  68 ++++++++++++++++
 drivers/net/ark/ark_rqp.c    |  91 ++++++++++++++++++++++
 drivers/net/ark/ark_rqp.h    |  75 ++++++++++++++++++
 7 files changed, 644 insertions(+)
 create mode 100644 drivers/net/ark/ark_mpu.c
 create mode 100644 drivers/net/ark/ark_mpu.h
 create mode 100644 drivers/net/ark/ark_pktdir.c
 create mode 100644 drivers/net/ark/ark_pktdir.h
 create mode 100644 drivers/net/ark/ark_rqp.c
 create mode 100644 drivers/net/ark/ark_rqp.h

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index 217bd34..2722062 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -48,6 +48,9 @@ LIBABIVER := 1
 #
 
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktdir.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_mpu.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_rqp.c
 
 
 # this lib depends upon:
diff --git a/drivers/net/ark/ark_mpu.c b/drivers/net/ark/ark_mpu.c
new file mode 100644
index 0000000..ed04dee
--- /dev/null
+++ b/drivers/net/ark/ark_mpu.c
@@ -0,0 +1,181 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_debug.h"
+#include "ark_mpu.h"
+
+uint16_t
+ark_api_num_queues(struct ark_mpu_t *mpu)
+{
+	return mpu->hw.num_queues;
+}
+
+uint16_t
+ark_api_num_queues_per_port(struct ark_mpu_t *mpu, uint16_t ark_ports)
+{
+	return mpu->hw.num_queues / ark_ports;
+}
+
+int
+ark_mpu_verify(struct ark_mpu_t *mpu, uint32_t obj_size)
+{
+	uint32_t version;
+
+	version = mpu->id.vernum & 0x0000fF00;
+	if ((mpu->id.idnum != 0x2055504d) ||
+	    (mpu->hw.obj_size != obj_size) ||
+	    (version != 0x00003100)) {
+		PMD_DRV_LOG(ERR,
+			    "   MPU module not found as expected %08x"
+			    " \"%c%c%c%c %c%c%c%c\"\n",
+			    mpu->id.idnum,
+			    mpu->id.id[0], mpu->id.id[1],
+			    mpu->id.id[2], mpu->id.id[3],
+			    mpu->id.ver[0], mpu->id.ver[1],
+			    mpu->id.ver[2], mpu->id.ver[3]);
+		PMD_DRV_LOG(ERR,
+			    "   MPU HW num_queues: %u hw_depth %u,"
+			    " obj_size: %u, obj_per_mrr: %u"
+			    " Expected size %u\n",
+			    mpu->hw.num_queues,
+			    mpu->hw.hw_depth,
+			    mpu->hw.obj_size,
+			    mpu->hw.obj_per_mrr,
+			    obj_size);
+		return -1;
+	}
+	return 0;
+}
+
+void
+ark_mpu_stop(struct ark_mpu_t *mpu)
+{
+	mpu->cfg.command = MPU_CMD_STOP;
+}
+
+void
+ark_mpu_start(struct ark_mpu_t *mpu)
+{
+	mpu->cfg.command = MPU_CMD_RUN;
+}
+
+int
+ark_mpu_reset(struct ark_mpu_t *mpu)
+{
+	int cnt = 0;
+
+	mpu->cfg.command = MPU_CMD_RESET;
+
+	while (mpu->cfg.command != MPU_CMD_IDLE) {
+		if (cnt++ > 1000)
+			break;
+		usleep(10);
+	}
+	if (mpu->cfg.command != MPU_CMD_IDLE) {
+		mpu->cfg.command = MPU_CMD_FORCE_RESET;
+		usleep(10);
+	}
+	ark_mpu_reset_stats(mpu);
+	return mpu->cfg.command != MPU_CMD_IDLE;
+}
+
+void
+ark_mpu_reset_stats(struct ark_mpu_t *mpu)
+{
+	mpu->stats.pci_request = 1;	/* reset stats */
+}
+
+int
+ark_mpu_configure(struct ark_mpu_t *mpu, phys_addr_t ring, uint32_t ring_size,
+		  int is_tx)
+{
+	ark_mpu_reset(mpu);
+
+	if (!rte_is_power_of_2(ring_size)) {
+		PMD_DRV_LOG(ERR, "ARK: Invalid ring size for MPU %d\n",
+			    ring_size);
+		return -1;
+	}
+
+	mpu->cfg.ring_base = ring;
+	mpu->cfg.ring_size = ring_size;
+	mpu->cfg.ring_mask = ring_size - 1;
+	mpu->cfg.min_host_move = is_tx ? 1 : mpu->hw.obj_per_mrr;
+	mpu->cfg.min_hw_move = mpu->hw.obj_per_mrr;
+	mpu->cfg.sw_prod_index = 0;
+	mpu->cfg.hw_cons_index = 0;
+	return 0;
+}
+
+void
+ark_mpu_dump(struct ark_mpu_t *mpu, const char *code, uint16_t qid)
+{
+	/* DUMP to see that we have started */
+	ARK_DEBUG_TRACE("ARKP MPU: %s Q: %3u sw_prod %u, hw_cons: %u\n",
+			code, qid,
+			mpu->cfg.sw_prod_index, mpu->cfg.hw_cons_index);
+	ARK_DEBUG_TRACE("ARKP MPU: %s state: %d count %d, reserved %d"
+			" data 0x%08x_%08x 0x%08x_%08x\n",
+			code,
+			mpu->debug.state, mpu->debug.count,
+			mpu->debug.reserved,
+			mpu->debug.peek[1],
+			mpu->debug.peek[0],
+			mpu->debug.peek[3],
+			mpu->debug.peek[2]
+			);
+	ARK_DEBUG_STATS("ARKP MPU: %s Q: %3u"
+			ARK_SU64 ARK_SU64 ARK_SU64 ARK_SU64
+			ARK_SU64 ARK_SU64 ARK_SU64 "\n",
+			code, qid,
+			"PCI Request:", mpu->stats.pci_request,
+			"Queue_empty", mpu->stats.q_empty,
+			"Queue_q1", mpu->stats.q_q1,
+			"Queue_q2", mpu->stats.q_q2,
+			"Queue_q3", mpu->stats.q_q3,
+			"Queue_q4", mpu->stats.q_q4,
+			"Queue_full", mpu->stats.q_full
+			);
+}
+
+void
+ark_mpu_dump_setup(struct ark_mpu_t *mpu, uint16_t q_id)
+{
+	ARK_DEBUG_TRACE("MPU Setup Q: %u"
+			ARK_SU64X "\n",
+			q_id,
+			"ring_base", mpu->cfg.ring_base
+			);
+}
diff --git a/drivers/net/ark/ark_mpu.h b/drivers/net/ark/ark_mpu.h
new file mode 100644
index 0000000..376c042
--- /dev/null
+++ b/drivers/net/ark/ark_mpu.h
@@ -0,0 +1,143 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_MPU_H_
+#define _ARK_MPU_H_
+
+#include <stdint.h>
+
+#include <rte_memory.h>
+
+/*
+ * MPU hardware structures
+ */
+
+#define ARK_MPU_ID 0x00
+struct ark_mpu_id_t {
+	union {
+	char id[4];
+	uint32_t idnum;
+	};
+	union {
+	char ver[4];
+	uint32_t vernum;
+	};
+	uint32_t phys_id;
+	uint32_t mrr_code;
+};
+
+#define ARK_MPU_HW 0x010
+struct ark_mpu_hw_t {
+	uint16_t num_queues;
+	uint16_t reserved;
+	uint32_t hw_depth;
+	uint32_t obj_size;
+	uint32_t obj_per_mrr;
+};
+
+#define ARK_MPU_CFG 0x040
+struct ark_mpu_cfg_t {
+	phys_addr_t ring_base;	/* phys_addr_t is a uint64_t */
+	uint32_t ring_size;
+	uint32_t ring_mask;
+	uint32_t min_host_move;
+	uint32_t min_hw_move;
+	volatile uint32_t sw_prod_index;
+	volatile uint32_t hw_cons_index;
+	volatile uint32_t command;
+};
+enum ARK_MPU_COMMAND {
+	MPU_CMD_IDLE = 1, MPU_CMD_RUN = 2, MPU_CMD_STOP = 4, MPU_CMD_RESET =
+	8, MPU_CMD_FORCE_RESET = 16, MPU_COMMAND_LIMIT = 0xfFFFFFFF
+};
+
+#define ARK_MPU_STATS 0x080
+struct ark_mpu_stats_t {
+	volatile uint64_t pci_request;
+	volatile uint64_t q_empty;
+	volatile uint64_t q_q1;
+	volatile uint64_t q_q2;
+	volatile uint64_t q_q3;
+	volatile uint64_t q_q4;
+	volatile uint64_t q_full;
+};
+
+#define ARK_MPU_DEBUG 0x0C0
+struct ark_mpu_debug_t {
+	volatile uint32_t state;
+	uint32_t reserved;
+	volatile uint32_t count;
+	volatile uint32_t take;
+	volatile uint32_t peek[4];
+};
+
+/*  Consolidated structure */
+struct ark_mpu_t {
+	struct ark_mpu_id_t id;
+	uint8_t reserved0[(ARK_MPU_HW - ARK_MPU_ID)
+					  - sizeof(struct ark_mpu_id_t)];
+	struct ark_mpu_hw_t hw;
+	uint8_t reserved1[(ARK_MPU_CFG - ARK_MPU_HW) -
+					  sizeof(struct ark_mpu_hw_t)];
+	struct ark_mpu_cfg_t cfg;
+	uint8_t reserved2[(ARK_MPU_STATS - ARK_MPU_CFG) -
+					  sizeof(struct ark_mpu_cfg_t)];
+	struct ark_mpu_stats_t stats;
+	uint8_t reserved3[(ARK_MPU_DEBUG - ARK_MPU_STATS) -
+					  sizeof(struct ark_mpu_stats_t)];
+	struct ark_mpu_debug_t debug;
+};
+
+uint16_t ark_api_num_queues(struct ark_mpu_t *mpu);
+uint16_t ark_api_num_queues_per_port(struct ark_mpu_t *mpu,
+	uint16_t ark_ports);
+int ark_mpu_verify(struct ark_mpu_t *mpu, uint32_t obj_size);
+void ark_mpu_stop(struct ark_mpu_t *mpu);
+void ark_mpu_start(struct ark_mpu_t *mpu);
+int ark_mpu_reset(struct ark_mpu_t *mpu);
+int ark_mpu_configure(struct ark_mpu_t *mpu, phys_addr_t ring,
+	uint32_t ring_size, int is_tx);
+
+void ark_mpu_dump(struct ark_mpu_t *mpu, const char *msg, uint16_t idx);
+void ark_mpu_dump_setup(struct ark_mpu_t *mpu, uint16_t qid);
+void ark_mpu_reset_stats(struct ark_mpu_t *mpu);
+
+static inline void
+ark_mpu_set_producer(struct ark_mpu_t *mpu, uint32_t idx)
+{
+	mpu->cfg.sw_prod_index = idx;
+}
+
+// #define    ark_mpu_set_producer(MPU, IDX) {(MPU)->cfg.sw_prod_index = (IDX);}
+
+#endif
diff --git a/drivers/net/ark/ark_pktdir.c b/drivers/net/ark/ark_pktdir.c
new file mode 100644
index 0000000..9a2a280
--- /dev/null
+++ b/drivers/net/ark/ark_pktdir.c
@@ -0,0 +1,83 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdint.h>
+#include <inttypes.h>
+
+#include "ark_pktdir.h"
+#include "ark_global.h"
+
+
+ark_pkt_dir_t
+ark_pktdir_init(void *base)
+{
+	struct ark_pkt_dir_inst *inst =
+		rte_malloc("ark_pkt_dir_inst",
+			   sizeof(struct ark_pkt_dir_inst),
+			   0);
+	inst->regs = (struct ark_pkt_dir_regs *)base;
+	inst->regs->ctrl = 0x00110110;	/* POR state */
+	return inst;
+}
+
+void
+ark_pktdir_uninit(ark_pkt_dir_t handle)
+{
+	struct ark_pkt_dir_inst *inst = (struct ark_pkt_dir_inst *)handle;
+
+	rte_free(inst);
+}
+
+void
+ark_pktdir_setup(ark_pkt_dir_t handle, uint32_t v)
+{
+	struct ark_pkt_dir_inst *inst = (struct ark_pkt_dir_inst *)handle;
+
+	inst->regs->ctrl = v;
+}
+
+uint32_t
+ark_pktdir_status(ark_pkt_dir_t handle)
+{
+	struct ark_pkt_dir_inst *inst = (struct ark_pkt_dir_inst *)handle;
+
+	return inst->regs->ctrl;
+}
+
+uint32_t
+ark_pktdir_stall_cnt(ark_pkt_dir_t handle)
+{
+	struct ark_pkt_dir_inst *inst = (struct ark_pkt_dir_inst *)handle;
+
+	return inst->regs->stall_cnt;
+}
diff --git a/drivers/net/ark/ark_pktdir.h b/drivers/net/ark/ark_pktdir.h
new file mode 100644
index 0000000..5fc8d41
--- /dev/null
+++ b/drivers/net/ark/ark_pktdir.h
@@ -0,0 +1,68 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_PKTDIR_H_
+#define _ARK_PKTDIR_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+
+#include <rte_eal.h>
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#define ARK_PKTDIR_BASE_ADR  0xa0000
+
+typedef void *ark_pkt_dir_t;
+
+struct ark_pkt_dir_regs {
+	uint32_t ctrl;
+	uint32_t status;
+	uint32_t stall_cnt;
+} __attribute__ ((packed));
+
+struct ark_pkt_dir_inst {
+	volatile struct ark_pkt_dir_regs *regs;
+};
+
+ark_pkt_dir_t ark_pktdir_init(void *base);
+void ark_pktdir_uninit(ark_pkt_dir_t handle);
+void ark_pktdir_setup(ark_pkt_dir_t handle, uint32_t v);
+uint32_t ark_pktdir_stall_cnt(ark_pkt_dir_t handle);
+uint32_t ark_pktdir_status(ark_pkt_dir_t handle);
+
+#endif
diff --git a/drivers/net/ark/ark_rqp.c b/drivers/net/ark/ark_rqp.c
new file mode 100644
index 0000000..a39a715
--- /dev/null
+++ b/drivers/net/ark/ark_rqp.c
@@ -0,0 +1,91 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_rqp.h"
+#include "ark_debug.h"
+
+/* ************************************************************************* */
+void
+ark_rqp_stats_reset(struct ark_rqpace_t *rqp)
+{
+	rqp->stats_clear = 1;
+	/* POR 992 */
+	/* rqp->cpld_max = 992; */
+	/* POR 64 */
+	/* rqp->cplh_max = 64; */
+}
+
+/* ************************************************************************* */
+void
+ark_rqp_dump(struct ark_rqpace_t *rqp)
+{
+	if (rqp->err_count_other != 0)
+		PMD_DRV_LOG(ERR,
+			    "ARKP RQP Errors noted: ctrl: %d cplh_hmax %d cpld_max %d"
+			    ARK_SU32
+			    ARK_SU32 "\n",
+			    rqp->ctrl, rqp->cplh_max, rqp->cpld_max,
+			    "Error Count", rqp->err_cnt,
+			    "Error General", rqp->err_count_other);
+
+	ARK_DEBUG_STATS("ARKP RQP Dump: ctrl: %d cplh_hmax %d cpld_max %d"
+			ARK_SU32
+			ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32
+			ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32
+			ARK_SU32 ARK_SU32 ARK_SU32
+			ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 "\n",
+			rqp->ctrl, rqp->cplh_max, rqp->cpld_max,
+			"Error Count", rqp->err_cnt,
+			"Error General", rqp->err_count_other,
+			"stall_pS", rqp->stall_ps,
+			"stall_pS Min", rqp->stall_ps_min,
+			"stall_pS Max", rqp->stall_ps_max,
+			"req_pS", rqp->req_ps,
+			"req_pS Min", rqp->req_ps_min,
+			"req_pS Max", rqp->req_ps_max,
+			"req_dWPS", rqp->req_dw_ps,
+			"req_dWPS Min", rqp->req_dw_ps_min,
+			"req_dWPS Max", rqp->req_dw_ps_max,
+			"cpl_pS", rqp->cpl_ps,
+			"cpl_pS Min", rqp->cpl_ps_min,
+			"cpl_pS Max", rqp->cpl_ps_max,
+			"cpl_dWPS", rqp->cpl_dw_ps,
+			"cpl_dWPS Min", rqp->cpl_dw_ps_min,
+			"cpl_dWPS Max", rqp->cpl_dw_ps_max,
+			"cplh pending", rqp->cplh_pending,
+			"cpld pending", rqp->cpld_pending,
+			"cplh pending max", rqp->cplh_pending_max,
+			"cpld pending max", rqp->cpld_pending_max);
+}
diff --git a/drivers/net/ark/ark_rqp.h b/drivers/net/ark/ark_rqp.h
new file mode 100644
index 0000000..4376d76
--- /dev/null
+++ b/drivers/net/ark/ark_rqp.h
@@ -0,0 +1,75 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_RQP_H_
+#define _ARK_RQP_H_
+
+#include <stdint.h>
+
+#include <rte_memory.h>
+
+/*
+ * RQ Pacing core hardware structure
+ */
+struct ark_rqpace_t {
+	volatile uint32_t ctrl;
+	volatile uint32_t stats_clear;
+	volatile uint32_t cplh_max;
+	volatile uint32_t cpld_max;
+	volatile uint32_t err_cnt;
+	volatile uint32_t stall_ps;
+	volatile uint32_t stall_ps_min;
+	volatile uint32_t stall_ps_max;
+	volatile uint32_t req_ps;
+	volatile uint32_t req_ps_min;
+	volatile uint32_t req_ps_max;
+	volatile uint32_t req_dw_ps;
+	volatile uint32_t req_dw_ps_min;
+	volatile uint32_t req_dw_ps_max;
+	volatile uint32_t cpl_ps;
+	volatile uint32_t cpl_ps_min;
+	volatile uint32_t cpl_ps_max;
+	volatile uint32_t cpl_dw_ps;
+	volatile uint32_t cpl_dw_ps_min;
+	volatile uint32_t cpl_dw_ps_max;
+	volatile uint32_t cplh_pending;
+	volatile uint32_t cpld_pending;
+	volatile uint32_t cplh_pending_max;
+	volatile uint32_t cpld_pending_max;
+	volatile uint32_t err_count_other;
+};
+
+void ark_rqp_dump(struct ark_rqpace_t *rqp);
+void ark_rqp_stats_reset(struct ark_rqpace_t *rqp);
+
+#endif
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 3/7] net/ark: HW API part 2 of 3
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3 Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 4/7] net/ark: HW API part 3 " Ed Czeck
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

Provide C-level interface for Arkville's internal HW resources
ddm and udm modules

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/Makefile  |   2 +
 drivers/net/ark/ark_ddm.c | 151 +++++++++++++++++++++++++++++++
 drivers/net/ark/ark_ddm.h | 154 +++++++++++++++++++++++++++++++
 drivers/net/ark/ark_udm.c | 226 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_udm.h | 184 +++++++++++++++++++++++++++++++++++++
 5 files changed, 717 insertions(+)
 create mode 100644 drivers/net/ark/ark_ddm.c
 create mode 100644 drivers/net/ark/ark_ddm.h
 create mode 100644 drivers/net/ark/ark_udm.c
 create mode 100644 drivers/net/ark/ark_udm.h

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index 2722062..529d92c 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -50,6 +50,8 @@ LIBABIVER := 1
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktdir.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_mpu.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ddm.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_udm.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_rqp.c
 
 
diff --git a/drivers/net/ark/ark_ddm.c b/drivers/net/ark/ark_ddm.c
new file mode 100644
index 0000000..a0389f2
--- /dev/null
+++ b/drivers/net/ark/ark_ddm.c
@@ -0,0 +1,151 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_debug.h"
+#include "ark_ddm.h"
+
+/* ************************************************************************* */
+int
+ark_ddm_verify(struct ark_ddm_t *ddm)
+{
+	if (sizeof(struct ark_ddm_t) != ARK_DDM_EXPECTED_SIZE) {
+		PMD_DRV_LOG(ERR, "ARK: DDM structure looks incorrect %d vs %zd\n",
+			    ARK_DDM_EXPECTED_SIZE, sizeof(struct ark_ddm_t));
+		return -1;
+	}
+
+	if (ddm->cfg.const0 != ARK_DDM_CONST) {
+		PMD_DRV_LOG(ERR, "ARK: DDM module not found as expected 0x%08x\n",
+			    ddm->cfg.const0);
+		return -1;
+	}
+	return 0;
+}
+
+void
+ark_ddm_start(struct ark_ddm_t *ddm)
+{
+	ddm->cfg.command = 1;
+}
+
+int
+ark_ddm_stop(struct ark_ddm_t *ddm, const int wait)
+{
+	int cnt = 0;
+
+	ddm->cfg.command = 2;
+	while (wait && (ddm->cfg.stop_flushed & 0x01) == 0) {
+		if (cnt++ > 1000)
+			return 1;
+
+		usleep(10);
+	}
+	return 0;
+}
+
+void
+ark_ddm_reset(struct ark_ddm_t *ddm)
+{
+	int status;
+
+	/* reset only works if ddm has stopped properly. */
+	status = ark_ddm_stop(ddm, 1);
+
+	if (status != 0) {
+		ARK_DEBUG_TRACE("ARKP: %s  stop failed  doing forced reset\n",
+				__func__);
+		ddm->cfg.command = 4;
+		usleep(10);
+	}
+	ddm->cfg.command = 3;
+}
+
+void
+ark_ddm_setup(struct ark_ddm_t *ddm, phys_addr_t cons_addr, uint32_t interval)
+{
+	ddm->setup.cons_write_index_addr = cons_addr;
+	ddm->setup.write_index_interval = interval / 4;	/* 4 ns period */
+}
+
+void
+ark_ddm_stats_reset(struct ark_ddm_t *ddm)
+{
+	ddm->cfg.tlp_stats_clear = 1;
+}
+
+void
+ark_ddm_dump(struct ark_ddm_t *ddm, const char *msg)
+{
+	ARK_DEBUG_TRACE("ARKP DDM Dump: %s Stopped: %d\n", msg,
+			ark_ddm_is_stopped(ddm)
+			);
+}
+
+void
+ark_ddm_dump_stats(struct ark_ddm_t *ddm, const char *msg)
+{
+	struct ark_ddm_stats_t *stats = &ddm->stats;
+
+	ARK_DEBUG_STATS("ARKP DDM Stats: %s"
+			ARK_SU64 ARK_SU64 ARK_SU64
+			"\n", msg,
+			"Bytes:", stats->tx_byte_count,
+			"Packets:", stats->tx_pkt_count,
+			"MBufs", stats->tx_mbuf_count);
+}
+
+int
+ark_ddm_is_stopped(struct ark_ddm_t *ddm)
+{
+	return (ddm->cfg.stop_flushed & 0x01) != 0;
+}
+
+uint64_t
+ark_ddm_queue_byte_count(struct ark_ddm_t *ddm)
+{
+	return ddm->queue_stats.byte_count;
+}
+
+uint64_t
+ark_ddm_queue_pkt_count(struct ark_ddm_t *ddm)
+{
+	return ddm->queue_stats.pkt_count;
+}
+
+void
+ark_ddm_queue_reset_stats(struct ark_ddm_t *ddm)
+{
+	ddm->queue_stats.byte_count = 1;
+}
diff --git a/drivers/net/ark/ark_ddm.h b/drivers/net/ark/ark_ddm.h
new file mode 100644
index 0000000..8208b12
--- /dev/null
+++ b/drivers/net/ark/ark_ddm.h
@@ -0,0 +1,154 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_DDM_H_
+#define _ARK_DDM_H_
+
+#include <stdint.h>
+
+#include <rte_memory.h>
+
+/* DDM core hardware structures */
+#define ARK_DDM_CFG 0x0000
+#define ARK_DDM_CONST 0xfacecafe
+struct ark_ddm_cfg_t {
+	uint32_t r0;
+	volatile uint32_t tlp_stats_clear;
+	uint32_t const0;
+	volatile uint32_t tag_max;
+	volatile uint32_t command;
+	volatile uint32_t stop_flushed;
+};
+
+#define ARK_DDM_STATS 0x0020
+struct ark_ddm_stats_t {
+	volatile uint64_t tx_byte_count;
+	volatile uint64_t tx_pkt_count;
+	volatile uint64_t tx_mbuf_count;
+};
+
+#define ARK_DDM_MRDQ 0x0040
+struct ark_ddm_mrdq_t {
+	volatile uint32_t mrd_q1;
+	volatile uint32_t mrd_q2;
+	volatile uint32_t mrd_q3;
+	volatile uint32_t mrd_q4;
+	volatile uint32_t mrd_full;
+};
+
+#define ARK_DDM_CPLDQ 0x0068
+struct ark_ddm_cpldq_t {
+	volatile uint32_t cpld_q1;
+	volatile uint32_t cpld_q2;
+	volatile uint32_t cpld_q3;
+	volatile uint32_t cpld_q4;
+	volatile uint32_t cpld_full;
+};
+
+#define ARK_DDM_MRD_PS 0x0090
+struct ark_ddm_mrd_ps_t {
+	volatile uint32_t mrd_ps_min;
+	volatile uint32_t mrd_ps_max;
+	volatile uint32_t mrd_full_ps_min;
+	volatile uint32_t mrd_full_ps_max;
+	volatile uint32_t mrd_dw_ps_min;
+	volatile uint32_t mrd_dw_ps_max;
+};
+
+#define ARK_DDM_QUEUE_STATS 0x00a8
+struct ark_ddm_qstats_t {
+	volatile uint64_t byte_count;
+	volatile uint64_t pkt_count;
+	volatile uint64_t mbuf_count;
+};
+
+#define ARK_DDM_CPLD_PS 0x00c0
+struct ark_ddm_cpld_ps_t {
+	volatile uint32_t cpld_ps_min;
+	volatile uint32_t cpld_ps_max;
+	volatile uint32_t cpld_full_ps_min;
+	volatile uint32_t cpld_full_ps_max;
+	volatile uint32_t cpld_dw_ps_min;
+	volatile uint32_t cpld_dw_ps_max;
+};
+
+#define ARK_DDM_SETUP  0x00e0
+struct ark_ddm_setup_t {
+	phys_addr_t cons_write_index_addr;
+	uint32_t write_index_interval;	/* 4ns each */
+	volatile uint32_t cons_index;
+};
+
+/*  Consolidated structure */
+struct ark_ddm_t {
+	struct ark_ddm_cfg_t cfg;
+	uint8_t reserved0[(ARK_DDM_STATS - ARK_DDM_CFG) -
+					  sizeof(struct ark_ddm_cfg_t)];
+	struct ark_ddm_stats_t stats;
+	uint8_t reserved1[(ARK_DDM_MRDQ - ARK_DDM_STATS) -
+					  sizeof(struct ark_ddm_stats_t)];
+	struct ark_ddm_mrdq_t mrdq;
+	uint8_t reserved2[(ARK_DDM_CPLDQ - ARK_DDM_MRDQ) -
+					  sizeof(struct ark_ddm_mrdq_t)];
+	struct ark_ddm_cpldq_t cpldq;
+	uint8_t reserved3[(ARK_DDM_MRD_PS - ARK_DDM_CPLDQ) -
+					  sizeof(struct ark_ddm_cpldq_t)];
+	struct ark_ddm_mrd_ps_t mrd_ps;
+	struct ark_ddm_qstats_t queue_stats;
+	struct ark_ddm_cpld_ps_t cpld_ps;
+	uint8_t reserved5[(ARK_DDM_SETUP - ARK_DDM_CPLD_PS) -
+					  sizeof(struct ark_ddm_cpld_ps_t)];
+	struct ark_ddm_setup_t setup;
+	uint8_t reserved_p[(256 - ARK_DDM_SETUP)
+					  - sizeof(struct ark_ddm_setup_t)];
+};
+
+#define ARK_DDM_EXPECTED_SIZE 256
+#define ARK_DDM_QOFFSET ARK_DDM_EXPECTED_SIZE
+
+/* DDM function prototype */
+int ark_ddm_verify(struct ark_ddm_t *ddm);
+void ark_ddm_start(struct ark_ddm_t *ddm);
+int ark_ddm_stop(struct ark_ddm_t *ddm, const int wait);
+void ark_ddm_reset(struct ark_ddm_t *ddm);
+void ark_ddm_stats_reset(struct ark_ddm_t *ddm);
+void ark_ddm_setup(struct ark_ddm_t *ddm, phys_addr_t cons_addr,
+	uint32_t interval);
+void ark_ddm_dump_stats(struct ark_ddm_t *ddm, const char *msg);
+void ark_ddm_dump(struct ark_ddm_t *ddm, const char *msg);
+int ark_ddm_is_stopped(struct ark_ddm_t *ddm);
+uint64_t ark_ddm_queue_byte_count(struct ark_ddm_t *ddm);
+uint64_t ark_ddm_queue_pkt_count(struct ark_ddm_t *ddm);
+void ark_ddm_queue_reset_stats(struct ark_ddm_t *ddm);
+
+#endif
diff --git a/drivers/net/ark/ark_udm.c b/drivers/net/ark/ark_udm.c
new file mode 100644
index 0000000..a4e94f9
--- /dev/null
+++ b/drivers/net/ark/ark_udm.c
@@ -0,0 +1,226 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_debug.h"
+#include "ark_udm.h"
+
+int
+ark_udm_verify(struct ark_udm_t *udm)
+{
+	if (sizeof(struct ark_udm_t) != ARK_UDM_EXPECT_SIZE) {
+		PMD_DRV_LOG(ERR,
+			    "ARK: UDM structure looks incorrect %d vs %zd\n",
+			    ARK_UDM_EXPECT_SIZE, sizeof(struct ark_udm_t));
+		return -1;
+	}
+
+	if (udm->setup.const0 != ARK_UDM_CONST) {
+		PMD_DRV_LOG(ERR,
+			    "ARK: UDM module not found as expected 0x%08x\n",
+			    udm->setup.const0);
+		return -1;
+	}
+	return 0;
+}
+
+int
+ark_udm_stop(struct ark_udm_t *udm, const int wait)
+{
+	int cnt = 0;
+
+	udm->cfg.command = 2;
+
+	while (wait && (udm->cfg.stop_flushed & 0x01) == 0) {
+		if (cnt++ > 1000)
+			return 1;
+
+		usleep(10);
+	}
+	return 0;
+}
+
+int
+ark_udm_reset(struct ark_udm_t *udm)
+{
+	int status;
+
+	status = ark_udm_stop(udm, 1);
+	if (status != 0) {
+		ARK_DEBUG_TRACE("ARKP: %s  stop failed  doing forced reset\n",
+				__func__);
+		udm->cfg.command = 4;
+		usleep(10);
+		udm->cfg.command = 3;
+		status = ark_udm_stop(udm, 0);
+		ARK_DEBUG_TRACE("ARKP: %s  stop status %d post failure"
+				" and forced reset\n",
+				__func__, status);
+	} else {
+		udm->cfg.command = 3;
+	}
+
+	return status;
+}
+
+void
+ark_udm_start(struct ark_udm_t *udm)
+{
+	udm->cfg.command = 1;
+}
+
+void
+ark_udm_stats_reset(struct ark_udm_t *udm)
+{
+	udm->pcibp.pci_clear = 1;
+	udm->tlp_ps.tlp_clear = 1;
+}
+
+void
+ark_udm_configure(struct ark_udm_t *udm,
+		  uint32_t headroom,
+		  uint32_t dataroom,
+		  uint32_t write_interval_ns)
+{
+	/* headroom and data room are in DWs in the UDM */
+	udm->cfg.dataroom = dataroom / 4;
+	udm->cfg.headroom = headroom / 4;
+
+	/* 4 NS period ns */
+	udm->rt_cfg.write_interval = write_interval_ns / 4;
+}
+
+void
+ark_udm_write_addr(struct ark_udm_t *udm, phys_addr_t addr)
+{
+	udm->rt_cfg.hw_prod_addr = addr;
+}
+
+int
+ark_udm_is_flushed(struct ark_udm_t *udm)
+{
+	return (udm->cfg.stop_flushed & 0x01) != 0;
+}
+
+uint64_t
+ark_udm_dropped(struct ark_udm_t *udm)
+{
+	return udm->qstats.q_pkt_drop;
+}
+
+uint64_t
+ark_udm_bytes(struct ark_udm_t *udm)
+{
+	return udm->qstats.q_byte_count;
+}
+
+uint64_t
+ark_udm_packets(struct ark_udm_t *udm)
+{
+	return udm->qstats.q_ff_packet_count;
+}
+
+void
+ark_udm_dump_stats(struct ark_udm_t *udm, const char *msg)
+{
+	ARK_DEBUG_STATS("ARKP UDM Stats: %s"
+			ARK_SU64 ARK_SU64 ARK_SU64 ARK_SU64 ARK_SU64 "\n",
+			msg,
+			"Pkts Received", udm->stats.rx_packet_count,
+			"Pkts Finalized", udm->stats.rx_sent_packets,
+			"Pkts Dropped", udm->tlp.pkt_drop,
+			"Bytes Count", udm->stats.rx_byte_count,
+			"MBuf Count", udm->stats.rx_mbuf_count);
+}
+
+void
+ark_udm_dump_queue_stats(struct ark_udm_t *udm, const char *msg, uint16_t qid)
+{
+	ARK_DEBUG_STATS("ARKP UDM Queue %3u Stats: %s"
+			ARK_SU64 ARK_SU64
+			ARK_SU64 ARK_SU64
+			ARK_SU64 "\n",
+			qid, msg,
+			"Pkts Received", udm->qstats.q_packet_count,
+			"Pkts Finalized", udm->qstats.q_ff_packet_count,
+			"Pkts Dropped", udm->qstats.q_pkt_drop,
+			"Bytes Count", udm->qstats.q_byte_count,
+			"MBuf Count", udm->qstats.q_mbuf_count);
+}
+
+void
+ark_udm_dump(struct ark_udm_t *udm, const char *msg)
+{
+	ARK_DEBUG_TRACE("ARKP UDM Dump: %s Stopped: %d\n", msg,
+			udm->cfg.stop_flushed);
+}
+
+void
+ark_udm_dump_setup(struct ark_udm_t *udm, uint16_t q_id)
+{
+	ARK_DEBUG_TRACE("UDM Setup Q: %u"
+			ARK_SU64X ARK_SU32 "\n",
+			q_id,
+			"hw_prod_addr", udm->rt_cfg.hw_prod_addr,
+			"prod_idx", udm->rt_cfg.prod_idx);
+}
+
+void
+ark_udm_dump_perf(struct ark_udm_t *udm, const char *msg)
+{
+	struct ark_udm_pcibp_t *bp = &udm->pcibp;
+
+	ARK_DEBUG_STATS("ARKP UDM Performance %s"
+			ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32
+			"\n",
+			msg,
+			"PCI Empty", bp->pci_empty,
+			"PCI Q1", bp->pci_q1,
+			"PCI Q2", bp->pci_q2,
+			"PCI Q3", bp->pci_q3,
+			"PCI Q4", bp->pci_q4,
+			"PCI Full", bp->pci_full);
+}
+
+void
+ark_udm_queue_stats_reset(struct ark_udm_t *udm)
+{
+	udm->qstats.q_byte_count = 1;
+}
+
+void
+ark_udm_queue_enable(struct ark_udm_t *udm, int enable)
+{
+	udm->qstats.q_enable = enable ? 1 : 0;
+}
diff --git a/drivers/net/ark/ark_udm.h b/drivers/net/ark/ark_udm.h
new file mode 100644
index 0000000..a2df891
--- /dev/null
+++ b/drivers/net/ark/ark_udm.h
@@ -0,0 +1,184 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_UDM_H_
+#define _ARK_UDM_H_
+
+#include <stdint.h>
+
+#include <rte_memory.h>
+
+/*
+ * UDM hardware structures
+ */
+
+struct ark_rx_meta {
+	uint64_t timestamp;
+	uint64_t user_data;
+	uint8_t port;
+	uint8_t dst_queue;
+	uint16_t pkt_len;
+};
+
+
+#define ARK_RX_WRITE_TIME_NS 2500
+#define ARK_UDM_SETUP 0
+#define ARK_UDM_CONST 0xbACECACE
+struct ark_udm_setup_t {
+	uint32_t r0;
+	uint32_t r4;
+	volatile uint32_t cycle_count;
+	uint32_t const0;
+};
+
+#define ARK_UDM_CFG 0x010
+struct ark_udm_cfg_t {
+	volatile uint32_t stop_flushed;	/* RO */
+	volatile uint32_t command;
+	uint32_t dataroom;
+	uint32_t headroom;
+};
+
+typedef enum {
+	ARK_UDM_START = 0x1,
+	ARK_UDM_STOP = 0x2,
+	ARK_UDM_RESET = 0x3
+} ark_udm_commands;
+
+#define ARK_UDM_STATS 0x020
+struct ark_udm_stats_t {
+	volatile uint64_t rx_byte_count;
+	volatile uint64_t rx_packet_count;
+	volatile uint64_t rx_mbuf_count;
+	volatile uint64_t rx_sent_packets;
+};
+
+#define ARK_UDM_PQ 0x040
+struct ark_udm_queue_stats_t {
+	volatile uint64_t q_byte_count;
+	volatile uint64_t q_packet_count;	/* includes drops */
+	volatile uint64_t q_mbuf_count;
+	volatile uint64_t q_ff_packet_count;
+	volatile uint64_t q_pkt_drop;
+	uint32_t q_enable;
+};
+
+#define ARK_UDM_TLP 0x0070
+struct ark_udm_tlp_t {
+	volatile uint64_t pkt_drop;	/* global */
+	volatile uint32_t tlp_q1;
+	volatile uint32_t tlp_q2;
+	volatile uint32_t tlp_q3;
+	volatile uint32_t tlp_q4;
+	volatile uint32_t tlp_full;
+};
+
+#define ARK_UDM_PCIBP 0x00a0
+struct ark_udm_pcibp_t {
+	volatile uint32_t pci_clear;
+	volatile uint32_t pci_empty;
+	volatile uint32_t pci_q1;
+	volatile uint32_t pci_q2;
+	volatile uint32_t pci_q3;
+	volatile uint32_t pci_q4;
+	volatile uint32_t pci_full;
+};
+
+#define ARK_UDM_TLP_PS 0x00bc
+struct ark_udm_tlp_ps_t {
+	volatile uint32_t tlp_clear;
+	volatile uint32_t tlp_ps_min;
+	volatile uint32_t tlp_ps_max;
+	volatile uint32_t tlp_full_ps_min;
+	volatile uint32_t tlp_full_ps_max;
+	volatile uint32_t tlp_dw_ps_min;
+	volatile uint32_t tlp_dw_ps_max;
+	volatile uint32_t tlp_pldw_ps_min;
+	volatile uint32_t tlp_pldw_ps_max;
+};
+
+#define ARK_UDM_RT_CFG 0x00e0
+struct ark_udm_rt_cfg_t {
+	phys_addr_t hw_prod_addr;
+	uint32_t write_interval;	/* 4ns cycles */
+	volatile uint32_t prod_idx;	/* RO */
+};
+
+/*  Consolidated structure */
+struct ark_udm_t {
+	struct ark_udm_setup_t setup;
+	struct ark_udm_cfg_t cfg;
+	struct ark_udm_stats_t stats;
+	struct ark_udm_queue_stats_t qstats;
+	uint8_t reserved1[(ARK_UDM_TLP - ARK_UDM_PQ) -
+					  sizeof(struct ark_udm_queue_stats_t)];
+	struct ark_udm_tlp_t tlp;
+	uint8_t reserved2[(ARK_UDM_PCIBP - ARK_UDM_TLP) -
+					  sizeof(struct ark_udm_tlp_t)];
+	struct ark_udm_pcibp_t pcibp;
+	struct ark_udm_tlp_ps_t tlp_ps;
+	struct ark_udm_rt_cfg_t rt_cfg;
+	int8_t reserved3[(0x100 - ARK_UDM_RT_CFG) -
+					  sizeof(struct ark_udm_rt_cfg_t)];
+};
+
+#define ARK_UDM_EXPECT_SIZE (0x00fc + 4)
+#define ARK_UDM_QOFFSET ARK_UDM_EXPECT_SIZE
+
+int ark_udm_verify(struct ark_udm_t *udm);
+int ark_udm_stop(struct ark_udm_t *udm, int wait);
+void ark_udm_start(struct ark_udm_t *udm);
+int ark_udm_reset(struct ark_udm_t *udm);
+void ark_udm_configure(struct ark_udm_t *udm,
+					   uint32_t headroom,
+					   uint32_t dataroom,
+					   uint32_t write_interval_ns);
+void ark_udm_write_addr(struct ark_udm_t *udm, phys_addr_t addr);
+void ark_udm_stats_reset(struct ark_udm_t *udm);
+void ark_udm_dump_stats(struct ark_udm_t *udm, const char *msg);
+void ark_udm_dump_queue_stats(struct ark_udm_t *udm, const char *msg,
+							  uint16_t qid);
+void ark_udm_dump(struct ark_udm_t *udm, const char *msg);
+void ark_udm_dump_perf(struct ark_udm_t *udm, const char *msg);
+void ark_udm_dump_setup(struct ark_udm_t *udm, uint16_t q_id);
+int ark_udm_is_flushed(struct ark_udm_t *udm);
+
+/* Per queue data */
+uint64_t ark_udm_dropped(struct ark_udm_t *udm);
+uint64_t ark_udm_bytes(struct ark_udm_t *udm);
+uint64_t ark_udm_packets(struct ark_udm_t *udm);
+
+void ark_udm_queue_stats_reset(struct ark_udm_t *udm);
+void ark_udm_queue_enable(struct ark_udm_t *udm, int enable);
+
+#endif
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 4/7] net/ark: HW API part 3 of 3
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3 Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 3/7] net/ark: HW API part 2 " Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 5/7] net/ark: Packet TX support initial version Ed Czeck
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

Provide C-level interface for Arkville's internal HW resources
pktgen and pktchk modules

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/Makefile      |   2 +
 drivers/net/ark/ark_pktchkr.c | 472 ++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_pktchkr.h | 114 ++++++++++
 drivers/net/ark/ark_pktgen.c  | 491 ++++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_pktgen.h  | 106 +++++++++
 5 files changed, 1185 insertions(+)
 create mode 100644 drivers/net/ark/ark_pktchkr.c
 create mode 100644 drivers/net/ark/ark_pktchkr.h
 create mode 100644 drivers/net/ark/ark_pktgen.c
 create mode 100644 drivers/net/ark/ark_pktgen.h

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index 529d92c..4a2c41a 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -48,6 +48,8 @@ LIBABIVER := 1
 #
 
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktgen.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktchkr.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktdir.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_mpu.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ddm.c
diff --git a/drivers/net/ark/ark_pktchkr.c b/drivers/net/ark/ark_pktchkr.c
new file mode 100644
index 0000000..8d0aaef
--- /dev/null
+++ b/drivers/net/ark/ark_pktchkr.c
@@ -0,0 +1,472 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <getopt.h>
+#include <sys/time.h>
+#include <locale.h>
+#include <unistd.h>
+
+#include "ark_pktchkr.h"
+#include "ark_debug.h"
+
+static int set_arg(char *arg, char *val);
+static int ark_pktchkr_is_gen_forever(ark_pkt_chkr_t handle);
+
+#define ARK_MAX_STR_LEN 64
+union OPTV {
+	int INT;
+	int BOOL;
+	uint64_t LONG;
+	char STR[ARK_MAX_STR_LEN];
+};
+
+enum OPTYPE {
+	OTINT,
+	OTLONG,
+	OTBOOL,
+	OTSTRING
+};
+
+struct OPTIONS {
+	char opt[ARK_MAX_STR_LEN];
+	enum OPTYPE t;
+	union OPTV v;
+};
+
+static struct OPTIONS toptions[] = {
+	{{"configure"}, OTBOOL, {1} },
+	{{"port"}, OTINT, {0} },
+	{{"mac-dump"}, OTBOOL, {0} },
+	{{"dg-mode"}, OTBOOL, {1} },
+	{{"run"}, OTBOOL, {0} },
+	{{"stop"}, OTBOOL, {0} },
+	{{"dump"}, OTBOOL, {0} },
+	{{"en_resync"}, OTBOOL, {0} },
+	{{"tuser_err_val"}, OTINT, {1} },
+	{{"gen_forever"}, OTBOOL, {0} },
+	{{"en_slaved_start"}, OTBOOL, {0} },
+	{{"vary_length"}, OTBOOL, {0} },
+	{{"incr_payload"}, OTINT, {0} },
+	{{"incr_first_byte"}, OTBOOL, {0} },
+	{{"ins_seq_num"}, OTBOOL, {0} },
+	{{"ins_time_stamp"}, OTBOOL, {1} },
+	{{"ins_udp_hdr"}, OTBOOL, {0} },
+	{{"num_pkts"}, OTLONG, .v.LONG = 10000000000000L},
+	{{"payload_byte"}, OTINT, {0x55} },
+	{{"pkt_spacing"}, OTINT, {60} },
+	{{"pkt_size_min"}, OTINT, {2005} },
+	{{"pkt_size_max"}, OTINT, {1514} },
+	{{"pkt_size_incr"}, OTINT, {1} },
+	{{"eth_type"}, OTINT, {0x0800} },
+	{{"src_mac_addr"}, OTLONG, .v.LONG = 0xdC3cF6425060L},
+	{{"dst_mac_addr"}, OTLONG, .v.LONG = 0x112233445566L},
+	{{"hdr_dW0"}, OTINT, {0x0016e319} },
+	{{"hdr_dW1"}, OTINT, {0x27150004} },
+	{{"hdr_dW2"}, OTINT, {0x76967bda} },
+	{{"hdr_dW3"}, OTINT, {0x08004500} },
+	{{"hdr_dW4"}, OTINT, {0x005276ed} },
+	{{"hdr_dW5"}, OTINT, {0x40004006} },
+	{{"hdr_dW6"}, OTINT, {0x56cfc0a8} },
+	{{"start_offset"}, OTINT, {0} },
+	{{"dst_ip"}, OTSTRING, .v.STR = "169.254.10.240"},
+	{{"dst_port"}, OTINT, {65536} },
+	{{"src_port"}, OTINT, {65536} },
+};
+
+ark_pkt_chkr_t
+ark_pktchkr_init(void *addr, int ord, int l2_mode)
+{
+	struct ark_pkt_chkr_inst *inst =
+		rte_malloc("ark_pkt_chkr_inst",
+			   sizeof(struct ark_pkt_chkr_inst), 0);
+	inst->sregs = (struct ark_pkt_chkr_stat_regs *)addr;
+	inst->cregs =
+		(struct ark_pkt_chkr_ctl_regs *)(((uint8_t *)addr) + 0x100);
+	inst->ordinal = ord;
+	inst->l2_mode = l2_mode;
+	return inst;
+}
+
+void
+ark_pktchkr_uninit(ark_pkt_chkr_t handle)
+{
+	rte_free(handle);
+}
+
+void
+ark_pktchkr_run(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->sregs->pkt_start_stop = 0;
+	inst->sregs->pkt_start_stop = 0x1;
+}
+
+int
+ark_pktchkr_stopped(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+	uint32_t r = inst->sregs->pkt_start_stop;
+
+	return (((r >> 16) & 1) == 1);
+}
+
+void
+ark_pktchkr_stop(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+	int wait_cycle = 10;
+
+	inst->sregs->pkt_start_stop = 0;
+	while (!ark_pktchkr_stopped(handle) && (wait_cycle > 0)) {
+		usleep(1000);
+		wait_cycle--;
+		ARK_DEBUG_TRACE("Waiting for pktchk %d to stop...\n",
+				inst->ordinal);
+	}
+	ARK_DEBUG_TRACE("pktchk %d stopped.\n", inst->ordinal);
+}
+
+int
+ark_pktchkr_is_running(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+	uint32_t r = inst->sregs->pkt_start_stop;
+
+	return ((r & 1) == 1);
+}
+
+static void
+ark_pktchkr_set_pkt_ctrl(ark_pkt_chkr_t handle,
+			 uint32_t gen_forever,
+			 uint32_t vary_length,
+			 uint32_t incr_payload,
+			 uint32_t incr_first_byte,
+			 uint32_t ins_seq_num,
+			 uint32_t ins_udp_hdr,
+			 uint32_t en_resync,
+			 uint32_t tuser_err_val,
+			 uint32_t ins_time_stamp)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+	uint32_t r = (tuser_err_val << 16) | (en_resync << 0);
+
+	inst->sregs->pkt_ctrl = r;
+	if (!inst->l2_mode)
+		ins_udp_hdr = 0;
+	r = ((gen_forever << 24) |
+	     (vary_length << 16) |
+	     (incr_payload << 12) |
+	     (incr_first_byte << 8) |
+	     (ins_time_stamp << 5) |
+	     (ins_seq_num << 4) |
+	     ins_udp_hdr);
+	inst->cregs->pkt_ctrl = r;
+}
+
+static
+int
+ark_pktchkr_is_gen_forever(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+	uint32_t r = inst->cregs->pkt_ctrl;
+
+	return (((r >> 24) & 1) == 1);
+}
+
+int
+ark_pktchkr_wait_done(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	if (ark_pktchkr_is_gen_forever(handle)) {
+		ARK_DEBUG_TRACE("Error: wait_done will not terminate"
+				" because gen_forever=1\n");
+		return -1;
+	}
+	int wait_cycle = 10;
+
+	while (!ark_pktchkr_stopped(handle) && (wait_cycle > 0)) {
+		usleep(1000);
+		wait_cycle--;
+		ARK_DEBUG_TRACE("Waiting for packet checker %d's"
+				" internal pktgen to finish sending...\n",
+				inst->ordinal);
+		ARK_DEBUG_TRACE("pktchk %d's pktgen done.\n", inst->ordinal);
+	}
+	return 0;
+}
+
+int
+ark_pktchkr_get_pkts_sent(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	return inst->cregs->pkts_sent;
+}
+
+void
+ark_pktchkr_set_payload_byte(ark_pkt_chkr_t handle, uint32_t b)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->pkt_payload = b;
+}
+
+void
+ark_pktchkr_set_pkt_size_min(ark_pkt_chkr_t handle, uint32_t x)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->pkt_size_min = x;
+}
+
+void
+ark_pktchkr_set_pkt_size_max(ark_pkt_chkr_t handle, uint32_t x)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->pkt_size_max = x;
+}
+
+void
+ark_pktchkr_set_pkt_size_incr(ark_pkt_chkr_t handle, uint32_t x)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->pkt_size_incr = x;
+}
+
+void
+ark_pktchkr_set_num_pkts(ark_pkt_chkr_t handle, uint32_t x)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->num_pkts = x;
+}
+
+void
+ark_pktchkr_set_src_mac_addr(ark_pkt_chkr_t handle, uint64_t mac_addr)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->src_mac_addr_h = (mac_addr >> 32) & 0xffff;
+	inst->cregs->src_mac_addr_l = mac_addr & 0xffffffff;
+}
+
+void
+ark_pktchkr_set_dst_mac_addr(ark_pkt_chkr_t handle, uint64_t mac_addr)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->dst_mac_addr_h = (mac_addr >> 32) & 0xffff;
+	inst->cregs->dst_mac_addr_l = mac_addr & 0xffffffff;
+}
+
+void
+ark_pktchkr_set_eth_type(ark_pkt_chkr_t handle, uint32_t x)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	inst->cregs->eth_type = x;
+}
+
+void
+ark_pktchkr_set_hdr_dW(ark_pkt_chkr_t handle, uint32_t *hdr)
+{
+	uint32_t i;
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	for (i = 0; i < 7; i++)
+		inst->cregs->hdr_dw[i] = hdr[i];
+}
+
+void
+ark_pktchkr_dump_stats(ark_pkt_chkr_t handle)
+{
+	struct ark_pkt_chkr_inst *inst = (struct ark_pkt_chkr_inst *)handle;
+
+	PMD_DRV_LOG(ERR, "pkts_rcvd      = (%'u)\n",
+		    inst->sregs->pkts_rcvd);
+	PMD_DRV_LOG(ERR, "bytes_rcvd     = (%'" PRIu64 ")\n",
+		    inst->sregs->bytes_rcvd);
+	PMD_DRV_LOG(ERR, "pkts_ok        = (%'u)\n",
+		    inst->sregs->pkts_ok);
+	PMD_DRV_LOG(ERR, "pkts_mismatch  = (%'u)\n",
+		    inst->sregs->pkts_mismatch);
+	PMD_DRV_LOG(ERR, "pkts_err       = (%'u)\n",
+		    inst->sregs->pkts_err);
+	PMD_DRV_LOG(ERR, "first_mismatch = (%'u)\n",
+		    inst->sregs->first_mismatch);
+	PMD_DRV_LOG(ERR, "resync_events  = (%'u)\n",
+		    inst->sregs->resync_events);
+	PMD_DRV_LOG(ERR, "pkts_missing   = (%'u)\n",
+		    inst->sregs->pkts_missing);
+	PMD_DRV_LOG(ERR, "min_latency    = (%'u)\n",
+		    inst->sregs->min_latency);
+	PMD_DRV_LOG(ERR, "max_latency    = (%'u)\n",
+		    inst->sregs->max_latency);
+}
+
+static struct OPTIONS *
+options(const char *id)
+{
+	unsigned int i;
+
+	for (i = 0; i < sizeof(toptions) / sizeof(struct OPTIONS); i++) {
+		if (strcmp(id, toptions[i].opt) == 0)
+			return &toptions[i];
+	}
+	PMD_DRV_LOG(ERR,
+		    "pktchkr: Could not find requested option!, option = %s\n",
+		    id);
+	return NULL;
+}
+
+static int
+set_arg(char *arg, char *val)
+{
+	struct OPTIONS *o = options(arg);
+
+	if (o) {
+		switch (o->t) {
+		case OTINT:
+		case OTBOOL:
+			o->v.INT = atoi(val);
+			break;
+		case OTLONG:
+			o->v.INT = atoll(val);
+			break;
+		case OTSTRING:
+			strncpy(o->v.STR, val, ARK_MAX_STR_LEN);
+			break;
+		}
+		return 1;
+	}
+	return 0;
+}
+
+/******
+ * Arg format = "opt0=v,opt_n=v ..."
+ ******/
+void
+ark_pktchkr_parse(char *args)
+{
+	char *argv, *v;
+	const char toks[] = "=\n\t\v\f \r";
+	argv = strtok(args, toks);
+	v = strtok(NULL, toks);
+	set_arg(argv, v);
+	while (argv && v) {
+		argv = strtok(NULL, toks);
+		v = strtok(NULL, toks);
+		if (argv && v)
+			set_arg(argv, v);
+	}
+}
+
+static int32_t parse_ipv4_string(char const *ip_address);
+static int32_t
+parse_ipv4_string(char const *ip_address)
+{
+	unsigned int ip[4];
+
+	if (sscanf(ip_address, "%u.%u.%u.%u",
+		   &ip[0], &ip[1], &ip[2], &ip[3]) != 4)
+		return 0;
+	return ip[3] + ip[2] * 0x100 + ip[1] * 0x10000ul + ip[0] * 0x1000000ul;
+}
+
+void
+ark_pktchkr_setup(ark_pkt_chkr_t handle)
+{
+	uint32_t hdr[7];
+	int32_t dst_ip = parse_ipv4_string(options("dst_ip")->v.STR);
+
+	if (!options("stop")->v.BOOL && options("configure")->v.BOOL) {
+		ark_pktchkr_set_payload_byte(handle,
+					     options("payload_byte")->v.INT);
+		ark_pktchkr_set_src_mac_addr(handle,
+					     options("src_mac_addr")->v.INT);
+		ark_pktchkr_set_dst_mac_addr(handle,
+					     options("dst_mac_addr")->v.LONG);
+
+		ark_pktchkr_set_eth_type(handle,
+					 options("eth_type")->v.INT);
+		if (options("dg-mode")->v.BOOL) {
+			hdr[0] = options("hdr_dW0")->v.INT;
+			hdr[1] = options("hdr_dW1")->v.INT;
+			hdr[2] = options("hdr_dW2")->v.INT;
+			hdr[3] = options("hdr_dW3")->v.INT;
+			hdr[4] = options("hdr_dW4")->v.INT;
+			hdr[5] = options("hdr_dW5")->v.INT;
+			hdr[6] = options("hdr_dW6")->v.INT;
+		} else {
+			hdr[0] = dst_ip;
+			hdr[1] = options("dst_port")->v.INT;
+			hdr[2] = options("src_port")->v.INT;
+			hdr[3] = 0;
+			hdr[4] = 0;
+			hdr[5] = 0;
+			hdr[6] = 0;
+		}
+		ark_pktchkr_set_hdr_dW(handle, hdr);
+		ark_pktchkr_set_num_pkts(handle,
+					 options("num_pkts")->v.INT);
+		ark_pktchkr_set_pkt_size_min(handle,
+					     options("pkt_size_min")->v.INT);
+		ark_pktchkr_set_pkt_size_max(handle,
+					     options("pkt_size_max")->v.INT);
+		ark_pktchkr_set_pkt_size_incr(handle,
+					      options("pkt_size_incr")->v.INT);
+		ark_pktchkr_set_pkt_ctrl(handle,
+					 options("gen_forever")->v.BOOL,
+					 options("vary_length")->v.BOOL,
+					 options("incr_payload")->v.BOOL,
+					 options("incr_first_byte")->v.BOOL,
+					 options("ins_seq_num")->v.INT,
+					 options("ins_udp_hdr")->v.BOOL,
+					 options("en_resync")->v.BOOL,
+					 options("tuser_err_val")->v.INT,
+					 options("ins_time_stamp")->v.INT);
+	}
+
+	if (options("stop")->v.BOOL)
+		ark_pktchkr_stop(handle);
+
+	if (options("run")->v.BOOL) {
+		ARK_DEBUG_TRACE("Starting packet checker on port %d\n",
+				options("port")->v.INT);
+		ark_pktchkr_run(handle);
+	}
+}
diff --git a/drivers/net/ark/ark_pktchkr.h b/drivers/net/ark/ark_pktchkr.h
new file mode 100644
index 0000000..bc078fd
--- /dev/null
+++ b/drivers/net/ark/ark_pktchkr.h
@@ -0,0 +1,114 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_PKTCHKR_H_
+#define _ARK_PKTCHKR_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+
+#include <rte_eal.h>
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#define ARK_PKTCHKR_BASE_ADR  0x90000
+
+typedef void *ark_pkt_chkr_t;
+
+struct ark_pkt_chkr_stat_regs {
+	uint32_t r0;
+	uint32_t pkt_start_stop;
+	uint32_t pkt_ctrl;
+	uint32_t pkts_rcvd;
+	uint64_t bytes_rcvd;
+	uint32_t pkts_ok;
+	uint32_t pkts_mismatch;
+	uint32_t pkts_err;
+	uint32_t first_mismatch;
+	uint32_t resync_events;
+	uint32_t pkts_missing;
+	uint32_t min_latency;
+	uint32_t max_latency;
+} __attribute__ ((packed));
+
+struct ark_pkt_chkr_ctl_regs {
+	uint32_t pkt_ctrl;
+	uint32_t pkt_payload;
+	uint32_t pkt_size_min;
+	uint32_t pkt_size_max;
+	uint32_t pkt_size_incr;
+	uint32_t num_pkts;
+	uint32_t pkts_sent;
+	uint32_t src_mac_addr_l;
+	uint32_t src_mac_addr_h;
+	uint32_t dst_mac_addr_l;
+	uint32_t dst_mac_addr_h;
+	uint32_t eth_type;
+	uint32_t hdr_dw[7];
+} __attribute__ ((packed));
+
+struct ark_pkt_chkr_inst {
+	struct rte_eth_dev_info *dev_info;
+	volatile struct ark_pkt_chkr_stat_regs *sregs;
+	volatile struct ark_pkt_chkr_ctl_regs *cregs;
+	int l2_mode;
+	int ordinal;
+};
+
+/*  packet checker functions */
+ark_pkt_chkr_t ark_pktchkr_init(void *addr, int ord, int l2_mode);
+void ark_pktchkr_uninit(ark_pkt_chkr_t handle);
+void ark_pktchkr_run(ark_pkt_chkr_t handle);
+int ark_pktchkr_stopped(ark_pkt_chkr_t handle);
+void ark_pktchkr_stop(ark_pkt_chkr_t handle);
+int ark_pktchkr_is_running(ark_pkt_chkr_t handle);
+int ark_pktchkr_get_pkts_sent(ark_pkt_chkr_t handle);
+void ark_pktchkr_set_payload_byte(ark_pkt_chkr_t handle, uint32_t b);
+void ark_pktchkr_set_pkt_size_min(ark_pkt_chkr_t handle, uint32_t x);
+void ark_pktchkr_set_pkt_size_max(ark_pkt_chkr_t handle, uint32_t x);
+void ark_pktchkr_set_pkt_size_incr(ark_pkt_chkr_t handle, uint32_t x);
+void ark_pktchkr_set_num_pkts(ark_pkt_chkr_t handle, uint32_t x);
+void ark_pktchkr_set_src_mac_addr(ark_pkt_chkr_t handle, uint64_t mac_addr);
+void ark_pktchkr_set_dst_mac_addr(ark_pkt_chkr_t handle, uint64_t mac_addr);
+void ark_pktchkr_set_eth_type(ark_pkt_chkr_t handle, uint32_t x);
+void ark_pktchkr_set_hdr_dW(ark_pkt_chkr_t handle, uint32_t *hdr);
+void ark_pktchkr_parse(char *args);
+void ark_pktchkr_setup(ark_pkt_chkr_t handle);
+void ark_pktchkr_dump_stats(ark_pkt_chkr_t handle);
+int ark_pktchkr_wait_done(ark_pkt_chkr_t handle);
+
+#endif
diff --git a/drivers/net/ark/ark_pktgen.c b/drivers/net/ark/ark_pktgen.c
new file mode 100644
index 0000000..ba2153b
--- /dev/null
+++ b/drivers/net/ark/ark_pktgen.c
@@ -0,0 +1,491 @@
+/*
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <getopt.h>
+#include <sys/time.h>
+#include <locale.h>
+#include <unistd.h>
+
+#include "ark_pktgen.h"
+#include "ark_debug.h"
+
+#define ARK_MAX_STR_LEN 64
+union OPTV {
+	int INT;
+	int BOOL;
+	uint64_t LONG;
+	char STR[ARK_MAX_STR_LEN];
+};
+
+enum OPTYPE {
+	OTINT,
+	OTLONG,
+	OTBOOL,
+	OTSTRING
+};
+
+struct OPTIONS {
+	char opt[ARK_MAX_STR_LEN];
+	enum OPTYPE t;
+	union OPTV v;
+};
+
+static struct OPTIONS toptions[] = {
+	{{"configure"}, OTBOOL, {1} },
+	{{"dg-mode"}, OTBOOL, {1} },
+	{{"run"}, OTBOOL, {0} },
+	{{"pause"}, OTBOOL, {0} },
+	{{"reset"}, OTBOOL, {0} },
+	{{"dump"}, OTBOOL, {0} },
+	{{"gen_forever"}, OTBOOL, {0} },
+	{{"en_slaved_start"}, OTBOOL, {0} },
+	{{"vary_length"}, OTBOOL, {0} },
+	{{"incr_payload"}, OTBOOL, {0} },
+	{{"incr_first_byte"}, OTBOOL, {0} },
+	{{"ins_seq_num"}, OTBOOL, {0} },
+	{{"ins_time_stamp"}, OTBOOL, {1} },
+	{{"ins_udp_hdr"}, OTBOOL, {0} },
+	{{"num_pkts"}, OTLONG, .v.LONG = 100000000},
+	{{"payload_byte"}, OTINT, {0x55} },
+	{{"pkt_spacing"}, OTINT, {130} },
+	{{"pkt_size_min"}, OTINT, {2006} },
+	{{"pkt_size_max"}, OTINT, {1514} },
+	{{"pkt_size_incr"}, OTINT, {1} },
+	{{"eth_type"}, OTINT, {0x0800} },
+	{{"src_mac_addr"}, OTLONG, .v.LONG = 0xdC3cF6425060L},
+	{{"dst_mac_addr"}, OTLONG, .v.LONG = 0x112233445566L},
+	{{"hdr_dW0"}, OTINT, {0x0016e319} },
+	{{"hdr_dW1"}, OTINT, {0x27150004} },
+	{{"hdr_dW2"}, OTINT, {0x76967bda} },
+	{{"hdr_dW3"}, OTINT, {0x08004500} },
+	{{"hdr_dW4"}, OTINT, {0x005276ed} },
+	{{"hdr_dW5"}, OTINT, {0x40004006} },
+	{{"hdr_dW6"}, OTINT, {0x56cfc0a8} },
+	{{"start_offset"}, OTINT, {0} },
+	{{"bytes_per_cycle"}, OTINT, {10} },
+	{{"shaping"}, OTBOOL, {0} },
+	{{"dst_ip"}, OTSTRING, .v.STR = "169.254.10.240"},
+	{{"dst_port"}, OTINT, {65536} },
+	{{"src_port"}, OTINT, {65536} },
+};
+
+ark_pkt_gen_t
+ark_pktgen_init(void *adr, int ord, int l2_mode)
+{
+	struct ark_pkt_gen_inst *inst =
+		rte_malloc("ark_pkt_gen_inst_pMD",
+			   sizeof(struct ark_pkt_gen_inst), 0);
+	inst->regs = (struct ark_pkt_gen_regs *)adr;
+	inst->ordinal = ord;
+	inst->l2_mode = l2_mode;
+	return inst;
+}
+
+void
+ark_pktgen_uninit(ark_pkt_gen_t handle)
+{
+	rte_free(handle);
+}
+
+void
+ark_pktgen_run(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	inst->regs->pkt_start_stop = 1;
+}
+
+uint32_t
+ark_pktgen_paused(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	uint32_t r = inst->regs->pkt_start_stop;
+
+	return (((r >> 16) & 1) == 1);
+}
+
+void
+ark_pktgen_pause(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	int cnt = 0;
+
+	inst->regs->pkt_start_stop = 0;
+
+	while (!ark_pktgen_paused(handle)) {
+		usleep(1000);
+		if (cnt++ > 100) {
+			PMD_DRV_LOG(ERR, "pktgen %d failed to pause.\n",
+				    inst->ordinal);
+			break;
+		}
+	}
+	ARK_DEBUG_TRACE("pktgen %d paused.\n", inst->ordinal);
+}
+
+void
+ark_pktgen_reset(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	if (!ark_pktgen_is_running(handle) &&
+	    !ark_pktgen_paused(handle)) {
+		ARK_DEBUG_TRACE
+			("pktgen %d is not running and is not paused. No need to reset.\n",
+			 inst->ordinal);
+		return;
+	}
+
+	if (ark_pktgen_is_running(handle) &&
+	    !ark_pktgen_paused(handle)) {
+		ARK_DEBUG_TRACE("pktgen %d is not paused. Pausing first.\n",
+				inst->ordinal);
+		ark_pktgen_pause(handle);
+	}
+
+	ARK_DEBUG_TRACE("Resetting pktgen %d.\n", inst->ordinal);
+	inst->regs->pkt_start_stop = (1 << 8);
+}
+
+uint32_t
+ark_pktgen_tx_done(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	uint32_t r = inst->regs->pkt_start_stop;
+
+	return (((r >> 24) & 1) == 1);
+}
+
+uint32_t
+ark_pktgen_is_running(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	uint32_t r = inst->regs->pkt_start_stop;
+
+	return ((r & 1) == 1);
+}
+
+uint32_t
+ark_pktgen_is_gen_forever(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	uint32_t r = inst->regs->pkt_ctrl;
+
+	return (((r >> 24) & 1) == 1);
+}
+
+void
+ark_pktgen_wait_done(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	if (ark_pktgen_is_gen_forever(handle))
+		PMD_DRV_LOG(ERR, "wait_done will not terminate because gen_forever=1\n");
+	int wait_cycle = 10;
+
+	while (!ark_pktgen_tx_done(handle) && (wait_cycle > 0)) {
+		usleep(1000);
+		wait_cycle--;
+		ARK_DEBUG_TRACE("Waiting for pktgen %d to finish sending...\n",
+				inst->ordinal);
+	}
+	ARK_DEBUG_TRACE("pktgen %d done.\n", inst->ordinal);
+}
+
+uint32_t
+ark_pktgen_get_pkts_sent(ark_pkt_gen_t handle)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	return inst->regs->pkts_sent;
+}
+
+void
+ark_pktgen_set_payload_byte(ark_pkt_gen_t handle, uint32_t b)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->pkt_payload = b;
+}
+
+void
+ark_pktgen_set_pkt_spacing(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->pkt_spacing = x;
+}
+
+void
+ark_pktgen_set_pkt_size_min(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->pkt_size_min = x;
+}
+
+void
+ark_pktgen_set_pkt_size_max(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->pkt_size_max = x;
+}
+
+void
+ark_pktgen_set_pkt_size_incr(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->pkt_size_incr = x;
+}
+
+void
+ark_pktgen_set_num_pkts(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->num_pkts = x;
+}
+
+void
+ark_pktgen_set_src_mac_addr(ark_pkt_gen_t handle, uint64_t mac_addr)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->src_mac_addr_h = (mac_addr >> 32) & 0xffff;
+	inst->regs->src_mac_addr_l = mac_addr & 0xffffffff;
+}
+
+void
+ark_pktgen_set_dst_mac_addr(ark_pkt_gen_t handle, uint64_t mac_addr)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->dst_mac_addr_h = (mac_addr >> 32) & 0xffff;
+	inst->regs->dst_mac_addr_l = mac_addr & 0xffffffff;
+}
+
+void
+ark_pktgen_set_eth_type(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+	inst->regs->eth_type = x;
+}
+
+void
+ark_pktgen_set_hdr_dW(ark_pkt_gen_t handle, uint32_t *hdr)
+{
+	uint32_t i;
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	for (i = 0; i < 7; i++)
+		inst->regs->hdr_dw[i] = hdr[i];
+}
+
+void
+ark_pktgen_set_start_offset(ark_pkt_gen_t handle, uint32_t x)
+{
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	inst->regs->start_offset = x;
+}
+
+static struct OPTIONS *
+options(const char *id)
+{
+	unsigned int i;
+
+	for (i = 0; i < sizeof(toptions) / sizeof(struct OPTIONS); i++) {
+		if (strcmp(id, toptions[i].opt) == 0)
+			return &toptions[i];
+	}
+
+	PMD_DRV_LOG
+		(ERR,
+		 "pktgen: Could not find requested option !!, "
+		 "option = %s\n",
+		 id
+		 );
+	return NULL;
+}
+
+static int pmd_set_arg(char *arg, char *val);
+static int
+pmd_set_arg(char *arg, char *val)
+{
+	struct OPTIONS *o = options(arg);
+
+	if (o) {
+		switch (o->t) {
+		case OTINT:
+		case OTBOOL:
+			o->v.INT = atoi(val);
+			break;
+		case OTLONG:
+			o->v.INT = atoll(val);
+			break;
+		case OTSTRING:
+			strncpy(o->v.STR, val, ARK_MAX_STR_LEN);
+			break;
+		}
+		return 1;
+	}
+	return 0;
+}
+
+/******
+ * Arg format = "opt0=v,opt_n=v ..."
+ ******/
+void
+ark_pktgen_parse(char *args)
+{
+	char *argv, *v;
+	const char toks[] = " =\n\t\v\f \r";
+	argv = strtok(args, toks);
+	v = strtok(NULL, toks);
+	pmd_set_arg(argv, v);
+	while (argv && v) {
+		argv = strtok(NULL, toks);
+		v = strtok(NULL, toks);
+		if (argv && v)
+			pmd_set_arg(argv, v);
+	}
+}
+
+static int32_t parse_ipv4_string(char const *ip_address);
+static int32_t
+parse_ipv4_string(char const *ip_address)
+{
+	unsigned int ip[4];
+
+	if (sscanf(ip_address, "%u.%u.%u.%u",
+		   &ip[0], &ip[1], &ip[2], &ip[3]) != 4)
+		return 0;
+	return ip[3] + ip[2] * 0x100 + ip[1] * 0x10000ul + ip[0] * 0x1000000ul;
+}
+
+static void
+ark_pktgen_set_pkt_ctrl(ark_pkt_gen_t handle,
+			uint32_t gen_forever,
+			uint32_t en_slaved_start,
+			uint32_t vary_length,
+			uint32_t incr_payload,
+			uint32_t incr_first_byte,
+			uint32_t ins_seq_num,
+			uint32_t ins_udp_hdr,
+			uint32_t ins_time_stamp)
+{
+	uint32_t r;
+	struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)handle;
+
+	if (!inst->l2_mode)
+		ins_udp_hdr = 0;
+
+	r = ((gen_forever << 24) |
+	     (en_slaved_start << 20) |
+	     (vary_length << 16) |
+	     (incr_payload << 12) |
+	     (incr_first_byte << 8) |
+	     (ins_time_stamp << 5) |
+	     (ins_seq_num << 4) |
+	     ins_udp_hdr);
+
+	inst->regs->bytes_per_cycle = options("bytes_per_cycle")->v.INT;
+	if (options("shaping")->v.BOOL)
+		r = r | (1 << 28);	/* enable shaping */
+
+	inst->regs->pkt_ctrl = r;
+}
+
+void
+ark_pktgen_setup(ark_pkt_gen_t handle)
+{
+	uint32_t hdr[7];
+	int32_t dst_ip = parse_ipv4_string(options("dst_ip")->v.STR);
+
+	if (!options("pause")->v.BOOL &&
+	    (!options("reset")->v.BOOL &&
+	     (options("configure")->v.BOOL))) {
+		ark_pktgen_set_payload_byte(handle,
+					    options("payload_byte")->v.INT);
+		ark_pktgen_set_src_mac_addr(handle,
+					    options("src_mac_addr")->v.INT);
+		ark_pktgen_set_dst_mac_addr(handle,
+					    options("dst_mac_addr")->v.LONG);
+		ark_pktgen_set_eth_type(handle,
+					options("eth_type")->v.INT);
+
+		if (options("dg-mode")->v.BOOL) {
+			hdr[0] = options("hdr_dW0")->v.INT;
+			hdr[1] = options("hdr_dW1")->v.INT;
+			hdr[2] = options("hdr_dW2")->v.INT;
+			hdr[3] = options("hdr_dW3")->v.INT;
+			hdr[4] = options("hdr_dW4")->v.INT;
+			hdr[5] = options("hdr_dW5")->v.INT;
+			hdr[6] = options("hdr_dW6")->v.INT;
+		} else {
+			hdr[0] = dst_ip;
+			hdr[1] = options("dst_port")->v.INT;
+			hdr[2] = options("src_port")->v.INT;
+			hdr[3] = 0;
+			hdr[4] = 0;
+			hdr[5] = 0;
+			hdr[6] = 0;
+		}
+		ark_pktgen_set_hdr_dW(handle, hdr);
+		ark_pktgen_set_num_pkts(handle,
+					options("num_pkts")->v.INT);
+		ark_pktgen_set_pkt_size_min(handle,
+					    options("pkt_size_min")->v.INT);
+		ark_pktgen_set_pkt_size_max(handle,
+					    options("pkt_size_max")->v.INT);
+		ark_pktgen_set_pkt_size_incr(handle,
+					     options("pkt_size_incr")->v.INT);
+		ark_pktgen_set_pkt_spacing(handle,
+					   options("pkt_spacing")->v.INT);
+		ark_pktgen_set_start_offset(handle,
+					    options("start_offset")->v.INT);
+		ark_pktgen_set_pkt_ctrl(handle,
+					options("gen_forever")->v.BOOL,
+					options("en_slaved_start")->v.BOOL,
+					options("vary_length")->v.BOOL,
+					options("incr_payload")->v.BOOL,
+					options("incr_first_byte")->v.BOOL,
+					options("ins_seq_num")->v.INT,
+					options("ins_udp_hdr")->v.BOOL,
+					options("ins_time_stamp")->v.INT);
+	}
+
+	if (options("pause")->v.BOOL)
+		ark_pktgen_pause(handle);
+
+	if (options("reset")->v.BOOL)
+		ark_pktgen_reset(handle);
+	if (options("run")->v.BOOL) {
+		ARK_DEBUG_TRACE("Starting packet generator on port %d\n",
+				options("port")->v.INT);
+		ark_pktgen_run(handle);
+	}
+}
diff --git a/drivers/net/ark/ark_pktgen.h b/drivers/net/ark/ark_pktgen.h
new file mode 100644
index 0000000..307edf0
--- /dev/null
+++ b/drivers/net/ark/ark_pktgen.h
@@ -0,0 +1,106 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_PKTGEN_H_
+#define _ARK_PKTGEN_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+
+#include <rte_eal.h>
+
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#define ARK_PKTGEN_BASE_ADR  0x10000
+
+typedef void *ark_pkt_gen_t;
+
+struct ark_pkt_gen_regs {
+	uint32_t r0;
+	volatile uint32_t pkt_start_stop;
+	volatile uint32_t pkt_ctrl;
+	uint32_t pkt_payload;
+	uint32_t pkt_spacing;
+	uint32_t pkt_size_min;
+	uint32_t pkt_size_max;
+	uint32_t pkt_size_incr;
+	volatile uint32_t num_pkts;
+	volatile uint32_t pkts_sent;
+	uint32_t src_mac_addr_l;
+	uint32_t src_mac_addr_h;
+	uint32_t dst_mac_addr_l;
+	uint32_t dst_mac_addr_h;
+	uint32_t eth_type;
+	uint32_t hdr_dw[7];
+	uint32_t start_offset;
+	uint32_t bytes_per_cycle;
+} __attribute__ ((packed));
+
+struct ark_pkt_gen_inst {
+	struct rte_eth_dev_info *dev_info;
+	struct ark_pkt_gen_regs *regs;
+	int l2_mode;
+	int ordinal;
+};
+
+/*  packet generator functions */
+ark_pkt_gen_t ark_pktgen_init(void *arg, int ord, int l2_mode);
+void ark_pktgen_uninit(ark_pkt_gen_t handle);
+void ark_pktgen_run(ark_pkt_gen_t handle);
+void ark_pktgen_pause(ark_pkt_gen_t handle);
+uint32_t ark_pktgen_paused(ark_pkt_gen_t handle);
+uint32_t ark_pktgen_is_gen_forever(ark_pkt_gen_t handle);
+uint32_t ark_pktgen_is_running(ark_pkt_gen_t handle);
+uint32_t ark_pktgen_tx_done(ark_pkt_gen_t handle);
+void ark_pktgen_reset(ark_pkt_gen_t handle);
+void ark_pktgen_wait_done(ark_pkt_gen_t handle);
+uint32_t ark_pktgen_get_pkts_sent(ark_pkt_gen_t handle);
+void ark_pktgen_set_payload_byte(ark_pkt_gen_t handle, uint32_t b);
+void ark_pktgen_set_pkt_spacing(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_pkt_size_min(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_pkt_size_max(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_pkt_size_incr(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_num_pkts(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_src_mac_addr(ark_pkt_gen_t handle, uint64_t mac_addr);
+void ark_pktgen_set_dst_mac_addr(ark_pkt_gen_t handle, uint64_t mac_addr);
+void ark_pktgen_set_eth_type(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_set_hdr_dW(ark_pkt_gen_t handle, uint32_t *hdr);
+void ark_pktgen_set_start_offset(ark_pkt_gen_t handle, uint32_t x);
+void ark_pktgen_parse(char *argv);
+void ark_pktgen_setup(ark_pkt_gen_t handle);
+
+#endif
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 5/7] net/ark: Packet TX support initial version
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
                   ` (2 preceding siblings ...)
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 4/7] net/ark: HW API part 3 " Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 6/7] net/ark: Packet RX " Ed Czeck
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

* Core TX packet moving functions
* Flesh out ark_adapter struct to support TX code
(not all fields used at this patch

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/Makefile        |   1 +
 drivers/net/ark/ark_ethdev.c    |  14 ++
 drivers/net/ark/ark_ethdev.h    |   2 +-
 drivers/net/ark/ark_ethdev_tx.c | 501 ++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_ethdev_tx.h |  62 +++++
 drivers/net/ark/ark_global.h    |  28 +++
 6 files changed, 607 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ark/ark_ethdev_tx.c
 create mode 100644 drivers/net/ark/ark_ethdev_tx.h

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index 4a2c41a..11dc373 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -48,6 +48,7 @@ LIBABIVER := 1
 #
 
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev_tx.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktgen.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktchkr.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktdir.c
diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 124b73c..0474e11 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -98,6 +98,20 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 
 };
 
+int
+ark_get_port_id(struct rte_eth_dev *dev, struct ark_adapter *ark)
+{
+	int n = ark->num_ports;
+	int i;
+
+	/* There has to be a smarter way to do this ... */
+	for (i = 0; i < n; i++) {
+		if (ark->port[i].eth_dev == dev)
+			return i;
+	}
+	ARK_DEBUG_TRACE("ARK: Device is NOT associated with a port !!");
+	return -1;
+}
 
 static int
 eth_ark_dev_init(struct rte_eth_dev *dev __rte_unused)
diff --git a/drivers/net/ark/ark_ethdev.h b/drivers/net/ark/ark_ethdev.h
index 08d7fb1..96895d3 100644
--- a/drivers/net/ark/ark_ethdev.h
+++ b/drivers/net/ark/ark_ethdev.h
@@ -34,6 +34,6 @@
 #ifndef _ARK_ETHDEV_H_
 #define _ARK_ETHDEV_H_
 
-/* STUB */
+int ark_get_port_id(struct rte_eth_dev *dev, struct ark_adapter *ark);
 
 #endif
diff --git a/drivers/net/ark/ark_ethdev_tx.c b/drivers/net/ark/ark_ethdev_tx.c
new file mode 100644
index 0000000..6532c8e
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev_tx.c
@@ -0,0 +1,501 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_ethdev_tx.h"
+#include "ark_global.h"
+#include "ark_mpu.h"
+#include "ark_ddm.h"
+#include "ark_ethdev.h"
+#include "ark_debug.h"
+
+#define ARK_TX_META_SIZE   32
+#define ARK_TX_META_OFFSET (RTE_PKTMBUF_HEADROOM - ARK_TX_META_SIZE)
+#define ARK_TX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM)
+#define ARK_TX_PAD_TO_60   1
+
+#ifdef RTE_LIBRTE_ARK_DEBUG_TX
+#define ARK_TX_DEBUG       1
+#define ARK_TX_DEBUG_JUMBO 1
+#else
+#define ARK_TX_DEBUG       0
+#define ARK_TX_DEBUG_JUMBO 0
+#endif
+
+/* ************************************************************************* */
+
+/* struct fixed in FPGA -- 16 bytes */
+
+/* TODO move to ark_ddm.h */
+struct ark_tx_meta {
+	uint64_t physaddr;
+	uint32_t delta_ns;
+	uint16_t data_len;		/* of this MBUF */
+#define   ARK_DDM_EOP   0x01
+#define   ARK_DDM_SOP   0x02
+	uint8_t flags;		/* bit 0 indicates last mbuf in chain. */
+	uint8_t reserved[1];
+};
+
+/* ************************************************************************* */
+struct ark_tx_queue {
+	struct ark_tx_meta *meta_q;
+	struct rte_mbuf **bufs;
+
+	/* handles for hw objects */
+	struct ark_mpu_t *mpu;
+	struct ark_ddm_t *ddm;
+
+	/* Stats HW tracks bytes and packets, need to count send errors */
+	uint64_t tx_errors;
+
+	uint32_t queue_size;
+	uint32_t queue_mask;
+
+	/* 3 indexs to the paired data rings. */
+	uint32_t prod_index;		/* where to put the next one */
+	uint32_t free_index;		/* mbuf has been freed */
+
+	// The queue Id is used to identify the HW Q
+	uint16_t phys_qid;
+	/* The queue Index within the dpdk device structures */
+	uint16_t queue_index;
+
+	uint32_t pad[1];
+
+	/* second cache line - fields only used in slow path */
+	MARKER cacheline1 __rte_cache_min_aligned;
+	uint32_t cons_index;		/* hw is done, can be freed */
+} __rte_cache_aligned;
+
+/* Forward declarations */
+static uint32_t eth_ark_tx_jumbo(struct ark_tx_queue *queue,
+				 struct rte_mbuf *mbuf);
+static int eth_ark_tx_hw_queue_config(struct ark_tx_queue *queue);
+static void free_completed_tx(struct ark_tx_queue *queue);
+
+static inline void
+ark_tx_hw_queue_stop(struct ark_tx_queue *queue)
+{
+	ark_mpu_stop(queue->mpu);
+}
+
+/* ************************************************************************* */
+static inline void
+eth_ark_tx_meta_from_mbuf(struct ark_tx_meta *meta,
+			  const struct rte_mbuf *mbuf,
+			  uint8_t flags)
+{
+	meta->physaddr = rte_mbuf_data_dma_addr(mbuf);
+	meta->delta_ns = 0;
+	meta->data_len = rte_pktmbuf_data_len(mbuf);
+	meta->flags = flags;
+}
+
+/* ************************************************************************* */
+uint16_t
+eth_ark_xmit_pkts_noop(void *vtxq __rte_unused,
+		       struct rte_mbuf **tx_pkts __rte_unused,
+		       uint16_t nb_pkts __rte_unused)
+{
+	return 0;
+}
+
+/* ************************************************************************* */
+uint16_t
+eth_ark_xmit_pkts(void *vtxq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+	struct ark_tx_queue *queue;
+	struct rte_mbuf *mbuf;
+	struct ark_tx_meta *meta;
+
+	uint32_t idx;
+	uint32_t prod_index_limit;
+	int stat;
+	uint16_t nb;
+
+	queue = (struct ark_tx_queue *)vtxq;
+
+	/* free any packets after the HW is done with them */
+	free_completed_tx(queue);
+
+	prod_index_limit = queue->queue_size + queue->free_index;
+
+	for (nb = 0;
+	     (nb < nb_pkts) && (queue->prod_index != prod_index_limit);
+	     ++nb) {
+		mbuf = tx_pkts[nb];
+
+		if (ARK_TX_PAD_TO_60) {
+			if (unlikely(rte_pktmbuf_pkt_len(mbuf) < 60)) {
+				/* this packet even if it is small can be split,
+				 * be sure to add to the end
+				 */
+				uint16_t to_add =
+					60 - rte_pktmbuf_pkt_len(mbuf);
+				char *appended =
+					rte_pktmbuf_append(mbuf, to_add);
+
+				if (appended == 0) {
+					/* This packet is in error,
+					 * we cannot send it so just
+					 * count it and delete it.
+					 */
+					queue->tx_errors += 1;
+					rte_pktmbuf_free(mbuf);
+					continue;
+				}
+				memset(appended, 0, to_add);
+			}
+		}
+
+		if (unlikely(mbuf->nb_segs != 1)) {
+			stat = eth_ark_tx_jumbo(queue, mbuf);
+			if (unlikely(stat != 0))
+				break;		/* Queue is full */
+		} else {
+			idx = queue->prod_index & queue->queue_mask;
+			queue->bufs[idx] = mbuf;
+			meta = &queue->meta_q[idx];
+			eth_ark_tx_meta_from_mbuf(meta,
+						  mbuf,
+						  ARK_DDM_SOP |
+						  ARK_DDM_EOP);
+			queue->prod_index++;
+		}
+	}
+
+	if (ARK_TX_DEBUG) {
+		if (nb != nb_pkts) {
+			PMD_DRV_LOG(ERR,
+				    "ARKP TX: Failure to send: req: %u"
+				    " sent: %u prod: "
+				    "%u cons: %u free: %u\n",
+				    nb_pkts, nb,
+				    queue->prod_index,
+				    queue->cons_index,
+				    queue->free_index);
+			ark_mpu_dump(queue->mpu,
+				     "TX Failure MPU: ",
+				     queue->phys_qid);
+		}
+	}
+
+	/* let fpga know producer index.  */
+	if (likely(nb != 0))
+		ark_mpu_set_producer(queue->mpu, queue->prod_index);
+
+	return nb;
+}
+
+/* ************************************************************************* */
+static uint32_t
+eth_ark_tx_jumbo(struct ark_tx_queue *queue, struct rte_mbuf *mbuf)
+{
+	struct rte_mbuf *next;
+	struct ark_tx_meta *meta;
+	uint32_t free_queue_space;
+	uint32_t idx;
+	uint8_t flags = ARK_DDM_SOP;
+
+	free_queue_space = queue->queue_mask -
+		(queue->prod_index - queue->free_index);
+	if (unlikely(free_queue_space < mbuf->nb_segs))
+		return -1;
+
+	if (ARK_TX_DEBUG_JUMBO) {
+		PMD_DRV_LOG(ERR,
+			    "ARKP  JUMBO TX len: %u segs: %u prod:"
+			    "%u cons: %u free: %u free_space: %u\n",
+			    mbuf->pkt_len, mbuf->nb_segs,
+			    queue->prod_index,
+			    queue->cons_index,
+			    queue->free_index, free_queue_space);
+	}
+
+	while (mbuf != NULL) {
+		next = mbuf->next;
+
+		idx = queue->prod_index & queue->queue_mask;
+		queue->bufs[idx] = mbuf;
+		meta = &queue->meta_q[idx];
+
+		flags |= (next == NULL) ? ARK_DDM_EOP : 0;
+		eth_ark_tx_meta_from_mbuf(meta, mbuf, flags);
+		queue->prod_index++;
+
+		flags &= ~ARK_DDM_SOP;	/* drop SOP flags */
+		mbuf = next;
+	}
+
+	return 0;
+}
+
+/* ************************************************************************* */
+int
+eth_ark_tx_queue_setup(struct rte_eth_dev *dev,
+		       uint16_t queue_idx,
+		       uint16_t nb_desc,
+		       unsigned int socket_id,
+		       const struct rte_eth_txconf *tx_conf __rte_unused)
+{
+	struct ark_adapter *ark = (struct ark_adapter *)dev->data->dev_private;
+	struct ark_tx_queue *queue;
+	int status;
+
+	/* TODO: divide the Q's evenly with the Vports */
+	int port = ark_get_port_id(dev, ark);
+	int qidx = port + queue_idx;	/* FIXME for multi queue */
+
+	if (!rte_is_power_of_2(nb_desc)) {
+		PMD_DRV_LOG(ERR,
+			    "DPDK Arkville configuration queue size"
+			    " must be power of two %u (%s)\n",
+			    nb_desc, __func__);
+		return -1;
+	}
+
+	/* Allocate queue struct */
+	queue =	rte_zmalloc_socket("Ark_tXQueue",
+				   sizeof(struct ark_tx_queue),
+				   64,
+				   socket_id);
+	if (queue == 0) {
+		PMD_DRV_LOG(ERR, "ARKP Failed to allocate tx "
+			    "queue memory in %s\n",
+			    __func__);
+		return -ENOMEM;
+	}
+
+	/* we use zmalloc no need to initialize fields */
+	queue->queue_size = nb_desc;
+	queue->queue_mask = nb_desc - 1;
+	queue->phys_qid = qidx;
+	queue->queue_index = queue_idx;
+	dev->data->tx_queues[queue_idx] = queue;
+
+	queue->meta_q =
+		rte_zmalloc_socket("Ark_tXQueue meta",
+				   nb_desc * sizeof(struct ark_tx_meta),
+				   64,
+				   socket_id);
+	queue->bufs =
+		rte_zmalloc_socket("Ark_tXQueue bufs",
+				   nb_desc * sizeof(struct rte_mbuf *),
+				   64,
+				   socket_id);
+
+	if (queue->meta_q == 0 || queue->bufs == 0) {
+		PMD_DRV_LOG(ERR, "Failed to allocate "
+			    "queue memory in %s\n", __func__);
+		rte_free(queue->meta_q);
+		rte_free(queue->bufs);
+		rte_free(queue);
+		return -ENOMEM;
+	}
+
+	queue->ddm = RTE_PTR_ADD(ark->ddm.v, qidx * ARK_DDM_QOFFSET);
+	queue->mpu = RTE_PTR_ADD(ark->mputx.v, qidx * ARK_MPU_QOFFSET);
+
+	status = eth_ark_tx_hw_queue_config(queue);
+
+	if (unlikely(status != 0)) {
+		rte_free(queue->meta_q);
+		rte_free(queue->bufs);
+		rte_free(queue);
+		return -1;		/* ERROR CODE */
+	}
+
+	return 0;
+}
+
+/* ************************************************************************* */
+static int
+eth_ark_tx_hw_queue_config(struct ark_tx_queue *queue)
+{
+	phys_addr_t queue_base, ring_base, prod_index_addr;
+	uint32_t write_interval_ns;
+
+	/* Verify HW -- MPU */
+	if (ark_mpu_verify(queue->mpu, sizeof(struct ark_tx_meta)))
+		return -1;
+
+	queue_base = rte_malloc_virt2phy(queue);
+	ring_base = rte_malloc_virt2phy(queue->meta_q);
+	prod_index_addr =
+		queue_base + offsetof(struct ark_tx_queue, cons_index);
+
+	ark_mpu_stop(queue->mpu);
+	ark_mpu_reset(queue->mpu);
+
+	/* Stop and Reset and configure MPU */
+	ark_mpu_configure(queue->mpu, ring_base, queue->queue_size, 1);
+
+	/*
+	 * Adjust the write interval based on queue size --
+	 * increase pcie traffic
+	 * when low mbuf count
+	 */
+	switch (queue->queue_size) {
+	case 128:
+		write_interval_ns = 500;
+		break;
+	case 256:
+		write_interval_ns = 500;
+		break;
+	case 512:
+		write_interval_ns = 1000;
+		break;
+	default:
+		write_interval_ns = 2000;
+		break;
+	}
+
+	// Completion address in UDM
+	ark_ddm_setup(queue->ddm, prod_index_addr, write_interval_ns);
+
+	return 0;
+}
+
+/* ************************************************************************* */
+void
+eth_ark_tx_queue_release(void *vtx_queue)
+{
+	struct ark_tx_queue *queue;
+
+	queue = (struct ark_tx_queue *)vtx_queue;
+
+	ark_tx_hw_queue_stop(queue);
+
+	queue->cons_index = queue->prod_index;
+	free_completed_tx(queue);
+
+	rte_free(queue->meta_q);
+	rte_free(queue->bufs);
+	rte_free(queue);
+}
+
+/* ************************************************************************* */
+int
+eth_ark_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+	struct ark_tx_queue *queue;
+	int cnt = 0;
+
+	queue = dev->data->tx_queues[queue_id];
+
+	/* Wait for DDM to send out all packets. */
+	while (queue->cons_index != queue->prod_index) {
+		usleep(100);
+		if (cnt++ > 10000)
+			return -1;
+	}
+
+	ark_mpu_stop(queue->mpu);
+	free_completed_tx(queue);
+
+	dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+	return 0;
+}
+
+int
+eth_ark_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+	struct ark_tx_queue *queue;
+
+	queue = dev->data->tx_queues[queue_id];
+	if (dev->data->tx_queue_state[queue_id] == RTE_ETH_QUEUE_STATE_STARTED)
+		return 0;
+
+	ark_mpu_start(queue->mpu);
+	dev->data->tx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
+	return 0;
+}
+
+/* ************************************************************************* */
+static void
+free_completed_tx(struct ark_tx_queue *queue)
+{
+	struct rte_mbuf *mbuf;
+	struct ark_tx_meta *meta;
+	uint32_t top_index;
+
+	top_index = queue->cons_index;	/* read once */
+	while (queue->free_index != top_index) {
+		meta = &queue->meta_q[queue->free_index & queue->queue_mask];
+		mbuf = queue->bufs[queue->free_index & queue->queue_mask];
+
+		if (likely((meta->flags & ARK_DDM_SOP) != 0)) {
+			/* ref count of the mbuf is checked in this call. */
+			rte_pktmbuf_free(mbuf);
+		}
+		queue->free_index++;
+	}
+}
+
+/* ************************************************************************* */
+void
+eth_tx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats)
+{
+	struct ark_tx_queue *queue;
+	struct ark_ddm_t *ddm;
+	uint64_t bytes, pkts;
+
+	queue = vqueue;
+	ddm = queue->ddm;
+
+	bytes = ark_ddm_queue_byte_count(ddm);
+	pkts = ark_ddm_queue_pkt_count(ddm);
+
+	stats->q_opackets[queue->queue_index] = pkts;
+	stats->q_obytes[queue->queue_index] = bytes;
+	stats->opackets += pkts;
+	stats->obytes += bytes;
+	stats->oerrors += queue->tx_errors;
+}
+
+void
+eth_tx_queue_stats_reset(void *vqueue)
+{
+	struct ark_tx_queue *queue;
+	struct ark_ddm_t *ddm;
+
+	queue = vqueue;
+	ddm = queue->ddm;
+
+	ark_ddm_queue_reset_stats(ddm);
+	queue->tx_errors = 0;
+}
diff --git a/drivers/net/ark/ark_ethdev_tx.h b/drivers/net/ark/ark_ethdev_tx.h
new file mode 100644
index 0000000..a503546
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev_tx.h
@@ -0,0 +1,62 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_ETHDEV_TX_H_
+#define _ARK_ETHDEV_TX_H_
+
+#include <stdint.h>
+
+/* Forward declarations */
+struct rte_mbuf;
+struct rte_eth_dev;
+struct rte_eth_stats;
+struct rte_eth_txconf;
+
+uint16_t eth_ark_xmit_pkts_noop(void *vtxq,
+				struct rte_mbuf **tx_pkts,
+				uint16_t nb_pkts);
+uint16_t eth_ark_xmit_pkts(void *vtxq,
+			   struct rte_mbuf **tx_pkts,
+			   uint16_t nb_pkts);
+int eth_ark_tx_queue_setup(struct rte_eth_dev *dev,
+			   uint16_t queue_idx,
+			   uint16_t nb_desc,
+			   unsigned int socket_id,
+			   const struct rte_eth_txconf *tx_conf);
+void eth_ark_tx_queue_release(void *vtx_queue);
+int eth_ark_tx_queue_stop(struct rte_eth_dev *dev, uint16_t queue_id);
+int eth_ark_tx_queue_start(struct rte_eth_dev *dev, uint16_t queue_id);
+void eth_tx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats);
+void eth_tx_queue_stats_reset(void *vqueue);
+
+#endif
diff --git a/drivers/net/ark/ark_global.h b/drivers/net/ark/ark_global.h
index 7cd62d5..60db9a9 100644
--- a/drivers/net/ark/ark_global.h
+++ b/drivers/net/ark/ark_global.h
@@ -47,6 +47,10 @@
 #include <rte_dev.h>
 #include <rte_version.h>
 
+#include "ark_pktdir.h"
+#include "ark_pktgen.h"
+#include "ark_pktchkr.h"
+
 #define ETH_ARK_ARG_MAXLEN	64
 #define ARK_SYSCTRL_BASE  0x0
 #define ARK_PKTGEN_BASE   0x10000
@@ -88,6 +92,12 @@ struct ark_adapter {
 	/* User extension private data */
 	void *user_data;
 
+	/* Pointers to packet generator and checker */
+	int start_pg;
+	ark_pkt_gen_t pg;
+	ark_pkt_chkr_t pc;
+	ark_pkt_dir_t pd;
+
 	struct ark_port port[ARK_MAX_PORTS];
 	int num_ports;
 
@@ -101,6 +111,24 @@ struct ark_adapter {
 
 	/* Application Bar */
 	uint8_t *a_bar;
+
+	/* Arkville demo block offsets */
+	def_ptr(sys_ctrl, sysctrl);
+	def_ptr(pkt_gen, pktgen);
+	def_ptr(mpu_rx, mpurx);
+	def_ptr(UDM, udm);
+	def_ptr(mpu_tx, mputx);
+	def_ptr(DDM, ddm);
+	def_ptr(CMAC, cmac);
+	def_ptr(external, external);
+	def_ptr(pkt_dir, pktdir);
+	def_ptr(pkt_chkr, pktchkr);
+
+	int started;
+	uint16_t rx_queues;
+	uint16_t tx_queues;
+
+	struct ark_rqpace_t *rqpacing;
 };
 
 typedef uint32_t *ark_t;
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 6/7] net/ark: Packet RX support initial version
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
                   ` (3 preceding siblings ...)
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 5/7] net/ark: Packet TX support initial version Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 7/7] net/ark: Arkville PMD component integration Ed Czeck
  2017-03-22 18:16 ` [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ferruh Yigit
  6 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck

* Core RX packet moving functions

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/Makefile        |   1 +
 drivers/net/ark/ark_ethdev_rx.c | 681 ++++++++++++++++++++++++++++++++++++++++
 drivers/net/ark/ark_ethdev_rx.h |  67 ++++
 3 files changed, 749 insertions(+)
 create mode 100644 drivers/net/ark/ark_ethdev_rx.c
 create mode 100644 drivers/net/ark/ark_ethdev_rx.h

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index 11dc373..615dfa2 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -48,6 +48,7 @@ LIBABIVER := 1
 #
 
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
+SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev_rx.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev_tx.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktgen.c
 SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_pktchkr.c
diff --git a/drivers/net/ark/ark_ethdev_rx.c b/drivers/net/ark/ark_ethdev_rx.c
new file mode 100644
index 0000000..106bf7e
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev_rx.c
@@ -0,0 +1,681 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <unistd.h>
+
+#include "ark_ethdev_rx.h"
+#include "ark_global.h"
+#include "ark_debug.h"
+#include "ark_ethdev.h"
+#include "ark_mpu.h"
+#include "ark_udm.h"
+
+#define ARK_RX_META_SIZE 32
+#define ARK_RX_META_OFFSET (RTE_PKTMBUF_HEADROOM - ARK_RX_META_SIZE)
+#define ARK_RX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM)
+
+#ifdef RTE_LIBRTE_ARK_DEBUG_RX
+#define ARK_RX_DEBUG 1
+#define ARK_FULL_DEBUG 1
+#else
+#define ARK_RX_DEBUG 0
+#define ARK_FULL_DEBUG 0
+#endif
+
+/* Forward declarations */
+struct ark_rx_queue;
+struct ark_rx_meta;
+
+static void dump_mbuf_data(struct rte_mbuf *mbuf, uint16_t lo, uint16_t hi);
+static void ark_ethdev_rx_dump(const char *name, struct ark_rx_queue *queue);
+static uint32_t eth_ark_rx_jumbo(struct ark_rx_queue *queue,
+				 struct ark_rx_meta *meta,
+				 struct rte_mbuf *mbuf0,
+				 uint32_t cons_index);
+static inline int eth_ark_rx_seed_mbufs(struct ark_rx_queue *queue);
+
+/* ************************************************************************* */
+struct ark_rx_queue {
+	/* array of mbufs to populate */
+	struct rte_mbuf **reserve_q;
+	/* array of physical addrresses of the mbuf data pointer */
+	/* This point is a virtual address */
+	phys_addr_t *paddress_q;
+	struct rte_mempool *mb_pool;
+
+	struct ark_udm_t *udm;
+	struct ark_mpu_t *mpu;
+
+	uint32_t queue_size;
+	uint32_t queue_mask;
+
+	uint32_t seed_index;		/* 1 set with an empty mbuf */
+	uint32_t cons_index;		/* 3 consumed by the driver */
+
+	/* The queue Id is used to identify the HW Q */
+	uint16_t phys_qid;
+
+	/* The queue Index is used within the dpdk device structures */
+	uint16_t queue_index;
+
+	uint32_t pad1;
+
+	/* separate cache line */
+	/* second cache line - fields only used in slow path */
+	MARKER cacheline1 __rte_cache_min_aligned;
+
+	volatile uint32_t prod_index;	/* 2 filled by the HW */
+} __rte_cache_aligned;
+
+/* ************************************************************************* */
+
+/* MATCHES struct in UDMDefines.bsv */
+
+/* ************************************************************************* */
+
+/* TODO  pick a better function name */
+static int
+eth_ark_rx_hw_setup(struct rte_eth_dev *dev,
+		    struct ark_rx_queue *queue,
+		    uint16_t rx_queue_id __rte_unused, uint16_t rx_queue_idx)
+{
+	phys_addr_t queue_base;
+	phys_addr_t phys_addr_q_base;
+	phys_addr_t phys_addr_prod_index;
+
+	queue_base = rte_malloc_virt2phy(queue);
+	phys_addr_prod_index = queue_base +
+		offsetof(struct ark_rx_queue, prod_index);
+
+	phys_addr_q_base = rte_malloc_virt2phy(queue->paddress_q);
+
+	/* Verify HW */
+	if (ark_mpu_verify(queue->mpu, sizeof(phys_addr_t))) {
+		PMD_DRV_LOG(ERR, "ARKP: Illegal configuration rx queue\n");
+		return -1;
+	}
+
+	/* Stop and Reset and configure MPU */
+	ark_mpu_configure(queue->mpu, phys_addr_q_base, queue->queue_size, 0);
+
+	ark_udm_write_addr(queue->udm, phys_addr_prod_index);
+
+	/* advance the valid pointer, but don't start until the queue starts */
+	ark_mpu_reset_stats(queue->mpu);
+
+	/* The seed is the producer index for the HW */
+	ark_mpu_set_producer(queue->mpu, queue->seed_index);
+	dev->data->rx_queue_state[rx_queue_idx] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+	return 0;
+}
+
+static inline void
+eth_ark_rx_update_cons_index(struct ark_rx_queue *queue, uint32_t cons_index)
+{
+	queue->cons_index = cons_index;
+	eth_ark_rx_seed_mbufs(queue);
+	ark_mpu_set_producer(queue->mpu, queue->seed_index);
+}
+
+/* ************************************************************************* */
+int
+eth_ark_dev_rx_queue_setup(struct rte_eth_dev *dev,
+			   uint16_t queue_idx,
+			   uint16_t nb_desc,
+			   unsigned int socket_id,
+			   const struct rte_eth_rxconf *rx_conf,
+			   struct rte_mempool *mb_pool)
+{
+	struct ark_adapter *ark = (struct ark_adapter *)dev->data->dev_private;
+	static int warning1;		/* = 0 */
+
+	struct ark_rx_queue *queue;
+	uint32_t i;
+	int status;
+
+	int port = ark_get_port_id(dev, ark);
+	int qidx = port + queue_idx;	/* TODO FIXME */
+
+	// TODO: We may already be setup, check here if there is nothing to do
+	/* Free memory prior to re-allocation if needed */
+	if (dev->data->rx_queues[queue_idx] != NULL) {
+		// TODO: release any allocated queues
+		dev->data->rx_queues[queue_idx] = NULL;
+	}
+
+	if (rx_conf != NULL && warning1 == 0) {
+		warning1 = 1;
+		PMD_DRV_LOG(INFO,
+			    "ARKP: Arkville PMD ignores  rte_eth_rxconf argument.\n");
+	}
+
+	if (RTE_PKTMBUF_HEADROOM < ARK_RX_META_SIZE) {
+		PMD_DRV_LOG(ERR,
+			    "Error: DPDK Arkville requires head room > %d bytes (%s)\n",
+			    ARK_RX_META_SIZE, __func__);
+		return -1;		/* ERROR CODE */
+	}
+
+	if (!rte_is_power_of_2(nb_desc)) {
+		PMD_DRV_LOG(ERR,
+			    "DPDK Arkville configuration queue size must be power of two %u (%s)\n",
+			    nb_desc, __func__);
+		return -1;		/* ERROR CODE */
+	}
+
+	/* Allocate queue struct */
+	queue = rte_zmalloc_socket("Ark_rXQueue",
+				   sizeof(struct ark_rx_queue),
+				   64,
+				   socket_id);
+	if (queue == 0) {
+		PMD_DRV_LOG(ERR, "Failed to allocate memory in %s\n", __func__);
+		return -ENOMEM;
+	}
+
+	/* NOTE zmalloc is used, no need to 0 indexes, etc. */
+	queue->mb_pool = mb_pool;
+	queue->phys_qid = qidx;
+	queue->queue_index = queue_idx;
+	queue->queue_size = nb_desc;
+	queue->queue_mask = nb_desc - 1;
+
+	queue->reserve_q =
+		rte_zmalloc_socket("Ark_rx_queue mbuf",
+				   nb_desc * sizeof(struct rte_mbuf *),
+				   64,
+				   socket_id);
+	queue->paddress_q =
+		rte_zmalloc_socket("Ark_rx_queue paddr",
+				   nb_desc * sizeof(phys_addr_t),
+				   64,
+				   socket_id);
+
+	if (queue->reserve_q == 0 || queue->paddress_q == 0) {
+		PMD_DRV_LOG(ERR,
+			    "Failed to allocate queue memory in %s\n",
+			    __func__);
+		rte_free(queue->reserve_q);
+		rte_free(queue->paddress_q);
+		rte_free(queue);
+		return -ENOMEM;
+	}
+
+	dev->data->rx_queues[queue_idx] = queue;
+	queue->udm = RTE_PTR_ADD(ark->udm.v, qidx * ARK_UDM_QOFFSET);
+	queue->mpu = RTE_PTR_ADD(ark->mpurx.v, qidx * ARK_MPU_QOFFSET);
+
+	/* populate mbuf reserve */
+	status = eth_ark_rx_seed_mbufs(queue);
+
+	/* MPU Setup */
+	if (status == 0)
+		status = eth_ark_rx_hw_setup(dev, queue, qidx, queue_idx);
+
+	if (unlikely(status != 0)) {
+		struct rte_mbuf *mbuf;
+
+		PMD_DRV_LOG(ERR, "ARKP Failed to initialize RX queue %d %s\n",
+			    qidx,
+			    __func__);
+		/* Free the mbufs allocated */
+		for (i = 0, mbuf = queue->reserve_q[0];
+		     i < nb_desc; ++i, mbuf++) {
+			rte_pktmbuf_free(mbuf);
+		}
+		rte_free(queue->reserve_q);
+		rte_free(queue->paddress_q);
+		rte_free(queue);
+		return -1;		/* ERROR CODE */
+	}
+
+	return 0;
+}
+
+/* ************************************************************************* */
+uint16_t
+eth_ark_recv_pkts_noop(void *rx_queue __rte_unused,
+		       struct rte_mbuf **rx_pkts __rte_unused,
+		       uint16_t nb_pkts __rte_unused)
+{
+	return 0;
+}
+
+/* ************************************************************************* */
+uint16_t
+eth_ark_recv_pkts(void *rx_queue,
+		  struct rte_mbuf **rx_pkts,
+		  uint16_t nb_pkts)
+{
+	struct ark_rx_queue *queue;
+	register uint32_t cons_index, prod_index;
+	uint16_t nb;
+	struct rte_mbuf *mbuf;
+	struct ark_rx_meta *meta;
+
+	queue = (struct ark_rx_queue *)rx_queue;
+	if (unlikely(queue == 0))
+		return 0;
+	if (unlikely(nb_pkts == 0))
+		return 0;
+	prod_index = queue->prod_index;
+	cons_index = queue->cons_index;
+	nb = 0;
+
+	while (prod_index != cons_index) {
+		mbuf = queue->reserve_q[cons_index & queue->queue_mask];
+		/* prefetch mbuf ? */
+		rte_mbuf_prefetch_part1(mbuf);
+		rte_mbuf_prefetch_part2(mbuf);
+
+		/* META DATA burried in buffer */
+		meta = RTE_PTR_ADD(mbuf->buf_addr, ARK_RX_META_OFFSET);
+
+		mbuf->port = meta->port;
+		mbuf->pkt_len = meta->pkt_len;
+		mbuf->data_len = meta->pkt_len;
+		mbuf->data_off = RTE_PKTMBUF_HEADROOM;
+		mbuf->udata64 = meta->user_data;
+		if (ARK_RX_DEBUG) {	/* debug use */
+			if ((meta->pkt_len > (1024 * 16)) ||
+			    (meta->pkt_len == 0)) {
+				PMD_DRV_LOG(INFO,
+					    "ARKP RX: Bad Meta Q: %u"
+					    " cons: %u prod: %u\n",
+					    queue->phys_qid,
+					    cons_index,
+					    queue->prod_index);
+
+				PMD_DRV_LOG(INFO,
+					    "       :  cons: %u prod: %u"
+					    " seed_index %u\n",
+					    cons_index,
+					    queue->prod_index,
+					    queue->seed_index);
+
+				PMD_DRV_LOG(INFO, "       :  UDM prod: %u"
+					    "  len: %u\n",
+					    queue->udm->rt_cfg.prod_idx,
+					    meta->pkt_len);
+				ark_mpu_dump(queue->mpu,
+					     "    ",
+					     queue->phys_qid);
+
+				dump_mbuf_data(mbuf, 0, 256);
+				/* its FUBAR so fix it */
+				mbuf->pkt_len = 63;
+				meta->pkt_len = 63;
+			}
+			mbuf->seqn = cons_index;
+		}
+
+		if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN))
+			cons_index = eth_ark_rx_jumbo
+				(queue, meta, mbuf, cons_index + 1);
+		else
+			cons_index += 1;
+
+		rx_pkts[nb] = mbuf;
+		nb++;
+		if (nb >= nb_pkts)
+			break;
+	}
+
+	if (unlikely(nb != 0))
+		/* report next free to FPGA */
+		eth_ark_rx_update_cons_index(queue, cons_index);
+
+	return nb;
+}
+
+/* ************************************************************************* */
+static uint32_t
+eth_ark_rx_jumbo(struct ark_rx_queue *queue,
+		 struct ark_rx_meta *meta,
+		 struct rte_mbuf *mbuf0,
+		 uint32_t cons_index)
+{
+	struct rte_mbuf *mbuf_prev;
+	struct rte_mbuf *mbuf;
+
+	uint16_t remaining;
+	uint16_t data_len;
+	uint8_t segments;
+
+	/* first buf populated by called */
+	mbuf_prev = mbuf0;
+	segments = 1;
+	data_len = RTE_MIN(meta->pkt_len, RTE_MBUF_DEFAULT_DATAROOM);
+	remaining = meta->pkt_len - data_len;
+	mbuf0->data_len = data_len;
+
+	/* TODO check that the data does not exceed prod_index! */
+	while (remaining != 0) {
+		data_len = RTE_MIN(remaining,
+				   RTE_MBUF_DEFAULT_DATAROOM +
+				   RTE_PKTMBUF_HEADROOM);
+
+		remaining -= data_len;
+		segments += 1;
+
+		mbuf = queue->reserve_q[cons_index & queue->queue_mask];
+		mbuf_prev->next = mbuf;
+		mbuf_prev = mbuf;
+		mbuf->data_len = data_len;
+		mbuf->data_off = 0;
+		if (ARK_RX_DEBUG)
+			mbuf->seqn = cons_index;	/* for debug only */
+
+		cons_index += 1;
+	}
+
+	mbuf0->nb_segs = segments;
+	return cons_index;
+}
+
+/* Drain the internal queue allowing hw to clear out. */
+static void
+eth_ark_rx_queue_drain(struct ark_rx_queue *queue)
+{
+	register uint32_t cons_index;
+	struct rte_mbuf *mbuf;
+
+	cons_index = queue->cons_index;
+
+	/* NOT performance optimized, since this is a one-shot call */
+	while ((cons_index ^ queue->prod_index) & queue->queue_mask) {
+		mbuf = queue->reserve_q[cons_index & queue->queue_mask];
+		rte_pktmbuf_free(mbuf);
+		cons_index++;
+		eth_ark_rx_update_cons_index(queue, cons_index);
+	}
+}
+
+uint32_t
+eth_ark_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+	struct ark_rx_queue *queue;
+
+	queue = dev->data->rx_queues[queue_id];
+	return (queue->prod_index - queue->cons_index);	/* mod arith */
+}
+
+/* ************************************************************************* */
+int
+eth_ark_rx_start_queue(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+	struct ark_rx_queue *queue;
+
+	queue = dev->data->rx_queues[queue_id];
+	if (queue == 0)
+		return -1;
+
+	dev->data->rx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
+	ark_mpu_set_producer(queue->mpu, queue->seed_index);
+	ark_mpu_start(queue->mpu);
+
+	ark_udm_queue_enable(queue->udm, 1);
+
+	return 0;
+}
+
+/* ************************************************************************* */
+
+/* Queue can be restarted.   data remains
+ */
+int
+eth_ark_rx_stop_queue(struct rte_eth_dev *dev, uint16_t queue_id)
+{
+	struct ark_rx_queue *queue;
+
+	queue = dev->data->rx_queues[queue_id];
+	if (queue == 0)
+		return -1;
+
+	ark_udm_queue_enable(queue->udm, 0);
+
+	dev->data->rx_queue_state[queue_id] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+	return 0;
+}
+
+/* ************************************************************************* */
+static inline int
+eth_ark_rx_seed_mbufs(struct ark_rx_queue *queue)
+{
+	uint32_t limit = queue->cons_index + queue->queue_size;
+	uint32_t seed_index = queue->seed_index;
+
+	uint32_t count = 0;
+	uint32_t seed_m = queue->seed_index & queue->queue_mask;
+
+	uint32_t nb = limit - seed_index;
+
+	/* Handle wrap around -- remainder is filled on the next call */
+	if (unlikely(seed_m + nb > queue->queue_size))
+		nb = queue->queue_size - seed_m;
+
+	struct rte_mbuf **mbufs = &queue->reserve_q[seed_m];
+	int status = rte_pktmbuf_alloc_bulk(queue->mb_pool, mbufs, nb);
+
+	if (unlikely(status != 0))
+		return -1;
+
+	if (ARK_RX_DEBUG) {		/* DEBUG */
+		while (count != nb) {
+			struct rte_mbuf *mbuf_init =
+				queue->reserve_q[seed_m + count];
+
+			memset(mbuf_init->buf_addr, -1, 512);
+			*((uint32_t *)mbuf_init->buf_addr) =
+				seed_index + count;
+			*(uint16_t *)RTE_PTR_ADD(mbuf_init->buf_addr, 4) =
+				queue->phys_qid;
+			count++;
+		}
+		count = 0;
+	} /* DEBUG */
+	queue->seed_index += nb;
+
+	/* Duff's device https://en.wikipedia.org/wiki/Duff's_device */
+	switch (nb % 4) {
+	case 0:
+		while (count != nb) {
+			queue->paddress_q[seed_m++] =
+				(*mbufs++)->buf_physaddr;
+			count++;
+		/* FALLTHROUGH */
+	case 3:
+		queue->paddress_q[seed_m++] =
+			(*mbufs++)->buf_physaddr;
+		count++;
+		/* FALLTHROUGH */
+	case 2:
+		queue->paddress_q[seed_m++] =
+			(*mbufs++)->buf_physaddr;
+		count++;
+		/* FALLTHROUGH */
+	case 1:
+		queue->paddress_q[seed_m++] =
+			(*mbufs++)->buf_physaddr;
+		count++;
+		/* FALLTHROUGH */
+
+		} /* while (count != nb) */
+	} /* switch */
+
+	return 0;
+}
+
+void
+eth_ark_rx_dump_queue(struct rte_eth_dev *dev, uint16_t queue_id,
+		      const char *msg)
+{
+	struct ark_rx_queue *queue;
+
+	queue = dev->data->rx_queues[queue_id];
+
+	ark_ethdev_rx_dump(msg, queue);
+}
+
+/* ************************************************************************* */
+/* Call on device closed no user API, queue is stopped */
+void
+eth_ark_dev_rx_queue_release(void *vqueue)
+{
+	struct ark_rx_queue *queue;
+	uint32_t i;
+
+	queue = (struct ark_rx_queue *)vqueue;
+	if (queue == 0)
+		return;
+
+	ark_udm_queue_enable(queue->udm, 0);
+	/* Stop the MPU since pointer are going away */
+	ark_mpu_stop(queue->mpu);
+
+	/* Need to clear out mbufs here, dropping packets along the way */
+	eth_ark_rx_queue_drain(queue);
+
+	for (i = 0; i < queue->queue_size; ++i)
+		rte_pktmbuf_free(queue->reserve_q[i]);
+
+	rte_free(queue->reserve_q);
+	rte_free(queue->paddress_q);
+	rte_free(queue);
+}
+
+void
+eth_rx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats)
+{
+	struct ark_rx_queue *queue;
+	struct ark_udm_t *udm;
+
+	queue = vqueue;
+	if (queue == 0)
+		return;
+	udm = queue->udm;
+
+	uint64_t ibytes = ark_udm_bytes(udm);
+	uint64_t ipackets = ark_udm_packets(udm);
+	uint64_t idropped = ark_udm_dropped(queue->udm);
+
+	stats->q_ipackets[queue->queue_index] = ipackets;
+	stats->q_ibytes[queue->queue_index] = ibytes;
+	stats->q_errors[queue->queue_index] = idropped;
+	stats->ipackets += ipackets;
+	stats->ibytes += ibytes;
+	stats->imissed += idropped;
+}
+
+void
+eth_rx_queue_stats_reset(void *vqueue)
+{
+	struct ark_rx_queue *queue;
+
+	queue = vqueue;
+	if (queue == 0)
+		return;
+
+	ark_mpu_reset_stats(queue->mpu);
+	ark_udm_queue_stats_reset(queue->udm);
+}
+
+void
+eth_ark_udm_force_close(struct rte_eth_dev *dev)
+{
+	struct ark_adapter *ark = (struct ark_adapter *)dev->data->dev_private;
+	struct ark_rx_queue *queue;
+	uint32_t index;
+	uint16_t i;
+
+	if (!ark_udm_is_flushed(ark->udm.v)) {
+		/* restart the MPUs */
+		PMD_DRV_LOG(ERR, "ARK: %s UDM not flushed\n", __func__);
+		for (i = 0; i < dev->data->nb_rx_queues; i++) {
+			queue = (struct ark_rx_queue *)dev->data->rx_queues[i];
+			if (queue == 0)
+				continue;
+
+			ark_mpu_start(queue->mpu);
+			/* Add some buffers */
+			index = 100000 + queue->seed_index;
+			ark_mpu_set_producer(queue->mpu, index);
+		}
+		/* Wait to allow data to pass */
+		usleep(100);
+
+		ARK_DEBUG_TRACE("UDM forced flush attempt, stopped = %d\n",
+				ark_udm_is_flushed(ark->udm.v));
+	}
+	ark_udm_reset(ark->udm.v);
+}
+
+static void
+ark_ethdev_rx_dump(const char *name, struct ark_rx_queue *queue)
+{
+	if (queue == NULL)
+		return;
+	ARK_DEBUG_TRACE("RX QUEUE %d -- %s", queue->phys_qid, name);
+	ARK_DEBUG_TRACE(ARK_SU32 ARK_SU32 ARK_SU32 ARK_SU32 "\n",
+			"queue_size", queue->queue_size,
+			"seed_index", queue->seed_index,
+			"prod_index", queue->prod_index,
+			"cons_index", queue->cons_index);
+
+	ark_mpu_dump(queue->mpu, name, queue->phys_qid);
+	ark_mpu_dump_setup(queue->mpu, queue->phys_qid);
+	ark_udm_dump(queue->udm, name);
+	ark_udm_dump_setup(queue->udm, queue->phys_qid);
+}
+
+/* Only used in debug */
+static void
+dump_mbuf_data(struct rte_mbuf *mbuf, uint16_t lo, uint16_t hi)
+{
+	uint16_t i, j;
+
+	PMD_DRV_LOG(ERR, " MBUF: %p len %d, off: %d, seq: %u\n", mbuf,
+		mbuf->pkt_len, mbuf->data_off, mbuf->seqn);
+	for (i = lo; i < hi; i += 16) {
+		uint8_t *dp = RTE_PTR_ADD(mbuf->buf_addr, i);
+
+		PMD_DRV_LOG(ERR, "  %6d:  ", i);
+		for (j = 0; j < 16; j++)
+			PMD_DRV_LOG(ERR, " %02x", dp[j]);
+
+		PMD_DRV_LOG(ERR, "\n");
+	}
+}
diff --git a/drivers/net/ark/ark_ethdev_rx.h b/drivers/net/ark/ark_ethdev_rx.h
new file mode 100644
index 0000000..e6c9fa2
--- /dev/null
+++ b/drivers/net/ark/ark_ethdev_rx.h
@@ -0,0 +1,67 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_ETHDEV_RX_H_
+#define _ARK_ETHDEV_RX_H_
+
+#include <stdint.h>
+
+/* Forward declarations */
+struct rte_mbuf;
+struct rte_mempool;
+struct rte_eth_dev;
+struct rte_eth_stats;
+struct rte_eth_rxconf;
+
+int eth_ark_dev_rx_queue_setup(struct rte_eth_dev *dev,
+			       uint16_t queue_idx,
+			       uint16_t nb_desc,
+			       unsigned int socket_id,
+			       const struct rte_eth_rxconf *rx_conf,
+			       struct rte_mempool *mp);
+uint32_t eth_ark_dev_rx_queue_count(struct rte_eth_dev *dev,
+				    uint16_t rx_queue_id);
+int eth_ark_rx_stop_queue(struct rte_eth_dev *dev, uint16_t queue_id);
+int eth_ark_rx_start_queue(struct rte_eth_dev *dev, uint16_t queue_id);
+uint16_t eth_ark_recv_pkts_noop(void *rx_queue, struct rte_mbuf **rx_pkts,
+				uint16_t nb_pkts);
+uint16_t eth_ark_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
+			   uint16_t nb_pkts);
+void eth_ark_dev_rx_queue_release(void *rx_queue);
+void eth_rx_queue_stats_get(void *vqueue, struct rte_eth_stats *stats);
+void eth_rx_queue_stats_reset(void *vqueue);
+void eth_ark_rx_dump_queue(struct rte_eth_dev *dev, uint16_t queue_id,
+			   const char *msg);
+void eth_ark_udm_force_close(struct rte_eth_dev *dev);
+
+#endif
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [dpdk-dev] [PATCH v3 7/7] net/ark: Arkville PMD component integration
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
                   ` (4 preceding siblings ...)
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 6/7] net/ark: Packet RX " Ed Czeck
@ 2017-03-21 21:43 ` Ed Czeck
  2017-03-22 18:16 ` [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ferruh Yigit
  6 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-21 21:43 UTC (permalink / raw)
  To: dev; +Cc: Ed Czeck, Shepard Siegel, John Miller

* Flesh out device configuration
* Add links dev_ops
* allow dynamic extension loading

Signed-off-by: Shepard Siegel <shepard.siegel@atomicrules.com>
Signed-off-by: John Miller <john.miller@atomicrules.com>
Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
---
 drivers/net/ark/ark_ethdev.c | 737 ++++++++++++++++++++++++++++++++++++++++++-
 drivers/net/ark/ark_ext.h    |  79 +++++
 drivers/net/ark/ark_global.h |  23 ++
 3 files changed, 836 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ark/ark_ext.h

diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
index 0474e11..09aa4a5 100644
--- a/drivers/net/ark/ark_ethdev.c
+++ b/drivers/net/ark/ark_ethdev.c
@@ -41,15 +41,42 @@
 #include "ark_global.h"
 #include "ark_debug.h"
 #include "ark_ethdev.h"
+#include "ark_ethdev_tx.h"
+#include "ark_ethdev_rx.h"
+#include "ark_mpu.h"
+#include "ark_ddm.h"
+#include "ark_udm.h"
+#include "ark_rqp.h"
+#include "ark_pktdir.h"
+#include "ark_pktgen.h"
+#include "ark_pktchkr.h"
 
 /*  Internal prototypes */
 static int eth_ark_check_args(const char *params);
 static int eth_ark_dev_init(struct rte_eth_dev *dev);
+static int ark_config_device(struct rte_eth_dev *dev);
 static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
 static int eth_ark_dev_configure(struct rte_eth_dev *dev);
+static int eth_ark_dev_start(struct rte_eth_dev *dev);
+static void eth_ark_dev_stop(struct rte_eth_dev *dev);
+static void eth_ark_dev_close(struct rte_eth_dev *dev);
 static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 				 struct rte_eth_dev_info *dev_info);
-
+static int eth_ark_dev_link_update(struct rte_eth_dev *dev,
+				   int wait_to_complete);
+static int eth_ark_dev_set_link_up(struct rte_eth_dev *dev);
+static int eth_ark_dev_set_link_down(struct rte_eth_dev *dev);
+static void eth_ark_dev_stats_get(struct rte_eth_dev *dev,
+				  struct rte_eth_stats *stats);
+static void eth_ark_dev_stats_reset(struct rte_eth_dev *dev);
+static void eth_ark_set_default_mac_addr(struct rte_eth_dev *dev,
+					 struct ether_addr *mac_addr);
+static void eth_ark_macaddr_add(struct rte_eth_dev *dev,
+				struct ether_addr *mac_addr,
+				uint32_t index,
+				uint32_t pool);
+static void eth_ark_macaddr_remove(struct rte_eth_dev *dev,
+				   uint32_t index);
 
 #define ARK_DEV_TO_PCI(eth_dev)			\
 	RTE_DEV_TO_PCI((eth_dev)->device)
@@ -94,8 +121,33 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 
 static const struct eth_dev_ops ark_eth_dev_ops = {
 	.dev_configure = eth_ark_dev_configure,
+	.dev_start = eth_ark_dev_start,
+	.dev_stop = eth_ark_dev_stop,
+	.dev_close = eth_ark_dev_close,
+
 	.dev_infos_get = eth_ark_dev_info_get,
 
+	.rx_queue_setup = eth_ark_dev_rx_queue_setup,
+	.rx_queue_count = eth_ark_dev_rx_queue_count,
+	.tx_queue_setup = eth_ark_tx_queue_setup,
+
+	.link_update = eth_ark_dev_link_update,
+	.dev_set_link_up = eth_ark_dev_set_link_up,
+	.dev_set_link_down = eth_ark_dev_set_link_down,
+
+	.rx_queue_start = eth_ark_rx_start_queue,
+	.rx_queue_stop = eth_ark_rx_stop_queue,
+
+	.tx_queue_start = eth_ark_tx_queue_start,
+	.tx_queue_stop = eth_ark_tx_queue_stop,
+
+	.stats_get = eth_ark_dev_stats_get,
+	.stats_reset = eth_ark_dev_stats_reset,
+
+	.mac_addr_add = eth_ark_macaddr_add,
+	.mac_addr_remove = eth_ark_macaddr_remove,
+	.mac_addr_set = eth_ark_set_default_mac_addr,
+
 };
 
 int
@@ -113,22 +165,366 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 	return -1;
 }
 
+static
+int
+check_for_ext(struct rte_eth_dev *dev __rte_unused,
+	      struct ark_adapter *ark __rte_unused)
+{
+	int found = 0;
+
+	/* Get the env */
+	const char *dllpath = getenv("ARK_EXT_PATH");
+
+	if (dllpath == NULL) {
+		ARK_DEBUG_TRACE("ARK EXT NO dll path specified\n");
+		return 0;
+	}
+	ARK_DEBUG_TRACE("ARK EXT found dll path at %s\n", dllpath);
+
+	/* Open and load the .so */
+	ark->d_handle = dlopen(dllpath, RTLD_LOCAL | RTLD_LAZY);
+	if (ark->d_handle == NULL)
+		PMD_DRV_LOG(ERR, "Could not load user extension %s\n", dllpath);
+	else
+		ARK_DEBUG_TRACE("SUCCESS: loaded user extension %s\n", dllpath);
+
+	/* Get the entry points */
+	ark->user_ext.dev_init =
+		(void *(*)(struct rte_eth_dev *, void *, int))
+		dlsym(ark->d_handle, "dev_init");
+	ARK_DEBUG_TRACE("device ext init pointer = %p\n",
+			ark->user_ext.dev_init);
+	ark->user_ext.dev_get_port_count =
+		(int (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_get_port_count");
+	ark->user_ext.dev_uninit =
+		(void (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_uninit");
+	ark->user_ext.dev_configure =
+		(int (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_configure");
+	ark->user_ext.dev_start =
+		(int (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_start");
+	ark->user_ext.dev_stop =
+		(void (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_stop");
+	ark->user_ext.dev_close =
+		(void (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_close");
+	ark->user_ext.link_update =
+		(int (*)(struct rte_eth_dev *, int, void *))
+		dlsym(ark->d_handle, "link_update");
+	ark->user_ext.dev_set_link_up =
+		(int (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_set_link_up");
+	ark->user_ext.dev_set_link_down =
+		(int (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "dev_set_link_down");
+	ark->user_ext.stats_get =
+		(void (*)(struct rte_eth_dev *, struct rte_eth_stats *,
+			  void *))
+		dlsym(ark->d_handle, "stats_get");
+	ark->user_ext.stats_reset =
+		(void (*)(struct rte_eth_dev *, void *))
+		dlsym(ark->d_handle, "stats_reset");
+	ark->user_ext.mac_addr_add =
+		(void (*)(struct rte_eth_dev *, struct ether_addr *, uint32_t,
+			  uint32_t, void *))
+		dlsym(ark->d_handle, "mac_addr_add");
+	ark->user_ext.mac_addr_remove =
+		(void (*)(struct rte_eth_dev *, uint32_t, void *))
+		dlsym(ark->d_handle, "mac_addr_remove");
+	ark->user_ext.mac_addr_set =
+		(void (*)(struct rte_eth_dev *, struct ether_addr *,
+			  void *))
+		dlsym(ark->d_handle, "mac_addr_set");
+
+	return found;
+}
+
 static int
-eth_ark_dev_init(struct rte_eth_dev *dev __rte_unused)
+eth_ark_dev_init(struct rte_eth_dev *dev)
 {
-	return -1;					/* STUB */
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	struct rte_pci_device *pci_dev;
+	int ret;
+
+	ark->eth_dev = dev;
+
+	ARK_DEBUG_TRACE("eth_ark_dev_init(struct rte_eth_dev *dev)\n");
+	gark[0] = ark;
+
+	/* Check to see if there is an extension that we need to load */
+	check_for_ext(dev, ark);
+	pci_dev = ARK_DEV_TO_PCI(dev);
+	rte_eth_copy_pci_info(dev, pci_dev);
+
+	if (pci_dev->device.devargs)
+		eth_ark_check_args(pci_dev->device.devargs->args);
+	else
+		PMD_DRV_LOG(INFO, "No Device args found\n");
+
+	/* Use dummy function until setup */
+	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
+	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
+
+	ark->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;
+	ark->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr;
+
+	ark->sysctrl.v  = (void *)&ark->bar0[ARK_SYSCTRL_BASE];
+	ark->mpurx.v  = (void *)&ark->bar0[ARK_MPU_RX_BASE];
+	ark->udm.v  = (void *)&ark->bar0[ARK_UDM_BASE];
+	ark->mputx.v  = (void *)&ark->bar0[ARK_MPU_TX_BASE];
+	ark->ddm.v  = (void *)&ark->bar0[ARK_DDM_BASE];
+	ark->cmac.v  = (void *)&ark->bar0[ARK_CMAC_BASE];
+	ark->external.v  = (void *)&ark->bar0[ARK_EXTERNAL_BASE];
+	ark->pktdir.v  = (void *)&ark->bar0[ARK_PKTDIR_BASE];
+	ark->pktgen.v  = (void *)&ark->bar0[ARK_PKTGEN_BASE];
+	ark->pktchkr.v  = (void *)&ark->bar0[ARK_PKTCHKR_BASE];
+
+	ark->rqpacing =
+		(struct ark_rqpace_t *)(ark->bar0 + ARK_RCPACING_BASE);
+	ark->started = 0;
+
+	ARK_DEBUG_TRACE
+		("Sys Ctrl Const = 0x%x  HW Commit_ID: %08x\n",
+		 ark->sysctrl.t32[4],
+		 rte_be_to_cpu_32(ark->sysctrl.t32[0x20 / 4]));
+	PMD_DRV_LOG(INFO, "ARKP PMD  HW Commit_ID: %08x\n",
+		    rte_be_to_cpu_32(ark->sysctrl.t32[0x20 / 4]));
+
+	/* If HW sanity test fails, return an error */
+	if (ark->sysctrl.t32[4] != 0xcafef00d) {
+		PMD_DRV_LOG(ERR,
+			    "HW Sanity test has failed, expected constant"
+			    " 0x%x, read 0x%x (%s)\n",
+			    0xcafef00d,
+			    ark->sysctrl.t32[4], __func__);
+		return -1;
+	}
+
+	PMD_DRV_LOG(INFO,
+		    "HW Sanity test has PASSED, expected constant"
+		    " 0x%x, read 0x%x (%s)\n",
+		    0xcafef00d, ark->sysctrl.t32[4], __func__);
+
+	/* We are a single function multi-port device. */
+	const unsigned int numa_node = rte_socket_id();
+	struct ether_addr adr;
+
+	ret = ark_config_device(dev);
+	dev->dev_ops = &ark_eth_dev_ops;
+
+	dev->data->mac_addrs = rte_zmalloc("ark", ETHER_ADDR_LEN, 0);
+	if (!dev->data->mac_addrs) {
+		PMD_DRV_LOG(ERR,
+			    "Failed to allocated memory for storing mac address"
+			    );
+	}
+	ether_addr_copy((struct ether_addr *)&adr, &dev->data->mac_addrs[0]);
+
+	if (ark->user_ext.dev_init) {
+		ark->user_data = ark->user_ext.dev_init(dev, ark->a_bar, 0);
+		if (!ark->user_data) {
+			PMD_DRV_LOG(INFO,
+				    "Failed to initialize PMD extension!"
+				    " continuing without it\n");
+			memset(&ark->user_ext, 0, sizeof(struct ark_user_ext));
+			dlclose(ark->d_handle);
+		}
+	}
+
+	/*
+	 * We will create additional devices based on the number of requested
+	 * ports
+	 */
+	int pc = 1;
+	int p;
+
+	if (ark->user_ext.dev_get_port_count) {
+		pc = ark->user_ext.dev_get_port_count(dev, ark->user_data);
+		ark->num_ports = pc;
+	} else {
+		ark->num_ports = 1;
+	}
+	for (p = 0; p < pc; p++) {
+		struct ark_port *port;
+
+		port = &ark->port[p];
+		struct rte_eth_dev_data *data = NULL;
+
+		port->id = p;
+
+		char name[RTE_ETH_NAME_MAX_LEN];
+
+		snprintf(name, sizeof(name), "arketh%d",
+			 dev->data->port_id + p);
+
+		if (p == 0) {
+			/* First port is already allocated by DPDK */
+			port->eth_dev = ark->eth_dev;
+			continue;
+		}
+
+		/* reserve an ethdev entry */
+		port->eth_dev = rte_eth_dev_allocate(name);
+		if (!port->eth_dev) {
+			PMD_DRV_LOG(ERR,
+				    "Could not allocate eth_dev for port %d\n",
+				    p);
+			goto error;
+		}
+
+		data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node);
+		if (!data) {
+			PMD_DRV_LOG(ERR,
+				    "Could not allocate eth_dev for port %d\n",
+				    p);
+			goto error;
+		}
+		data->port_id = ark->eth_dev->data->port_id + p;
+		port->eth_dev->data = data;
+		port->eth_dev->device = &pci_dev->device;
+		port->eth_dev->data->dev_private = ark;
+		port->eth_dev->driver = ark->eth_dev->driver;
+		port->eth_dev->dev_ops = ark->eth_dev->dev_ops;
+		port->eth_dev->tx_pkt_burst = ark->eth_dev->tx_pkt_burst;
+		port->eth_dev->rx_pkt_burst = ark->eth_dev->rx_pkt_burst;
+
+		rte_eth_copy_pci_info(port->eth_dev, pci_dev);
+
+		port->eth_dev->data->mac_addrs =
+			rte_zmalloc(name, ETHER_ADDR_LEN, 0);
+		if (!port->eth_dev->data->mac_addrs) {
+			PMD_DRV_LOG(ERR,
+				    "Memory allocation for MAC failed!"
+				    " Exiting.\n");
+			goto error;
+		}
+		ether_addr_copy(&adr,
+				&port->eth_dev->data->mac_addrs[0]);
+
+		if (ark->user_ext.dev_init)
+			ark->user_data =
+				ark->user_ext.dev_init(dev, ark->a_bar, p);
+	}
+
+	return ret;
+
+ error:
+	if (dev->data->mac_addrs)
+		rte_free(dev->data->mac_addrs);
+
+	for (p = 0; p < pc; p++) {
+		if (ark->port[p].eth_dev->data)
+			rte_free(ark->port[p].eth_dev->data);
+		if (ark->port[p].eth_dev->data->mac_addrs)
+			rte_free(ark->port[p].eth_dev->data->mac_addrs);
+	}
+
+	return -1;
 }
 
+/*
+ *Initial device configuration when device is opened
+ * setup the DDM, and UDM
+ * Called once per PCIE device
+ */
+static int
+ark_config_device(struct rte_eth_dev *dev)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	uint16_t num_q, i;
+	struct ark_mpu_t *mpu;
+
+	/*
+	 * Make sure that the packet director, generator and checker are in a
+	 * known state
+	 */
+	ark->start_pg = 0;
+	ark->pg = ark_pktgen_init(ark->pktgen.v, 0, 1);
+	ark_pktgen_reset(ark->pg);
+	ark->pc = ark_pktchkr_init(ark->pktchkr.v, 0, 1);
+	ark_pktchkr_stop(ark->pc);
+	ark->pd = ark_pktdir_init(ark->pktdir.v);
+
+	/* Verify HW */
+	if (ark_udm_verify(ark->udm.v))
+		return -1;
+	if (ark_ddm_verify(ark->ddm.v))
+		return -1;
+
+	/* UDM */
+	if (ark_udm_reset(ark->udm.v)) {
+		PMD_DRV_LOG(ERR, "Unable to stop and reset UDM\n");
+		return -1;
+	}
+	/* Keep in reset until the MPU are cleared */
+
+	/* MPU reset */
+	mpu = ark->mpurx.v;
+	num_q = ark_api_num_queues(mpu);
+	ark->rx_queues = num_q;
+	for (i = 0; i < num_q; i++) {
+		ark_mpu_reset(mpu);
+		mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
+	}
+
+	ark_udm_stop(ark->udm.v, 0);
+	ark_udm_configure(ark->udm.v,
+			  RTE_PKTMBUF_HEADROOM,
+			  RTE_MBUF_DEFAULT_DATAROOM,
+			  ARK_RX_WRITE_TIME_NS);
+	ark_udm_stats_reset(ark->udm.v);
+	ark_udm_stop(ark->udm.v, 0);
+
+	/* TX -- DDM */
+	if (ark_ddm_stop(ark->ddm.v, 1))
+		PMD_DRV_LOG(ERR, "Unable to stop DDM\n");
+
+	mpu = ark->mputx.v;
+	num_q = ark_api_num_queues(mpu);
+	ark->tx_queues = num_q;
+	for (i = 0; i < num_q; i++) {
+		ark_mpu_reset(mpu);
+		mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
+	}
+
+	ark_ddm_reset(ark->ddm.v);
+	ark_ddm_stats_reset(ark->ddm.v);
+
+	ark_ddm_stop(ark->ddm.v, 0);
+	ark_rqp_stats_reset(ark->rqpacing);
+
+	return 0;
+}
 
 static int
 eth_ark_dev_uninit(struct rte_eth_dev *dev)
 {
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
 
+	if (ark->user_ext.dev_uninit)
+		ark->user_ext.dev_uninit(dev, ark->user_data);
+
+	ark_pktgen_uninit(ark->pg);
+	ark_pktchkr_uninit(ark->pc);
+
 	dev->dev_ops = NULL;
 	dev->rx_pkt_burst = NULL;
 	dev->tx_pkt_burst = NULL;
+	if (dev->data->mac_addrs)
+		rte_free(dev->data->mac_addrs);
+	if (dev->data)
+		rte_free(dev->data);
+
 	return 0;
 }
 
@@ -136,16 +532,214 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 eth_ark_dev_configure(struct rte_eth_dev *dev __rte_unused)
 {
 	ARK_DEBUG_TRACE("ARKP: In %s\n", __func__);
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	eth_ark_dev_set_link_up(dev);
+	if (ark->user_ext.dev_configure)
+		return ark->user_ext.dev_configure(dev, ark->user_data);
+	return 0;
+}
+
+static void *
+delay_pg_start(void *arg)
+{
+	struct ark_adapter *ark = (struct ark_adapter *)arg;
+
+	/* This function is used exclusively for regression testing, We
+	 * perform a blind sleep here to ensure that the external test
+	 * application has time to setup the test before we generate packets
+	 */
+	usleep(100000);
+	ark_pktgen_run(ark->pg);
+	return NULL;
+}
+
+static int
+eth_ark_dev_start(struct rte_eth_dev *dev)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	int i;
+
+	ARK_DEBUG_TRACE("ARKP: In eth_ark_dev_start\n");
+
+	/* RX Side */
+	/* start UDM */
+	ark_udm_start(ark->udm.v);
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		eth_ark_rx_start_queue(dev, i);
+
+	/* TX Side */
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		eth_ark_tx_queue_start(dev, i);
+
+	/* start DDM */
+	ark_ddm_start(ark->ddm.v);
+
+	ark->started = 1;
+	/* set xmit and receive function */
+	dev->rx_pkt_burst = &eth_ark_recv_pkts;
+	dev->tx_pkt_burst = &eth_ark_xmit_pkts;
+
+	if (ark->start_pg)
+		ark_pktchkr_run(ark->pc);
+
+	if (ark->start_pg && (ark_get_port_id(dev, ark) == 0)) {
+		pthread_t thread;
+
+		/* Delay packet generatpr start allow the hardware to be ready
+		 * This is only used for sanity checking with internal generator
+		 */
+		pthread_create(&thread, NULL, delay_pg_start, ark);
+	}
+
+	if (ark->user_ext.dev_start)
+		ark->user_ext.dev_start(dev, ark->user_data);
+
 	return 0;
 }
 
 static void
+eth_ark_dev_stop(struct rte_eth_dev *dev)
+{
+	uint16_t i;
+	int status;
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	struct ark_mpu_t *mpu;
+
+	ARK_DEBUG_TRACE("ARKP: In eth_ark_dev_stop\n");
+
+	if (ark->started == 0)
+		return;
+	ark->started = 0;
+
+	/* Stop the extension first */
+	if (ark->user_ext.dev_stop)
+		ark->user_ext.dev_stop(dev, ark->user_data);
+
+	/* Stop the packet generator */
+	if (ark->start_pg)
+		ark_pktgen_pause(ark->pg);
+
+	dev->rx_pkt_burst = &eth_ark_recv_pkts_noop;
+	dev->tx_pkt_burst = &eth_ark_xmit_pkts_noop;
+
+	/* STOP TX Side */
+	for (i = 0; i < dev->data->nb_tx_queues; i++) {
+		status = eth_ark_tx_queue_stop(dev, i);
+		if (status != 0) {
+			uint8_t port = dev->data->port_id;
+			PMD_DRV_LOG(ERR,
+				    "ARKP tx_queue stop anomaly"
+				    " port %u, queue %u\n",
+				    port, i);
+		}
+	}
+
+	/* Stop DDM */
+	/* Wait up to 0.1 second.  each stop is upto 1000 * 10 useconds */
+	for (i = 0; i < 10; i++) {
+		status = ark_ddm_stop(ark->ddm.v, 1);
+		if (status == 0)
+			break;
+	}
+	if (status || i != 0) {
+		PMD_DRV_LOG(ERR, "DDM stop anomaly. status:"
+			    " %d iter: %u. (%s)\n",
+			    status,
+			    i,
+			    __func__);
+		ark_ddm_dump(ark->ddm.v, "Stop anomaly");
+
+		mpu = ark->mputx.v;
+		for (i = 0; i < ark->tx_queues; i++) {
+			ark_mpu_dump(mpu, "DDM failure dump", i);
+			mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
+		}
+	}
+
+	/* STOP RX Side */
+	/* Stop UDM  multiple tries attempted */
+	for (i = 0; i < 10; i++) {
+		status = ark_udm_stop(ark->udm.v, 1);
+		if (status == 0)
+			break;
+	}
+	if (status || i != 0) {
+		PMD_DRV_LOG(ERR, "UDM stop anomaly. status %d iter: %u. (%s)\n",
+			    status, i, __func__);
+		ark_udm_dump(ark->udm.v, "Stop anomaly");
+
+		mpu = ark->mpurx.v;
+		for (i = 0; i < ark->rx_queues; i++) {
+			ark_mpu_dump(mpu, "UDM Stop anomaly", i);
+			mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);
+		}
+	}
+
+	ark_udm_dump_stats(ark->udm.v, "Post stop");
+	ark_udm_dump_perf(ark->udm.v, "Post stop");
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		eth_ark_rx_dump_queue(dev, i, __func__);
+
+	/* Stop the packet checker if it is running */
+	if (ark->start_pg) {
+		ark_pktchkr_dump_stats(ark->pc);
+		ark_pktchkr_stop(ark->pc);
+	}
+}
+
+static void
+eth_ark_dev_close(struct rte_eth_dev *dev)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	uint16_t i;
+
+	if (ark->user_ext.dev_close)
+		ark->user_ext.dev_close(dev, ark->user_data);
+
+	eth_ark_dev_stop(dev);
+	eth_ark_udm_force_close(dev);
+
+	/*
+	 * TODO This should only be called once for the device during shutdown
+	 */
+	ark_rqp_dump(ark->rqpacing);
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++) {
+		eth_ark_tx_queue_release(dev->data->tx_queues[i]);
+		dev->data->tx_queues[i] = 0;
+	}
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++) {
+		eth_ark_dev_rx_queue_release(dev->data->rx_queues[i]);
+		dev->data->rx_queues[i] = 0;
+	}
+}
+
+static void
 eth_ark_dev_info_get(struct rte_eth_dev *dev,
 		     struct rte_eth_dev_info *dev_info)
 {
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+	struct ark_mpu_t *tx_mpu = RTE_PTR_ADD(ark->bar0, ARK_MPU_TX_BASE);
+	struct ark_mpu_t *rx_mpu = RTE_PTR_ADD(ark->bar0, ARK_MPU_RX_BASE);
+
+	uint16_t ports = ark->num_ports;
+
 	/* device specific configuration */
 	memset(dev_info, 0, sizeof(*dev_info));
 
+	dev_info->max_rx_queues = ark_api_num_queues_per_port(rx_mpu, ports);
+	dev_info->max_tx_queues = ark_api_num_queues_per_port(tx_mpu, ports);
+	dev_info->max_mac_addrs = 0;
+	dev_info->if_index = 0;
 	dev_info->max_rx_pktlen = (16 * 1024) - 128;
 	dev_info->min_rx_bufsize = 1024;
 	dev_info->rx_offload_capa = 0;
@@ -175,6 +769,120 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 	dev_info->driver_name = dev->data->drv_name;
 }
 
+static int
+eth_ark_dev_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+	ARK_DEBUG_TRACE("ARKP: link status = %d\n",
+			dev->data->dev_link.link_status);
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.link_update) {
+		return ark->user_ext.link_update
+			(dev, wait_to_complete,
+			 ark->user_data);
+	}
+	return 0;
+}
+
+static int
+eth_ark_dev_set_link_up(struct rte_eth_dev *dev)
+{
+	dev->data->dev_link.link_status = 1;
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.dev_set_link_up)
+		return ark->user_ext.dev_set_link_up(dev, ark->user_data);
+	return 0;
+}
+
+static int
+eth_ark_dev_set_link_down(struct rte_eth_dev *dev)
+{
+	dev->data->dev_link.link_status = 0;
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.dev_set_link_down)
+		return ark->user_ext.dev_set_link_down(dev, ark->user_data);
+	return 0;
+}
+
+static void
+eth_ark_dev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
+{
+	uint16_t i;
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	stats->ipackets = 0;
+	stats->ibytes = 0;
+	stats->opackets = 0;
+	stats->obytes = 0;
+	stats->imissed = 0;
+	stats->oerrors = 0;
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		eth_tx_queue_stats_get(dev->data->tx_queues[i], stats);
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		eth_rx_queue_stats_get(dev->data->rx_queues[i], stats);
+	if (ark->user_ext.stats_get)
+		ark->user_ext.stats_get(dev, stats, ark->user_data);
+}
+
+static void
+eth_ark_dev_stats_reset(struct rte_eth_dev *dev)
+{
+	uint16_t i;
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		eth_tx_queue_stats_reset(dev->data->rx_queues[i]);
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		eth_rx_queue_stats_reset(dev->data->rx_queues[i]);
+	if (ark->user_ext.stats_reset)
+		ark->user_ext.stats_reset(dev, ark->user_data);
+}
+
+static void
+eth_ark_macaddr_add(struct rte_eth_dev *dev,
+		    struct ether_addr *mac_addr,
+		    uint32_t index,
+		    uint32_t pool)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.mac_addr_add)
+		ark->user_ext.mac_addr_add(dev,
+					   mac_addr,
+					   index,
+					   pool,
+					   ark->user_data);
+}
+
+static void
+eth_ark_macaddr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.mac_addr_remove)
+		ark->user_ext.mac_addr_remove(dev, index, ark->user_data);
+}
+
+static void
+eth_ark_set_default_mac_addr(struct rte_eth_dev *dev,
+			     struct ether_addr *mac_addr)
+{
+	struct ark_adapter *ark =
+		(struct ark_adapter *)dev->data->dev_private;
+
+	if (ark->user_ext.mac_addr_set)
+		ark->user_ext.mac_addr_set(dev, mac_addr, ark->user_data);
+}
 
 static inline int
 process_pktdir_arg(const char *key, const char *value,
@@ -220,6 +928,12 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 	unsigned int k_idx;
 	struct rte_kvargs_pair *pair = NULL;
 
+	/*
+	 * TODO: the index of gark[index] should be associated with phy dev
+	 * map
+	 */
+	struct ark_adapter *ark = gark[0];
+
 	kvlist = rte_kvargs_parse(params, valid_arguments);
 	if (kvlist == NULL)
 		return 0;
@@ -254,8 +968,25 @@ static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
 		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTCHKR_ARG);
 	}
 
+	/* Setup the packet director */
+	ark_pktdir_setup(ark->pd, pkt_dir_v);
 	ARK_DEBUG_TRACE("INFO: packet director set to 0x%x\n", pkt_dir_v);
 
+	/* Setup the packet generator */
+	if (pkt_gen_args[0]) {
+		PMD_DRV_LOG(INFO, "Setting up the packet generator\n");
+		ark_pktgen_parse(pkt_gen_args);
+		ark_pktgen_reset(ark->pg);
+		ark_pktgen_setup(ark->pg);
+		ark->start_pg = 1;
+	}
+
+	/* Setup the packet checker */
+	if (pkt_chkr_args[0]) {
+		ark_pktchkr_parse(pkt_chkr_args);
+		ark_pktchkr_setup(ark->pc);
+	}
+
 	return 1;
 }
 
diff --git a/drivers/net/ark/ark_ext.h b/drivers/net/ark/ark_ext.h
new file mode 100644
index 0000000..0786d1f
--- /dev/null
+++ b/drivers/net/ark/ark_ext.h
@@ -0,0 +1,79 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright (c) 2015-2017 Atomic Rules LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of copyright holder nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _ARK_EXT_H_
+#define _ARK_EXT_H_
+
+/*
+ * Called post PMD init.
+ * The implementation returns its private data that gets passed into
+ * all other functions as user_data
+ * The ARK extension implementation MUST implement this function
+ */
+void *dev_init(struct rte_eth_dev *dev, void *a_bar, int port_id);
+
+/* Called during device shutdown */
+void dev_uninit(struct rte_eth_dev *dev, void *user_data);
+
+/* This call is optional and allows the
+ * extension to specify the number of supported ports.
+ */
+uint8_t dev_get_port_count(struct rte_eth_dev *dev, void *user_data);
+
+/*
+ * The following functions are optional and are directly mapped
+ * from the DPDK PMD ops structure.
+ * Each function if implemented is called after the ARK PMD
+ * implementation executes.
+ */
+int dev_configure(struct rte_eth_dev *dev, void *user_data);
+int dev_start(struct rte_eth_dev *dev, void *user_data);
+void dev_stop(struct rte_eth_dev *dev, void *user_data);
+void dev_close(struct rte_eth_dev *dev, void *user_data);
+int link_update(struct rte_eth_dev *dev, int wait_to_complete,
+	void *user_data);
+int dev_set_link_up(struct rte_eth_dev *dev, void *user_data);
+int dev_set_link_down(struct rte_eth_dev *dev, void *user_data);
+void stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats,
+	void *user_data);
+void stats_reset(struct rte_eth_dev *dev, void *user_data);
+void mac_addr_add(struct rte_eth_dev *dev,
+	struct ether_addr *macadr,
+				  uint32_t index,
+				  uint32_t pool,
+				  void *user_data);
+void mac_addr_remove(struct rte_eth_dev *dev, uint32_t index, void *user_data);
+void mac_addr_set(struct rte_eth_dev *dev, struct ether_addr *mac_addr,
+	void *user_data);
+
+#endif
diff --git a/drivers/net/ark/ark_global.h b/drivers/net/ark/ark_global.h
index 60db9a9..7b3c333 100644
--- a/drivers/net/ark/ark_global.h
+++ b/drivers/net/ark/ark_global.h
@@ -88,6 +88,28 @@ struct ark_port {
 	int id;
 };
 
+struct ark_user_ext {
+	void *(*dev_init)(struct rte_eth_dev *, void *abar, int port_id);
+	void (*dev_uninit)(struct rte_eth_dev *, void *);
+	int (*dev_get_port_count)(struct rte_eth_dev *, void *);
+	int (*dev_configure)(struct rte_eth_dev *, void *);
+	int (*dev_start)(struct rte_eth_dev *, void *);
+	void (*dev_stop)(struct rte_eth_dev *, void *);
+	void (*dev_close)(struct rte_eth_dev *, void *);
+	int (*link_update)(struct rte_eth_dev *, int wait_to_complete, void *);
+	int (*dev_set_link_up)(struct rte_eth_dev *, void *);
+	int (*dev_set_link_down)(struct rte_eth_dev *, void *);
+	void (*stats_get)(struct rte_eth_dev *, struct rte_eth_stats *, void *);
+	void (*stats_reset)(struct rte_eth_dev *, void *);
+	void (*mac_addr_add)(struct rte_eth_dev *,
+						  struct ether_addr *,
+						 uint32_t,
+						 uint32_t,
+						 void *);
+	void (*mac_addr_remove)(struct rte_eth_dev *, uint32_t, void *);
+	void (*mac_addr_set)(struct rte_eth_dev *, struct ether_addr *, void *);
+};
+
 struct ark_adapter {
 	/* User extension private data */
 	void *user_data;
@@ -105,6 +127,7 @@ struct ark_adapter {
 	struct rte_eth_dev *eth_dev;
 
 	void *d_handle;
+	struct ark_user_ext user_ext;
 
 	/* Our Bar 0 */
 	uint8_t *bar0;
-- 
1.9.1

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3 Ed Czeck
@ 2017-03-21 22:15   ` Stephen Hemminger
  2017-03-22  0:13     ` Ed Czeck
  0 siblings, 1 reply; 13+ messages in thread
From: Stephen Hemminger @ 2017-03-21 22:15 UTC (permalink / raw)
  To: Ed Czeck; +Cc: dev

On Tue, 21 Mar 2017 17:43:36 -0400
Ed Czeck <ed.czeck@atomicrules.com> wrote:

> +	volatile uint32_t ctrl;
> +	volatile uint32_t stats_clear;
> +	volatile uint32_t cplh_max;

You are using a lot of 'volatile' in this driver.
In general volatile should be reserved for known software shared data structures
rather than every device register and statistics. I.e volatile is an inefficient
replacement for proper use of barriers. The DPDK is not as stringent as Linux
kernel (the kernel community considers almost all use of volatile a bug).
See kernel document. process/volatile-considered-harmful.rst

Do you understand that volatile requires compiler to generate the most paranoid code possible.
Basically every reference turns into a barrier.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3
  2017-03-21 22:15   ` Stephen Hemminger
@ 2017-03-22  0:13     ` Ed Czeck
  2017-03-22  0:30       ` Stephen Hemminger
  0 siblings, 1 reply; 13+ messages in thread
From: Ed Czeck @ 2017-03-22  0:13 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev

Hi Stephen,

Yes I understand the paranoid issue of volatile modifier, which is the
reason it was coded in this manner.

The struct with volatile are memory mapped IO structures.  These structure
are not instantiated in memory and hence do not incur concurrent (host)
software access.  The a vast majority of the fields as read-only (as viewed
by the host.)  The concurrency model is that hardware is the producer, and
the host the consumer.  There is no handshake other than the host must not
expect the memory location to remain constant -- that is optimization off.

As for performance, these structures are not in any critical path, so he
optimization path is correctness not performance.

Do you have a recommendation for changing this code without using the
volatile modifier?  I have requirements from the hardware in the pattern of:
write a 1 to location x
read from location y until it become 0
write a 0 to location x
How can this work without a volatile modified on x and y?

Best,
Ed.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3
  2017-03-22  0:13     ` Ed Czeck
@ 2017-03-22  0:30       ` Stephen Hemminger
  2017-03-22 14:36         ` Ed Czeck
  0 siblings, 1 reply; 13+ messages in thread
From: Stephen Hemminger @ 2017-03-22  0:30 UTC (permalink / raw)
  To: Ed Czeck; +Cc: dev

On Tue, 21 Mar 2017 20:13:48 -0400
Ed Czeck <ed.czeck@atomicrules.com> wrote:

> Hi Stephen,
> 
> Yes I understand the paranoid issue of volatile modifier, which is the
> reason it was coded in this manner.
> 
> The struct with volatile are memory mapped IO structures.  These structure
> are not instantiated in memory and hence do not incur concurrent (host)
> software access.  The a vast majority of the fields as read-only (as viewed
> by the host.)  The concurrency model is that hardware is the producer, and
> the host the consumer.  There is no handshake other than the host must not
> expect the memory location to remain constant -- that is optimization off.
> 
> As for performance, these structures are not in any critical path, so he
> optimization path is correctness not performance.
> 
> Do you have a recommendation for changing this code without using the
> volatile modifier?  I have requirements from the hardware in the pattern of:
> write a 1 to location x
> read from location y until it become 0
> write a 0 to location x
> How can this work without a volatile modified on x and y?
> 
> Best,
> Ed.

Learn to use explicit memory barriers. DPDK has rmb()/wmb() just like kernel.

Once again good explanation of memory barriers in kernel doc.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3
  2017-03-22  0:30       ` Stephen Hemminger
@ 2017-03-22 14:36         ` Ed Czeck
  0 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-22 14:36 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev

Hi Stephen,

The issue is not ordering or synchronizing IO read and write operations, so
memory barriers are not the solution and will not address the compile
optimizations which the volatile qualifier avoids.  The linux kernel guide
states: "The volatile storage class was originally meant for memory-mapped
I/O registers."  This is the usage in this case.

The use of rte_read32() and other functions in that family are merely
wrappers to casts to volatile. These functions and other wrappers are used
throughout the drivers/net code.  However, the use of these functions
requires that every access point be manually decided and checked.  By
adding the volatile qualifier to the struct definition, there is no need to
use these wrappers, since the struct definition is true to the physical
hardware implementation.

On Tue, Mar 21, 2017 at 8:30 PM, Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Tue, 21 Mar 2017 20:13:48 -0400
> Ed Czeck <ed.czeck@atomicrules.com> wrote:
>
> > Hi Stephen,
> >
> > Yes I understand the paranoid issue of volatile modifier, which is the
> > reason it was coded in this manner.
> >
> > The struct with volatile are memory mapped IO structures.  These
> structure
> > are not instantiated in memory and hence do not incur concurrent (host)
> > software access.  The a vast majority of the fields as read-only (as
> viewed
> > by the host.)  The concurrency model is that hardware is the producer,
> and
> > the host the consumer.  There is no handshake other than the host must
> not
> > expect the memory location to remain constant -- that is optimization
> off.
> >
> > As for performance, these structures are not in any critical path, so he
> > optimization path is correctness not performance.
> >
> > Do you have a recommendation for changing this code without using the
> > volatile modifier?  I have requirements from the hardware in the pattern
> of:
> > write a 1 to location x
> > read from location y until it become 0
> > write a 0 to location x
> > How can this work without a volatile modified on x and y?
> >
> > Best,
> > Ed.
>
> Learn to use explicit memory barriers. DPDK has rmb()/wmb() just like
> kernel.
>
> Once again good explanation of memory barriers in kernel doc.
>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub
  2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
                   ` (5 preceding siblings ...)
  2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 7/7] net/ark: Arkville PMD component integration Ed Czeck
@ 2017-03-22 18:16 ` Ferruh Yigit
  2017-03-23  1:09   ` Ed Czeck
  6 siblings, 1 reply; 13+ messages in thread
From: Ferruh Yigit @ 2017-03-22 18:16 UTC (permalink / raw)
  To: Ed Czeck, dev

On 3/21/2017 9:43 PM, Ed Czeck wrote:
> Enable Arkville on supported configurations
> Add overview documentation
> Minimum driver support for valid compile
> 
> 
> Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
> ---
>  MAINTAINERS                                 |   8 +
>  config/common_base                          |  11 ++
>  config/defconfig_arm-armv7a-linuxapp-gcc    |   1 +
>  config/defconfig_ppc_64-power8-linuxapp-gcc |   1 +
>  doc/guides/nics/ark.rst                     | 237 +++++++++++++++++++++++
>  doc/guides/nics/features/ark.ini            |  15 ++
>  doc/guides/nics/index.rst                   |   1 +
>  drivers/net/Makefile                        |   1 +
>  drivers/net/ark/Makefile                    |  63 +++++++
>  drivers/net/ark/ark_debug.h                 |  74 ++++++++
>  drivers/net/ark/ark_ethdev.c                | 281 ++++++++++++++++++++++++++++
>  drivers/net/ark/ark_ethdev.h                |  39 ++++
>  drivers/net/ark/ark_global.h                | 108 +++++++++++
>  drivers/net/ark/rte_pmd_ark_version.map     |   4 +
>  mk/rte.app.mk                               |   1 +
>  15 files changed, 845 insertions(+)
>  create mode 100644 doc/guides/nics/ark.rst
>  create mode 100644 doc/guides/nics/features/ark.ini
>  create mode 100644 drivers/net/ark/Makefile
>  create mode 100644 drivers/net/ark/ark_debug.h
>  create mode 100644 drivers/net/ark/ark_ethdev.c
>  create mode 100644 drivers/net/ark/ark_ethdev.h
>  create mode 100644 drivers/net/ark/ark_global.h
>  create mode 100644 drivers/net/ark/rte_pmd_ark_version.map
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 0c78b58..8043d75 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -278,6 +278,14 @@ M: Evgeny Schemeilin <evgenys@amazon.com>
>  F: drivers/net/ena/
>  F: doc/guides/nics/ena.rst
>  
> +Atomic Rules ark

Should prefer uppercase "ARK" here?

> +M: Shepard Siegel <shepard.siegel@atomicrules.com>
> +M: Ed Czeck       <ed.czeck@atomicrules.com>
> +M: John Miller    <john.miller@atomicrules.com>
> +F: /drivers/net/ark/

Can you please drop the leading "/". There is a script
"check-maintainers.sh", which is broken with that.

> +F: doc/guides/nics/ark.rst
> +F: doc/guides/nics/features/ark.ini
> +
>  Broadcom bnxt
>  M: Stephen Hurd <stephen.hurd@broadcom.com>
>  M: Ajit Khaparde <ajit.khaparde@broadcom.com>
> diff --git a/config/common_base b/config/common_base
> index 37aa1e1..0916c44 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -353,6 +353,17 @@ CONFIG_RTE_LIBRTE_QEDE_FW=""
>  CONFIG_RTE_LIBRTE_PMD_AF_PACKET=n
>  
>  #
> +# Compile ARK PMD
> +#
> +CONFIG_RTE_LIBRTE_ARK_PMD=y
> +CONFIG_RTE_LIBRTE_ARK_PAD_TX=y
> +CONFIG_RTE_LIBRTE_ARK_DEBUG_RX=n
> +CONFIG_RTE_LIBRTE_ARK_DEBUG_TX=n
> +CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS=n
> +CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE=n
> +
> +

Extra line

> +#
>  # Compile the TAP PMD
>  # It is enabled by default for Linux only.
>  #
> diff --git a/config/defconfig_arm-armv7a-linuxapp-gcc b/config/defconfig_arm-armv7a-linuxapp-gcc
> index d9bd2a8..6d2b5e0 100644
> --- a/config/defconfig_arm-armv7a-linuxapp-gcc
> +++ b/config/defconfig_arm-armv7a-linuxapp-gcc
> @@ -61,6 +61,7 @@ CONFIG_RTE_SCHED_VECTOR=n
>  
>  # cannot use those on ARM
>  CONFIG_RTE_KNI_KMOD=n
> +CONFIG_RTE_LIBRTE_ARK_PMD=n
>  CONFIG_RTE_LIBRTE_EM_PMD=n
>  CONFIG_RTE_LIBRTE_IGB_PMD=n
>  CONFIG_RTE_LIBRTE_CXGBE_PMD=n
> diff --git a/config/defconfig_ppc_64-power8-linuxapp-gcc b/config/defconfig_ppc_64-power8-linuxapp-gcc
> index 35f7fb6..89bc396 100644
> --- a/config/defconfig_ppc_64-power8-linuxapp-gcc
> +++ b/config/defconfig_ppc_64-power8-linuxapp-gcc
> @@ -48,6 +48,7 @@ CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
>  
>  # Note: Initially, all of the PMD drivers compilation are turned off on Power
>  # Will turn on them only after the successful testing on Power
> +CONFIG_RTE_LIBRTE_ARK_PMD=n

Is it not tested or known that it is not supported?

>  CONFIG_RTE_LIBRTE_IXGBE_PMD=n
>  CONFIG_RTE_LIBRTE_I40E_PMD=n
>  CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
> diff --git a/doc/guides/nics/ark.rst b/doc/guides/nics/ark.rst
> new file mode 100644
> index 0000000..72fb8d6
> --- /dev/null
> +++ b/doc/guides/nics/ark.rst
> @@ -0,0 +1,237 @@
> +.. BSD LICENSE
> +
> +    Copyright (c) 2015-2017 Atomic Rules LLC
> +    All rights reserved.
> +
> +    Redistribution and use in source and binary forms, with or without
> +    modification, are permitted provided that the following conditions
> +    are met:
> +
> +    * Redistributions of source code must retain the above copyright
> +    notice, this list of conditions and the following disclaimer.
> +    * Redistributions in binary form must reproduce the above copyright
> +    notice, this list of conditions and the following disclaimer in
> +    the documentation and/or other materials provided with the
> +    distribution.
> +    * Neither the name of Atomic Rules LLC nor the names of its
> +    contributors may be used to endorse or promote products derived
> +    from this software without specific prior written permission.
> +
> +    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +ARK Poll Mode Driver
> +====================
> +
> +The ARK PMD is a DPDK poll-mode driver for the Atomic Rules Arkville
> +(ARK) family of devices.
> +
> +More information can be found at the `Atomic Rules website
> +<http://atomicrules.com>`_.
> +
> +Overview
> +--------
> +
> +The Atomic Rules Arkville product is DPDK and AXI compliant product
> +that marshals packets across a PCIe conduit between host DPDK mbufs and
> +FPGA AXI streams.
> +
> +The ARK PMD, and the spirit of the overall Arkville product,
> +has been to take the DPDK API/ABI as a fixed specification;
> +then implement much of the business logic in FPGA RTL circuits.
> +The approach of *working backwards* from the DPDK API/ABI and having
> +the GPP host software *dictate*, while the FPGA hardware *copes*,
> +results in significant performance gains over a naive implementation.
> +
> +While this document describes the ARK PMD software, it is helpful to
> +understand what the FPGA hardware is and is not. The Arkville RTL
> +component provides a single PCIe Physical Function (PF) supporting
> +some number of RX/Ingress and TX/Egress Queues. The ARK PMD controls
> +the Arkville core through a dedicated opaque Core BAR (CBAR).
> +To allow users full freedom for their own FPGA application IP,
> +an independent FPGA Application BAR (ABAR) is provided.
> +
> +One popular way to imagine Arkville's FPGA hardware aspect is as the
> +FPGA PCIe-facing side of a so-called Smart NIC. The Arkville core does
> +not contain any MACs, and is link-speed independent, as well as
> +agnostic to the number of physical ports the application chooses to
> +use. The ARK driver exposes the familiar PMD interface to allow packet
> +movement to and from mbufs across multiple queues.
> +
> +However FPGA RTL applications could contain a universe of added
> +functionality that an Arkville RTL core does not provide or can
> +not anticipate. To allow for this expectation of user-defined
> +innovation, the ARK PMD provides a dynamic mechanism of adding
> +capabilities without having to modify the ARK PMD.
> +
> +The ARK PMD is intended to support all instances of the Arkville
> +RTL Core, regardless of configuration, FPGA vendor, or target
> +board. While specific capabilities such as number of physical
> +hardware queue-pairs are negotiated; the driver is designed to
> +remain constant over a broad and extendable feature set.
> +
> +Intentionally, Arkville by itself DOES NOT provide common NIC
> +capabilities such as offload or receive-side scaling (RSS).
> +These capabilities would be viewed as a gate-level "tax" on
> +Green-box FPGA applications that do not require such function.
> +Instead, they can be added as needed with essentially no
> +overhead to the FPGA Application.
> +
> +Data Path Interface
> +-------------------
> +
> +Ingress RX and Egress TX operation is by the nominal DPDK API .
> +The driver supports single-port, multi-queue for both RX and TX.
> +
> +Refer to ``ark_ethdev.h`` for the list of supported methods to
> +act upon RX and TX Queues.
> +
> +Configuration Information
> +-------------------------
> +
> +**DPDK Configuration Parameters**
> +
> +  The following configuration options are available for the ARK PMD:
> +
> +   * **CONFIG_RTE_LIBRTE_ARK_PMD** (default y): Enables or disables inclusion
> +     of the ARK PMD driver in the DPDK compilation.

Missing "CONFIG_RTE_LIBRTE_ARK_PAD_TX"

> +
> +   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_RX** (default n): Enables or disables debug
> +     logging and internal checking of RX ingress logic within the ARK PMD driver.
> +
> +   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TX** (default n): Enables or disables debug
> +     logging and internal checking of TX egress logic within the ARK PMD driver.
> +
> +   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS** (default n): Enables or disables debug
> +     logging of detailed packet and performance statistics gathered in
> +     the PMD and FPGA.
> +
> +   * **CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE** (default n): Enables or disables debug
> +     logging of detailed PMD events and status.
> +
> +

Can you also please document the device arguments in this file?

> +Building DPDK
> +-------------
> +
> +See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
> +instructions on how to build DPDK.
> +
> +By default the ARK PMD library will be built into the DPDK library.
> +
> +For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
> +documentation that comes with DPDK suite <linux_gsg>`.
> +
> +Supported ARK RTL PCIe Instances
> +--------------------------------
> +
> +ARK PMD supports the following Arkville RTL PCIe instances including:
> +
> +* ``1d6c:100d`` - AR-ARKA-FX0 [Arkville 32B DPDK Data Mover]
> +* ``1d6c:100e`` - AR-ARKA-FX1 [Arkville 64B DPDK Data Mover]
> +
> +Supported Operating Systems
> +---------------------------
> +
> +Any Linux distribution fulfilling the conditions described in ``System Requirements``
> +section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK Release Notes*.
> +
> +Supported Features
> +------------------
> +
> +* Dynamic ARK PMD extensions
> +* Multiple receive and transmit queues
> +* Jumbo frames up to 9K
> +* Hardware Statistics
> +
> +Unsupported Features
> +--------------------
> +
> +Features that may be part of, or become part of, the Arkville RTL IP that are
> +not currently supported or exposed by the ARK PMD include:
> +
> +* PCIe SR-IOV Virtual Functions (VFs)
> +* Arkville's Packet Generator Control and Status
> +* Arkville's Packet Director Control and Status
> +* Arkville's Packet Checker Control and Status
> +* Arkville's Timebase Management
> +
> +Pre-Requisites
> +--------------
> +
> +#. Prepare the system as recommended by DPDK suite.  This includes environment
> +   variables, hugepages configuration, tool-chains and configuration
> +
> +#. Insert igb_uio kernel module using the command 'modprobe igb_uio'
> +
> +#. Bind the intended ARK device to igb_uio module
> +
> +At this point the system should be ready to run DPDK applications. Once the
> +application runs to completion, the ARK PMD can be detached from igb_uio if necessary.
> +
> +Usage Example
> +-------------
> +
> +This section demonstrates how to launch **testpmd** with Atomic Rules ARK
> +devices managed by librte_pmd_ark.
> +
> +#. Load the kernel modules:
> +
> +   .. code-block:: console
> +
> +      modprobe uio
> +      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
> +
> +   .. note::
> +
> +      The ARK PMD driver depends upon the igb_uio user space I/O kernel module
> +
> +#. Mount and request huge pages:
> +
> +   .. code-block:: console
> +
> +      mount -t hugetlbfs nodev /mnt/huge
> +      echo 256 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
> +
> +#. Bind UIO driver to ARK device at 0000:01:00.0 (using dpdk-devbind.py):
> +
> +   .. code-block:: console
> +
> +      ./usertools/dpdk-devbind.py --bind=igb_uio 0000:01:00.0
> +
> +   .. note::
> +
> +      The last argument to dpdk-devbind.py is the 4-tuple that indentifies a specific PCIe
> +      device. You can use lspci -d 1d6c: to indentify all Atomic Rules devices in the system,
> +      and thus determine the correct 4-tuple argument to dpdk-devbind.py
> +
> +#. Start testpmd with basic parameters:
> +
> +   .. code-block:: console
> +
> +      ./x86_64-native-linuxapp-gcc/app/testpmd -l 0-3 -n 4 -- -i
> +
> +   Example output:
> +
> +   .. code-block:: console
> +
> +      [...]
> +      EAL: PCI device 0000:01:00.0 on NUMA socket -1
> +      EAL:   probe driver: 1d6c:100e rte_ark_pmd
> +      EAL:   PCI memory mapped at 0x7f9b6c400000
> +      PMD: eth_ark_dev_init(): Initializing 0:2:0.1
> +      ARKP PMD CommitID: 378f3a67
> +      Configuring Port 0 (socket 0)
> +      Port 0: DC:3C:F6:00:00:01
> +      Checking link statuses...
> +      Port 0 Link Up - speed 100000 Mbps - full-duplex
> +      Done
> +      testpmd>
> diff --git a/doc/guides/nics/features/ark.ini b/doc/guides/nics/features/ark.ini
> new file mode 100644
> index 0000000..dc8a0e2
> --- /dev/null
> +++ b/doc/guides/nics/features/ark.ini
> @@ -0,0 +1,15 @@
> +;
> +; Supported features of the 'ark' poll mode driver.
> +;
> +; Refer to default.ini for the full list of available PMD features.
> +;
> +[Features]
> +Queue start/stop     = Y
> +Jumbo frame          = Y
> +Scattered Rx         = Y
> +Basic stats          = Y
> +Stats per queue      = Y
> +FW version           = Y

Features can be added with the patch that adds functionality. I believe
above features not supported with current patch.

> +Linux UIO            = Y
> +x86-64               = Y
> +Usage doc            = Y
> diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
> index 87f9334..381d82c 100644
> --- a/doc/guides/nics/index.rst
> +++ b/doc/guides/nics/index.rst
> @@ -36,6 +36,7 @@ Network Interface Controller Drivers
>      :numbered:
>  
>      overview
> +    ark
>      bnx2x
>      bnxt
>      cxgbe
> diff --git a/drivers/net/Makefile b/drivers/net/Makefile
> index a16f25e..ea9868b 100644
> --- a/drivers/net/Makefile
> +++ b/drivers/net/Makefile
> @@ -32,6 +32,7 @@
>  include $(RTE_SDK)/mk/rte.vars.mk
>  
>  DIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += af_packet
> +DIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark
>  DIRS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD) += bnx2x
>  DIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += bonding
>  DIRS-$(CONFIG_RTE_LIBRTE_CXGBE_PMD) += cxgbe
> diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
> new file mode 100644
> index 0000000..217bd34
> --- /dev/null
> +++ b/drivers/net/ark/Makefile
> @@ -0,0 +1,63 @@
> +# BSD LICENSE
> +#
> +# Copyright (c) 2015-2017 Atomic Rules LLC
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +# * Redistributions of source code must retain the above copyright
> +#   notice, this list of conditions and the following disclaimer.
> +# * Redistributions in binary form must reproduce the above copyright
> +#   notice, this list of conditions and the following disclaimer in
> +#   the documentation and/or other materials provided with the
> +#   distribution.
> +# * Neither the name of copyright holder nor the names of its
> +#   contributors may be used to endorse or promote products derived
> +#   from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +#
> +# library name
> +#
> +LIB = librte_pmd_ark.a
> +
> +CFLAGS += -O3 -I./
> +CFLAGS += $(WERROR_FLAGS)
> +
> +EXPORT_MAP := rte_pmd_ark_version.map
> +
> +LIBABIVER := 1
> +
> +#
> +# all source are stored in SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD)

No need to put config option to comment, SRCS-y looks more proper.

> +#
> +
> +SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_ethdev.c
> +
> +
> +# this lib depends upon:
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_mbuf
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_ether
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_kvargs
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_eal
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/librte_mempool

> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/libpthread
> +DEPDIRS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += lib/libdl

DEPDIRS is for internal library dependencies. Please use LDLIBS for
external dependencies, like:

LDLIBS += -lpthread
LDLIBS += -ldl

> +
> +
> +include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/drivers/net/ark/ark_debug.h b/drivers/net/ark/ark_debug.h
> new file mode 100644
> index 0000000..a108c28
> --- /dev/null
> +++ b/drivers/net/ark/ark_debug.h
> @@ -0,0 +1,74 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright (c) 2015-2017 Atomic Rules LLC
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of copyright holder nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _ARK_DEBUG_H_
> +#define _ARK_DEBUG_H_
> +
> +#include <inttypes.h>
> +#include <rte_log.h>
> +
> +/* Format specifiers for string data pairs */
> +#define ARK_SU32  "\n\t%-20s    %'20" PRIu32
> +#define ARK_SU64  "\n\t%-20s    %'20" PRIu64
> +#define ARK_SU64X "\n\t%-20s    %#20" PRIx64
> +#define ARK_SPTR  "\n\t%-20s    %20p"
> +
> +#define ARK_TRACE_ON(fmt, ...) \
> +	PMD_DRV_LOG(ERR, fmt, ##__VA_ARGS__)
> +
> +#define ARK_TRACE_OFF(fmt, ...) \
> +	do {if (0) PMD_DRV_LOG(ERR, fmt, ##__VA_ARGS__); } while (0)
> +
> +/* Debug macro for reporting Packet stats */
> +#ifdef RTE_LIBRTE_ARK_DEBUG_STATS
> +#define ARK_DEBUG_STATS(fmt, ...) ARK_TRACE_ON(fmt, ##__VA_ARGS__)
> +#else
> +#define ARK_DEBUG_STATS(fmt, ...)  ARK_TRACE_OFF(fmt, ##__VA_ARGS__)
> +#endif
> +
> +/* Debug macro for tracing full behavior*/
> +#ifdef RTE_LIBRTE_ARK_DEBUG_TRACE
> +#define ARK_DEBUG_TRACE(fmt, ...)  ARK_TRACE_ON(fmt, ##__VA_ARGS__)
> +#else
> +#define ARK_DEBUG_TRACE(fmt, ...)  ARK_TRACE_OFF(fmt, ##__VA_ARGS__)
> +#endif
> +
> +#ifdef ARK_STD_LOG

How this define passed? Should it be something like
RTE_LIBRTE_ARK_DEBUG_DRIVER config option?

> +#define PMD_DRV_LOG(level, fmt, args...) \
> +	fprintf(stderr, fmt, args)

It is possible to use rte log functions instead of fprintf to stderr.

> +#else
> +#define PMD_DRV_LOG(level, fmt, args...) \
> +	RTE_LOG(level, PMD, "%s(): " fmt, __func__, ## args)
> +#endif
> +
> +#endif

CONFIG_RTE_LIBRTE_ARK_DEBUG_RX, CONFIG_RTE_LIBRTE_ARK_DEBUG_TX not used,
if so can be removed.

> diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c
> new file mode 100644
> index 0000000..124b73c
> --- /dev/null
> +++ b/drivers/net/ark/ark_ethdev.c
> @@ -0,0 +1,281 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright (c) 2015-2017 Atomic Rules LLC
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of copyright holder nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#include <unistd.h>
> +#include <sys/stat.h>
> +#include <dlfcn.h>
> +
> +#include <rte_kvargs.h>
> +#include <rte_vdev.h>
> +
> +#include "ark_global.h"
> +#include "ark_debug.h"
> +#include "ark_ethdev.h"
> +
> +/*  Internal prototypes */
> +static int eth_ark_check_args(const char *params);
> +static int eth_ark_dev_init(struct rte_eth_dev *dev);
> +static int eth_ark_dev_uninit(struct rte_eth_dev *eth_dev);
> +static int eth_ark_dev_configure(struct rte_eth_dev *dev);
> +static void eth_ark_dev_info_get(struct rte_eth_dev *dev,
> +				 struct rte_eth_dev_info *dev_info);
> +
> +
> +#define ARK_DEV_TO_PCI(eth_dev)			\
> +	RTE_DEV_TO_PCI((eth_dev)->device)
> +
> +#define ARK_MAX_ARG_LEN 256
> +static uint32_t pkt_dir_v;
> +static char pkt_gen_args[ARK_MAX_ARG_LEN];
> +static char pkt_chkr_args[ARK_MAX_ARG_LEN];
> +
> +#define ARK_PKTGEN_ARG "Pkt_gen"
> +#define ARK_PKTCHKR_ARG "Pkt_chkr"
> +#define ARK_PKTDIR_ARG "Pkt_dir"

Is it possible to add one line comments to device arguments. For example
what "Pkt_dir" (packet director) is for?

> +
> +static const char * const valid_arguments[] = {
> +	ARK_PKTGEN_ARG,
> +	ARK_PKTCHKR_ARG,
> +	ARK_PKTDIR_ARG,
> +	"iface",

Why not make this one too a define?

> +	NULL
> +};
> +
> +#define MAX_ARK_PHYS 16
> +struct ark_adapter *gark[MAX_ARK_PHYS];
> +
> +static const struct rte_pci_id pci_id_ark_map[] = {
> +	{RTE_PCI_DEVICE(0x1d6c, 0x100d)},
> +	{RTE_PCI_DEVICE(0x1d6c, 0x100e)},
> +	{.vendor_id = 0, /* sentinel */ },
> +};
> +
> +static struct eth_driver rte_ark_pmd = {
> +	.pci_drv = {
> +		.probe = rte_eth_dev_pci_probe,
> +		.remove = rte_eth_dev_pci_remove,
> +		.id_table = pci_id_ark_map,
> +		.drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC
> +	},
> +	.eth_dev_init = eth_ark_dev_init,
> +	.eth_dev_uninit = eth_ark_dev_uninit,
> +	.dev_private_size = sizeof(struct ark_adapter),
> +};
> +
> +static const struct eth_dev_ops ark_eth_dev_ops = {
> +	.dev_configure = eth_ark_dev_configure,
> +	.dev_infos_get = eth_ark_dev_info_get,
> +

Extra line.

> +};
> +
> +

Extra line.

> +static int
> +eth_ark_dev_init(struct rte_eth_dev *dev __rte_unused)
> +{
> +	return -1;					/* STUB */

You may want to set ark_eth_dev_ops here, since they already implemented.

And for proper .dev_infos_get implementation, you may want to have [1] here:

[1]
rte_eth_copy_pci_info(eth_dev, pci_dev);
eth_dev->data->dev_flags |= RTE_ETH_DEV_DETACHABLE;

Also you may want to parse device arguments in this stage.

> +}
> +
> +
> +static int
> +eth_ark_dev_uninit(struct rte_eth_dev *dev)
> +{
> +	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> +		return 0;
> +
> +	dev->dev_ops = NULL;
> +	dev->rx_pkt_burst = NULL;
> +	dev->tx_pkt_burst = NULL;
> +	return 0;
> +}
> +
> +static int
> +eth_ark_dev_configure(struct rte_eth_dev *dev __rte_unused)
> +{
> +	ARK_DEBUG_TRACE("ARKP: In %s\n", __func__);
> +	return 0;
> +}
> +
> +static void
> +eth_ark_dev_info_get(struct rte_eth_dev *dev,
> +		     struct rte_eth_dev_info *dev_info)
> +{
> +	/* device specific configuration */
> +	memset(dev_info, 0, sizeof(*dev_info));

memset not required, since already done by ethdev abstraction layer,
specially desc_lim values already overwritten below.

> +
> +	dev_info->max_rx_pktlen = (16 * 1024) - 128;
> +	dev_info->min_rx_bufsize = 1024;

Using some macros instead of hardcoded values helps to understand values.

> +	dev_info->rx_offload_capa = 0;
> +	dev_info->tx_offload_capa = 0;
> +
> +	dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
> +		.nb_max = 4096 * 4,
> +		.nb_min = 512,	/* HW Q size for RX */
> +		.nb_align = 2,};
> +
> +	dev_info->tx_desc_lim = (struct rte_eth_desc_lim) {
> +		.nb_max = 4096 * 4,
> +		.nb_min = 256,	/* HW Q size for TX */
> +		.nb_align = 2,};
> +
> +	dev_info->rx_offload_capa = 0;
> +	dev_info->tx_offload_capa = 0;

Dublication, please check ~10 lines above.
Also not required to set 0 at all because of memset.

> +
> +	/* ARK PMD supports all line rates, how do we indicate that here ?? */
> +	dev_info->speed_capa = (ETH_LINK_SPEED_1G |
> +				ETH_LINK_SPEED_10G |
> +				ETH_LINK_SPEED_25G |
> +				ETH_LINK_SPEED_40G |
> +				ETH_LINK_SPEED_50G |
> +				ETH_LINK_SPEED_100G);
> +	dev_info->pci_dev = ARK_DEV_TO_PCI(dev);
> +	dev_info->driver_name = dev->data->drv_name;

setting driver_name not required, ethdev layer will overwrite this value.

And to have driver_name correct, rte_eth_copy_pci_info() should be
called, please check above [1].

> +}
> +
> +
> +static inline int
> +process_pktdir_arg(const char *key, const char *value,
> +		   void *extra_args __rte_unused)
> +{
> +	ARK_DEBUG_TRACE("In process_pktdir_arg, key = %s, value = %s\n",
> +			key, value);

The general usage of DEBUG_TRACE is providing backtrace log, function
enterance / exit informations. I guess, that is why it has been
controlled by different config option.
Here what you need looks like regular debugging functions, PMD_DRV_LOG /
RTE_LOG variant.

> +	pkt_dir_v = strtol(value, NULL, 16);
> +	ARK_DEBUG_TRACE("pkt_dir_v = 0x%x\n", pkt_dir_v);
> +	return 0;
> +}
> +
> +static inline int
> +process_file_args(const char *key, const char *value, void *extra_args)
> +{
> +	ARK_DEBUG_TRACE("**** IN process_pktgen_arg, key = %s, value = %s\n",
> +			key, value);
> +	char *args = (char *)extra_args;
> +
> +	/* Open the configuration file */
> +	FILE *file = fopen(value, "r");
> +	char line[256];
> +	int first = 1;
> +
> +	while (fgets(line, sizeof(line), file)) {
> +		/* ARK_DEBUG_TRACE("%s\n", line); */

Please remove dead code.

> +		if (first) {
> +			strncpy(args, line, ARK_MAX_ARG_LEN);

Can this overflow args variable? Any way to prevent possible crash?

> +			first = 0;
> +		} else {
> +			strncat(args, line, ARK_MAX_ARG_LEN);
> +		}
> +	}
> +	ARK_DEBUG_TRACE("file = %s\n", args);
> +	fclose(file);
> +	return 0;
> +}
> +
> +static int
> +eth_ark_check_args(const char *params)
> +{
> +	struct rte_kvargs *kvlist;
> +	unsigned int k_idx;
> +	struct rte_kvargs_pair *pair = NULL;
> +
> +	kvlist = rte_kvargs_parse(params, valid_arguments);
> +	if (kvlist == NULL)
> +		return 0;
> +
> +	pkt_gen_args[0] = 0;
> +	pkt_chkr_args[0] = 0;
> +
> +	for (k_idx = 0; k_idx < kvlist->count; k_idx++) {
> +		pair = &kvlist->pairs[k_idx];
> +		ARK_DEBUG_TRACE("**** Arg passed to PMD = %s:%s\n", pair->key,
> +				pair->value);
> +	}
> +
> +	if (rte_kvargs_process(kvlist,
> +			       ARK_PKTDIR_ARG,
> +			       &process_pktdir_arg,
> +			       NULL) != 0) {
> +		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTDIR_ARG);
> +	}
> +
> +	if (rte_kvargs_process(kvlist,
> +			       ARK_PKTGEN_ARG,
> +			       &process_file_args,
> +			       pkt_gen_args) != 0) {
> +		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTGEN_ARG);
> +	}
> +
> +	if (rte_kvargs_process(kvlist,
> +			       ARK_PKTCHKR_ARG,
> +			       &process_file_args,
> +			       pkt_chkr_args) != 0) {
> +		PMD_DRV_LOG(ERR, "Unable to parse arg %s\n", ARK_PKTCHKR_ARG);
> +	}

Not processing "iface" device argument?

> +
> +	ARK_DEBUG_TRACE("INFO: packet director set to 0x%x\n", pkt_dir_v);
> +
> +	return 1;
> +}
> +
> +
> +/*
> + * PCIE

Can you elaborate this comment?

> + */
> +static int
> +pmd_ark_probe(const char *name, const char *params)
> +{
> +	RTE_LOG(INFO, PMD, "Initializing pmd_ark for %s params = %s\n", name,
> +		params);
> +
> +	/* Parse off the v index */
> +
> +	eth_ark_check_args(params);
> +	return 0;
> +}
> +
> +static int
> +pmd_ark_remove(const char *name)
> +{
> +	RTE_LOG(INFO, PMD, "Closing ark %s ethdev on numa socket %u\n", name,
> +		rte_socket_id());
> +	return 1;
> +}
> +
> +static struct rte_vdev_driver pmd_ark_drv = {
> +	.probe = pmd_ark_probe,
> +	.remove = pmd_ark_remove,
> +};

Sorry, I am confused here.
Why both virtual and physical initialization routine exists together?
This PMD for physical PCI device, right?

> +
> +RTE_PMD_REGISTER_VDEV(net_ark, pmd_ark_drv);
> +RTE_PMD_REGISTER_ALIAS(net_ark, eth_ark);
> +RTE_PMD_REGISTER_PCI(eth_ark, rte_ark_pmd.pci_drv);
> +RTE_PMD_REGISTER_KMOD_DEP(net_ark, "* igb_uio | uio_pci_generic ");
> +RTE_PMD_REGISTER_PCI_TABLE(eth_ark, pci_id_ark_map);

Can add RTE_PMD_REGISTER_PARAM_STRING macro.

> diff --git a/drivers/net/ark/ark_ethdev.h b/drivers/net/ark/ark_ethdev.h
> new file mode 100644
> index 0000000..08d7fb1
> --- /dev/null
> +++ b/drivers/net/ark/ark_ethdev.h
> @@ -0,0 +1,39 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright (c) 2015-2017 Atomic Rules LLC
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of copyright holder nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _ARK_ETHDEV_H_
> +#define _ARK_ETHDEV_H_
> +
> +/* STUB */
> +
> +#endif
> diff --git a/drivers/net/ark/ark_global.h b/drivers/net/ark/ark_global.h
> new file mode 100644
> index 0000000..7cd62d5
> --- /dev/null
> +++ b/drivers/net/ark/ark_global.h
> @@ -0,0 +1,108 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright (c) 2015-2017 Atomic Rules LLC
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of copyright holder nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _ARK_GLOBAL_H_
> +#define _ARK_GLOBAL_H_
> +
> +#include <time.h>
> +#include <assert.h>
> +
> +#include <rte_mbuf.h>
> +#include <rte_ethdev.h>
> +#include <rte_malloc.h>
> +#include <rte_memcpy.h>
> +#include <rte_string_fns.h>
> +#include <rte_cycles.h>
> +#include <rte_kvargs.h>
> +#include <rte_dev.h>
> +#include <rte_version.h>
> +
> +#define ETH_ARK_ARG_MAXLEN	64
> +#define ARK_SYSCTRL_BASE  0x0
> +#define ARK_PKTGEN_BASE   0x10000
> +#define ARK_MPU_RX_BASE   0x20000
> +#define ARK_UDM_BASE      0x30000
> +#define ARK_MPU_TX_BASE   0x40000
> +#define ARK_DDM_BASE      0x60000
> +#define ARK_CMAC_BASE     0x80000
> +#define ARK_PKTDIR_BASE   0xa0000
> +#define ARK_PKTCHKR_BASE  0x90000
> +#define ARK_RCPACING_BASE 0xb0000
> +#define ARK_EXTERNAL_BASE 0x100000
> +#define ARK_MPU_QOFFSET   0x00100
> +#define ARK_MAX_PORTS     8
> +
> +#define offset8(n)     n
> +#define offset16(n)   ((n) / 2)
> +#define offset32(n)   ((n) / 4)
> +#define offset64(n)   ((n) / 8)
> +
> +/*
> + * Structure to store private data for each PF/VF instance.
> + */
> +#define def_ptr(type, name) \
> +	union type {		   \
> +		uint64_t *t64;	   \
> +		uint32_t *t32;	   \
> +		uint16_t *t16;	   \
> +		uint8_t  *t8;	   \
> +		void     *v;	   \
> +	} name
> +
> +struct ark_port {
> +	struct rte_eth_dev *eth_dev;
> +	int id;
> +};
> +
> +struct ark_adapter {
> +	/* User extension private data */
> +	void *user_data;
> +
> +	struct ark_port port[ARK_MAX_PORTS];
> +	int num_ports;
> +
> +	/* Common for both PF and VF */
> +	struct rte_eth_dev *eth_dev;
> +
> +	void *d_handle;
> +
> +	/* Our Bar 0 */
> +	uint8_t *bar0;
> +
> +	/* Application Bar */
> +	uint8_t *a_bar;
> +};
> +
> +typedef uint32_t *ark_t;
> +
> +#endif
> diff --git a/drivers/net/ark/rte_pmd_ark_version.map b/drivers/net/ark/rte_pmd_ark_version.map
> new file mode 100644
> index 0000000..7f84780
> --- /dev/null
> +++ b/drivers/net/ark/rte_pmd_ark_version.map
> @@ -0,0 +1,4 @@
> +DPDK_2.0 {

DPDK_17.05

> +	 local: *;
> +
> +};
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> index 0e0b600..da23898 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -104,6 +104,7 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
>  # plugins (link only if static libraries)
>  
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET)  += -lrte_pmd_af_packet
> +_LDLIBS-$(CONFIG_RTE_LIBRTE_ARK_PMD)        += -lrte_pmd_ark
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_BNX2X_PMD)      += -lrte_pmd_bnx2x -lz
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_BNXT_PMD)       += -lrte_pmd_bnxt
>  _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_BOND)       += -lrte_pmd_bond
> 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub
  2017-03-22 18:16 ` [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ferruh Yigit
@ 2017-03-23  1:09   ` Ed Czeck
  0 siblings, 0 replies; 13+ messages in thread
From: Ed Czeck @ 2017-03-23  1:09 UTC (permalink / raw)
  To: Ferruh Yigit; +Cc: dev

Thank you Furruh,
A new patch series has been uploaded.
Ed.

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2017-03-23  1:10 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-21 21:43 [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 2/7] net/ark: HW API part 1 of 3 Ed Czeck
2017-03-21 22:15   ` Stephen Hemminger
2017-03-22  0:13     ` Ed Czeck
2017-03-22  0:30       ` Stephen Hemminger
2017-03-22 14:36         ` Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 3/7] net/ark: HW API part 2 " Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 4/7] net/ark: HW API part 3 " Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 5/7] net/ark: Packet TX support initial version Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 6/7] net/ark: Packet RX " Ed Czeck
2017-03-21 21:43 ` [dpdk-dev] [PATCH v3 7/7] net/ark: Arkville PMD component integration Ed Czeck
2017-03-22 18:16 ` [dpdk-dev] [PATCH v3 1/7] net/ark: PMD for Atomic Rules Arkville driver stub Ferruh Yigit
2017-03-23  1:09   ` Ed Czeck

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).