* [dpdk-dev] [PATCH v2 0/7] support switch management
@ 2019-12-11 9:51 Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
` (7 more replies)
0 siblings, 8 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:51 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Xiaojun Liu (7):
net/fm10k: add i2c sbus registers definition
net/fm10k: add some modules of port
net/fm10k: add config ffu statistics support
net/fm10k: add flow and switch management
net/fm10k: add switch initialization
net/fm10k: add mirror and filter ctrl
net/fm10k: add dpdk port mapping
drivers/net/fm10k/Makefile | 22 +
drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
drivers/net/fm10k/switch/fm10k_config.h | 171 ++
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2202 ++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
drivers/net/fm10k/switch/fm10k_sm.h | 78 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
drivers/net/fm10k/switch/fm10k_switch.c | 2562 +++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
26 files changed, 17188 insertions(+), 34 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 15:48 ` Jerin Jacob
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 2/7] net/fm10k: add some modules of port Xiaojun Liu
` (6 subsequent siblings)
7 siblings, 2 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
To support switch management, add the following files:
Add fm10k/switch/fm10k_debug.h(define log Macros).
Add fm10k/switch/fm10k_regs.h(define all the registers).
Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
Add fm10k/switch/fm10k_i2c.c(support I2C access).
Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
Add fm10k/switch/fm10k_sbus.c(support SBUS access).
and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
add fm10k_i2c.c and fm10k_sbus.c).
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 13 +
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2202 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 ++++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_switch.h | 336 +++++
8 files changed, 3266 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 55e9cd5..aca6bfb 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -11,6 +11,9 @@ LIB = librte_pmd_fm10k.a
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
CFLAGS += -DALLOW_EXPERIMENTAL_API
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+CFLAGS += -DENABLE_FM10K_MANAGEMENT
+endif
EXPORT_MAP := rte_pmd_fm10k_version.map
@@ -62,6 +65,10 @@ $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER))
VPATH += $(SRCDIR)/base
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+VPATH += $(SRCDIR)/switch
+endif
+
#
# all source are stored in SRCS-y
# base driver is based on the package of cid-fm10k.2017.01.24.tar.gz
@@ -75,6 +82,12 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_common.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_mbx.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
+
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+endif
+
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/fm10k/switch/fm10k_debug.h b/drivers/net/fm10k/switch/fm10k_debug.h
new file mode 100644
index 0000000..6bf72a8
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_debug.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_DEBUG_H_
+#define _FM10K_DEBUG_H_
+
+
+#define FM10K_SW_ERR(...) PMD_INIT_LOG(ERR, __VA_ARGS__)
+#define FM10K_SW_INFO(...) PMD_INIT_LOG(INFO, __VA_ARGS__)
+#define FM10K_SW_TRACE(...) PMD_INIT_LOG(DEBUG, __VA_ARGS__)
+
+#define FM10K_SW_ASSERT(...) do {} while (0)
+
+#define FM10K_SW_STATS_TRACE_ENABLE 1
+#define FM10K_SW_FFU_CONF_TRACE_ENABLE 0
+#define FM10K_SW_MIRROR_TRACE_ENABLE 0
+
+#endif /* _FM10K_DEBUG_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.c b/drivers/net/fm10k/switch/fm10k_i2c.c
new file mode 100644
index 0000000..28b0c34
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.c
@@ -0,0 +1,310 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+
+static void fm10k_i2c_init(struct fm10k_i2c *);
+
+struct fm10k_i2c *
+fm10k_i2c_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_i2c *i2c;
+
+ FM10K_SW_TRACE("i2c: attaching");
+
+ i2c = (struct fm10k_i2c *)rte_zmalloc("fm10k_i2c",
+ sizeof(struct fm10k_i2c), 0);
+ if (i2c == NULL) {
+ FM10K_SW_INFO("i2c: failed to allocate context");
+ goto fail;
+ }
+
+ i2c->sw = sw;
+ pthread_mutex_init(&i2c->req_lock, NULL);
+ pthread_mutex_init(&i2c->bus_lock, NULL);
+ sem_init(&i2c->req_cv, 0, 0);
+
+ fm10k_i2c_init(i2c);
+
+ FM10K_SW_TRACE("i2c: attach successful");
+ return i2c;
+fail:
+ if (i2c)
+ fm10k_i2c_detach(i2c);
+ return NULL;
+}
+
+void
+fm10k_i2c_detach(struct fm10k_i2c *i2c)
+{
+ FM10K_SW_TRACE("i2c: detaching");
+
+ rte_free(i2c);
+}
+
+static void
+fm10k_i2c_init(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_device_info *cfg = sw->info;
+ uint32_t freq = FM10K_SW_I2C_CFG_DIVIDER_400_KHZ;
+ uint32_t data;
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ freq = FM10K_SW_I2C_CFG_DIVIDER_100_KHZ;
+
+ /* clear any pending interrupt */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ /* 400 KHz, master mode, unmask interrupt */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CFG);
+ data &= ~FM10K_SW_I2C_CFG_SLAVE_ENABLE;
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ FM10K_SW_REPLACE_REG_FIELD(data, I2C_CFG_DIVIDER, freq, data);
+ data &= ~FM10K_SW_I2C_CFG_INTERRUPT_MASK;
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CFG, data);
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ /* reset I2C */
+ fm10k_gpio_output_set(sw, 5, 1);
+}
+
+unsigned int
+fm10k_i2c_intr(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_i2c_req *req;
+ int i;
+ uint32_t data[3];
+ uint32_t ctrl;
+
+ req = i2c->cur_req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CTRL);
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ req->status = FM10K_SW_REG_FIELD(ctrl, I2C_CTRL_COMMAND_COMPLETED);
+
+ if ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->status == FM10K_SW_I2C_COMPLETION_NORMAL) {
+ for (i = 0; i < FM10K_SW_HOWMANY(req->read_len, 4, 4); i++)
+ data[i] = fm10k_read_switch_reg
+ (sw, FM10K_SW_I2C_DATA(i));
+
+ for (i = 0; i < req->read_len; i++)
+ req->msg[i] =
+ (data[i / 4] >> (24 - (i % 4) * 8)) & 0xff;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ sem_post(&i2c->req_cv);
+
+ return 1;
+}
+
+int
+fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ int i;
+ uint32_t ctrl;
+ uint32_t data[3];
+
+ if (((req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->write_len > FM10K_SW_I2C_MSG_MAX) ||
+ ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ ((req->read_len == 0 ||
+ req->read_len > FM10K_SW_I2C_MSG_MAX))))
+ return (-1);
+
+ FM10K_SW_TRACE("i2c: initiating command %u", req->cmd);
+
+ ctrl =
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_ADDR, req->addr << 1) |
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_COMMAND, req->cmd);
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) {
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_W, req->write_len);
+
+ data[0] = 0;
+ data[1] = 0;
+ data[2] = 0;
+
+ for (i = 0; i < req->write_len; i++)
+ data[i / 4] |= req->msg[i] << (24 - (i % 4) * 8);
+
+ for (i = 0; i < FM10K_SW_HOWMANY(req->write_len, 4, 4); i++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_I2C_DATA(i), data[i]);
+ }
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD)
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_R, req->read_len);
+
+ req->status = FM10K_SW_I2C_COMPLETION_RUNNING;
+ i2c->cur_req = req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* zero command field */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, 0);
+ /* initiate command */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, ctrl);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ while (req->status == FM10K_SW_I2C_COMPLETION_RUNNING)
+ sem_wait(&i2c->req_cv);
+
+ return 0;
+}
+
+int
+fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_RD;
+ req.read_len = 1;
+ req.msg[0] = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR_RD;
+ req.write_len = 1;
+ req.read_len = 1;
+ req.msg[0] = reg;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 1;
+ req.msg[0] = data;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 2;
+ req.msg[0] = data0;
+ req.msg[1] = data1;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.h b/drivers/net/fm10k/switch/fm10k_i2c.h
new file mode 100644
index 0000000..f835afe
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_I2C_H_
+#define _FM10K_SW_I2C_H_
+
+#include <semaphore.h>
+#include <pthread.h>
+#include "rte_spinlock.h"
+#include "fm10k_debug.h"
+
+#define FM10K_SW_I2C_MSG_MAX 12
+
+struct fm10k_i2c_req {
+ uint8_t addr; /* 7-bit address */
+ uint8_t cmd; /* FM10K_SW_I2C_COMMAND_* */
+ uint8_t write_len;
+ uint8_t read_len;
+ uint8_t status; /* FM10K_SW_I2C_COMPLETION_ */
+ uint8_t msg[FM10K_SW_I2C_MSG_MAX];
+};
+
+struct fm10k_i2c {
+ struct fm10k_switch *sw;
+ pthread_mutex_t bus_lock;
+ pthread_mutex_t req_lock;
+ sem_t req_cv;
+ struct fm10k_i2c_req *cur_req;
+};
+
+#define FM10K_SW_I2C_LOCK(i2c_) \
+ pthread_mutex_lock(&(i2c_)->bus_lock)
+#define FM10K_SW_I2C_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&(i2c_)->bus_lock)
+
+#define FM10K_SW_I2C_REQ_LOCK(i2c_) \
+ pthread_mutex_lock(&((i2c_)->req_lock))
+#define FM10K_SW_I2C_REQ_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&((i2c_)->req_lock))
+
+struct fm10k_i2c *fm10k_i2c_attach(struct fm10k_switch *sw);
+void fm10k_i2c_detach(struct fm10k_i2c *i2c);
+unsigned int fm10k_i2c_intr(struct fm10k_i2c *i2c);
+int fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req);
+int fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result);
+int fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result);
+int fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data);
+int fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1);
+int fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr);
+
+#endif /* _FM10K_SW_I2C_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_regs.h b/drivers/net/fm10k/switch/fm10k_regs.h
new file mode 100644
index 0000000..29bb95a
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_regs.h
@@ -0,0 +1,2202 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_REGS_H_
+#define _FM10K_REGS_H_
+
+#include <stdint.h>
+
+
+/* Convert a 32-bit word offset into a byte offset */
+#define FM10K_SW_REG_OFF(wo_) (wo_)
+#define FM10K_SW_MASK32(max_bit_, min_bit_) \
+ ((0xffffffffU << (min_bit_)) & \
+ (0xffffffffU >> (31 - (max_bit_))))
+#define FM10K_SW_REG_FIELD(r_, name_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) >> FM10K_SW_##name_##_lsb)
+#define FM10K_SW_REG_FIELD_IDX(r_, name_, i_, i1_, i2_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i_), \
+ FM10K_SW_##name_##_lsb(i2_))) >> \
+ FM10K_SW_##name_##_lsb(i3_))
+#define FM10K_SW_MAKE_REG_FIELD(name_, v_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb))
+#define FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_, i1_, i2_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb(i_)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) | \
+ FM10K_SW_MAKE_REG_FIELD(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_)))
+#define FM10K_SW_MASK64(max_bit_, min_bit_) \
+ ((0xffffffffffffffffULL << (min_bit_)) & \
+ (0xffffffffffffffffULL >> (63 - (max_bit_))))
+#define FM10K_SW_REG_FIELD64(r_, name_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) >> \
+ FM10K_SW_##name_##_lsb64)
+#define FM10K_SW_REG_FIELD_IDX64(r_, name_, i_, i1_, i2_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i_), \
+ FM10K_SW_##name_##_lsb64(i1_))) >> \
+ FM10K_SW_##name_##_lsb64(i2_))
+#define FM10K_SW_MAKE_REG_FIELD64(name_, v_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64))
+#define FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_, i1_, i2_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64(i_)) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD64(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) | \
+ FM10K_SW_MAKE_REG_FIELD64(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX64(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_)))
+
+/* These operate on arrays of 32-bit words */
+#define FM10K_SW_SET_ARRAY_BIT(a_, name_, b_, a1_) \
+ ((a_)[FM10K_SW_##name_##_bit / 32] = \
+ ((a1_)[FM10K_SW_##name_##_bit / 32] & \
+ ~(1 << (FM10K_SW_##name_##_bit % 32))) | \
+ (((b_) & 0x1) << (FM10K_SW_##name_##_bit % 32)))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_MAKE_ARRAY_FIELD(name_, v_) \
+ (((uint32_t)(v_) << (FM10K_SW_##name_##_lsb % 32)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_REPLACE_ARRAY_FIELD(a_, name_, v_, a1_) \
+ ((a_)[FM10K_SW_##name_##_lsb / 32] = \
+ (((a1_)[FM10K_SW_##name_##_lsb / 32] & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32)) | \
+ FM10K_SW_MAKE_ARRAY_FIELD(name_, v_)))
+
+
+/*
+ * BAR0 registers
+ */
+/* Interrupt throttle timer selection values */
+#define FM10K_SW_INT_TIMER_0 0
+#define FM10K_SW_INT_TIMER_1 1
+#define FM10K_SW_INT_TIMER_IMMEDIATE 2
+#define FM10K_SW_INT_TIMER_DISABLED 3
+
+#define FM10K_SW_CTRL FM10K_SW_REG_OFF(0x0)
+#define FM10K_SW_CTRL_BAR4_ALLOWED (1 << 2)
+#define FM10K_SW_CTRL_EXT FM10K_SW_REG_OFF(0x1)
+#define FM10K_SW_CTRL_EXT_NS_DIS (1 << 0)
+#define FM10K_SW_CTRL_EXT_RO_DIS (1 << 1)
+#define FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK (1 << 2)
+#define FM10K_SW_EXVET FM10K_SW_REG_OFF(0x2)
+#define FM10K_SW_GCR FM10K_SW_REG_OFF(0x3)
+#define FM10K_SW_FACTPS FM10K_SW_REG_OFF(0x4)
+#define FM10K_SW_GCR_EXT FM10K_SW_REG_OFF(0x5)
+#define FM10K_SW_EICR FM10K_SW_REG_OFF(0x6)
+#define FM10K_SW_EICR_PCA_FAULT_SHIFT 0
+#define FM10K_SW_EICR_PCA_FAULT \
+ (1 << FM10K_SW_EICR_PCA_FAULT_SHIFT)
+#define FM10K_SW_EICR_THI_FAULT_SHIFT 2
+#define FM10K_SW_EICR_THI_FAULT \
+ (1 << FM10K_SW_EICR_THI_FAULT_SHIFT)
+#define FM10K_SW_EICR_FUM_FAULT_SHIFT 5
+#define FM10K_SW_EICR_FUM_FAULT \
+ (1 << FM10K_SW_EICR_FUM_FAULT_SHIFT)
+#define FM10K_SW_EICR_MAILBOX_SHIFT 6
+#define FM10K_SW_EICR_MAILBOX \
+ (1 << FM10K_SW_EICR_MAILBOX_SHIFT)
+#define FM10K_SW_EICR_SWITCH_READY_SHIFT 7
+#define FM10K_SW_EICR_SWITCH_READY \
+ (1 << FM10K_SW_EICR_SWITCH_READY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_NREADY_SHIFT 8
+#define FM10K_SW_EICR_SWITCH_NREADY \
+ (1 << FM10K_SW_EICR_SWITCH_NREADY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_INT_SHIFT 9
+#define FM10K_SW_EICR_SWITCH_INT \
+ (1 << FM10K_SW_EICR_SWITCH_INT_SHIFT)
+#define FM10K_SW_EICR_SRAM_ERROR_SHIFT 10
+#define FM10K_SW_EICR_SRAM_ERROR \
+ (1 << FM10K_SW_EICR_SRAM_ERROR_SHIFT)
+#define FM10K_SW_EICR_VFLR_SHIFT 11
+#define FM10K_SW_EICR_VFLR \
+ (1 << FM10K_SW_EICR_VFLR_SHIFT)
+#define FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT 12
+#define FM10K_SW_EICR_MAX_HOLD_TIME \
+ (1 << FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT)
+#define FM10K_SW_EIMR FM10K_SW_REG_OFF(0x7)
+#define FM10K_SW_EIMR_DISABLE_ALL 0x55555555
+#define FM10K_SW_EIMR_NO_CHANGE 0x0
+#define FM10K_SW_EIMR_DISABLE 0x1
+#define FM10K_SW_EIMR_ENABLE 0x2
+#define FM10K_SW_EIMR_FIELD(i_, v_) \
+ (FM10K_SW_EIMR_##v_ << (FM10K_SW_EICR_##i_##_SHIFT * 2))
+#define FM10K_SW_EIMR_ENABLED(e_, i_) \
+ ((((e_) >> (FM10K_SW_EICR_##i_##_SHIFT * 2)) & 0x3) == \
+ FM10K_SW_EIMR_ENABLE)
+#define FM10K_SW_PCA_FAULT FM10K_SW_REG_OFF(0x8)
+#define FM10K_SW_THI_FAULT FM10K_SW_REG_OFF(0x10)
+#define FM10K_SW_FUM_FAULT FM10K_SW_REG_OFF(0x1C)
+#define FM10K_SW_MAXHOLDQ(n_) FM10K_SW_REG_OFF(0x20 + (n_))
+#define FM10K_SW_MAXHOLDQ_ENTRIES 8
+#define FM10K_SW_SM_AREA FM10K_SW_REG_OFF(0x28)
+#define FM10K_SW_DGLORTMAP(n_) FM10K_SW_REG_OFF(0x30 + (n_))
+#define FM10K_SW_DGLORTMAP_ENTRIES 8
+#define FM10K_SW_DGLORTMAP_MATCH_ANY 0x00000000
+#define FM10K_SW_DGLORTMAP_MATCH_NONE 0x0000ffff
+#define FM10K_SW_DGLORTMAP_VALUE_lsb 0
+#define FM10K_SW_DGLORTMAP_VALUE_msb 15
+#define FM10K_SW_DGLORTMAP_MASK_lsb 16
+#define FM10K_SW_DGLORTMAP_MASK_msb 31
+#define FM10K_SW_DGLORTDEC(n_) FM10K_SW_REG_OFF(0x38 + (n_))
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_lsb 0
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_msb 3
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_lsb 4
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_msb 6
+#define FM10K_SW_DGLORTDEC_VSI_BASE_lsb 7
+#define FM10K_SW_DGLORTDEC_VSI_BASE_msb 13
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_lsb 14
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_msb 15
+#define FM10K_SW_DGLORTDEC_Q_BASE_lsb 16
+#define FM10K_SW_DGLORTDEC_Q_BASE_msb 23
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_MAX 7
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_lsb 24
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_msb 26
+#define FM10K_SW_DGLORTDEC_INNTER_RSS (1 << 27)
+#define FM10K_SW_TUNNEL_CFG FM10K_SW_REG_OFF(0x40)
+#define FM10K_SW_SWPRI_MAP(pri_) FM10K_SW_REG_OFF(0x50 + (pri_))
+#define FM10K_SW_RSSRK(f_, n_) \
+ FM10K_SW_REG_OFF(0x800 + 0x10 * (f_) + (n_))
+#define FM10K_SW_RSSRK_ENTRIES 10
+#define FM10K_SW_RETA(f_, n_) \
+ FM10K_SW_REG_OFF(0x1000 + 0x20 * (f_) + (n_))
+#define FM10K_SW_RETA_ENTRIES 32
+#define FM10K_SW_RETA_ENTRY(e0_, e1_, e2_, e3_) \
+ (((e0_) & 0xff) | \
+ (((e1_) & 0xff) << 8) | \
+ (((e2_) & 0xff) << 16) | \
+ (((e3_) & 0xff) << 24))
+#define FM10K_SW_TC_CREDIT(g_) FM10K_SW_REG_OFF(0x2000 + (g_))
+#define FM10K_SW_TC_MAXCREDIT(g_) FM10K_SW_REG_OFF(0x2040 + (g_))
+#define FM10K_SW_TC_RATE(g_) FM10K_SW_REG_OFF(0x2080 + (g_))
+#define FM10K_SW_TC_RATE_STATUS FM10K_SW_REG_OFF(0x20C0)
+#define FM10K_SW_PAUSE FM10K_SW_REG_OFF(0x20C2)
+#define FM10K_SW_DMA_CTRL FM10K_SW_REG_OFF(0x20C3)
+#define FM10K_SW_DMA_CTRL_TX_ENABLE (1 << 0)
+#define FM10K_SW_DMA_CTRL_TX_HOST_PENDING (1 << 1)
+#define FM10K_SW_DMA_CTRL_TX_DATA (1 << 2)
+#define FM10K_SW_DMA_CTRL_TX_ACTIVE (1 << 3)
+#define FM10K_SW_DMA_CTRL_RX_ENABLE (1 << 4)
+#define FM10K_SW_DMA_CTRL_RX_HOST_PENDING (1 << 5)
+#define FM10K_SW_DMA_CTRL_RX_DATA (1 << 6)
+#define FM10K_SW_DMA_CTRL_RX_ACTIVE (1 << 7)
+#define FM10K_SW_DMA_CTRL_RX_DESC_SIZE_32 (1 << 8)
+#define FM10K_SW_DMA_CTRL_MIN_MSS_lsb 9
+#define FM10K_SW_DMA_CTRL_MIN_MSS_msb 22
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_lsb 23
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_msb 27
+#define FM10K_SW_DMA_CTRL_DATA_PATH_RESET (1 << 29)
+#define FM10K_SW_DMA_CTRL_MAX_QS_lsb 30
+#define FM10K_SW_DMA_CTRL_MAX_QS_msb 31
+#define FM10K_SW_DMA_CTRL_MAX_QS_256 0
+#define FM10K_SW_DMA_CTRL_MAX_QS_128 1
+#define FM10K_SW_DMA_CTRL_MAX_QS_64 2
+#define FM10K_SW_DMA_CTRL2 FM10K_SW_REG_OFF(0x20C4)
+#define FM10K_SW_DTXTCPFLGL FM10K_SW_REG_OFF(0x20C5)
+#define FM10K_SW_DTXTCPFLGH FM10K_SW_REG_OFF(0x20C6)
+#define FM10K_SW_TPH_CTRL FM10K_SW_REG_OFF(0x20C7)
+#define FM10K_SW_MRQC(f_) \
+ FM10K_SW_REG_OFF(0x2100 + (f_))
+#define FM10K_SW_MRQC_TCPIPV4 (1 << 0)
+#define FM10K_SW_MRQC_IPV4 (1 << 1)
+#define FM10K_SW_MRQC_IPV6 (1 << 4)
+#define FM10K_SW_MRQC_TCPIPV6 (1 << 5)
+#define FM10K_SW_MRQC_UDPIPV4 (1 << 6)
+#define FM10K_SW_MRQC_UDPIPV6 (1 << 7)
+#define FM10K_SW_TQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x2800 + \
+ (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : \
+ ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_RQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x3000 + \
+ (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : \
+ ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_STATS_TIMEOUT FM10K_SW_REG_OFF(0x3800)
+#define FM10K_SW_STATS_UR FM10K_SW_REG_OFF(0x3801)
+#define FM10K_SW_STATS_CA FM10K_SW_REG_OFF(0x3802)
+#define FM10K_SW_STATS_UM FM10K_SW_REG_OFF(0x3803)
+#define FM10K_SW_STATS_XEC FM10K_SW_REG_OFF(0x3804)
+#define FM10K_SW_STATS_VLAN_DROP FM10K_SW_REG_OFF(0x3805)
+#define FM10K_SW_STATS_LOOPBACK_DROP FM10K_SW_REG_OFF(0x3806)
+#define FM10K_SW_STATS_NODESC_DROP FM10K_SW_REG_OFF(0x3807)
+#define FM10K_SW_RRTIME_CFG FM10K_SW_REG_OFF(0x3808)
+#define FM10K_SW_RRTIME_LIMIT(n_) \
+ FM10K_SW_REG_OFF(0x380C + 0x40 * (n_))
+#define FM10K_SW_RRTIME_COUNT(n_) \
+ FM10K_SW_REG_OFF(0x3810 + 0x40 * (n_))
+#define FM10K_SW_SYSTIME FM10K_SW_REG_OFF(0x3814)
+#define FM10K_SW_SYSTIME0 FM10K_SW_REG_OFF(0x3816)
+#define FM10K_SW_SYSTIME_CFG FM10K_SW_REG_OFF(0x3818)
+#define FM10K_SW_PFVFBME FM10K_SW_REG_OFF(0x381A)
+#define FM10K_SW_PHYADDR FM10K_SW_REG_OFF(0x381C)
+#define FM10K_SW_RDBAL(q_) \
+ FM10K_SW_REG_OFF(0x4000 + 0x40 * (q_))
+#define FM10K_SW_RDBAH(q_) \
+ FM10K_SW_REG_OFF(0x4001 + 0x40 * (q_))
+#define FM10K_SW_RDLEN(q_) \
+ FM10K_SW_REG_OFF(0x4002 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x4003 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_RXCTRL_HEADER_TPHEN (1 << 6)
+#define FM10K_SW_TPH_RXCTRL_PAYLOAD_TPHEN (1 << 7)
+#define FM10K_SW_TPH_RXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_RXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_RXCTRL_DATA_WRITE_RO_EN (1 << 13)
+#define FM10K_SW_TPH_RXCTRL_REP_HEADER_RO_EN (1 << 15)
+#define FM10K_SW_RDH(q_) \
+ FM10K_SW_REG_OFF(0x4004 + 0x40 * (q_))
+#define FM10K_SW_RDT(q_) \
+ FM10K_SW_REG_OFF(0x4005 + 0x40 * (q_))
+#define FM10K_SW_RDT_RDT_lsb 0
+#define FM10K_SW_RDT_RDT_msb 15
+#define FM10K_SW_RXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x4006 + 0x40 * (q_))
+#define FM10K_SW_RXQCTL_ENABLE (1 << 0)
+#define FM10K_SW_RXQCTL_VF_lsb 2
+#define FM10K_SW_RXQCTL_VF_msb 7
+#define FM10K_SW_RXQCTL_OWNED_BY_VF (1 << 8)
+#define FM10K_SW_RXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x4007 + 0x40 * (q_))
+#define FM10K_SW_RXDCTL_MAX_TIME_lsb 0
+#define FM10K_SW_RXDCTL_MAX_TIME_msb 7
+#define FM10K_SW_RXDCTL_WRITE_BACK_IMM (1 << 8)
+#define FM10K_SW_RXDCTL_DROP_ON_EMPTY (1 << 9)
+#define FM10K_SW_RXDCTL_WRITE_RSS_HASH (1 << 10)
+#define FM10K_SW_RXINT(q_) \
+ FM10K_SW_REG_OFF(0x4008 + 0x40 * (q_))
+#define FM10K_SW_RXINT_INT_lsb 0
+#define FM10K_SW_RXINT_INT_msb 7
+#define FM10K_SW_RXINT_INT_TIMER_lsb 8
+#define FM10K_SW_RXINT_INT_TIMER_msb 9
+#define FM10K_SW_SRRCTL(q_) \
+ FM10K_SW_REG_OFF(0x4009 + 0x40 * (q_))
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_lsb 0
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_msb 7
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_lsb 8
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_msb 13
+#define FM10K_SW_SRRCTL_DESC_TYPE_lsb 14
+#define FM10K_SW_SRRCTL_DESC_TYPE_msb 15
+#define FM10K_SW_SRRCTL_DESC_TYPE_NO_SPLIT 0
+#define FM10K_SW_SRRCTL_DESC_TYPE_HDR_SPLIT 1
+#define FM10K_SW_SRRCTL_DESC_TYPE_SIZE_SPLIT 2
+#define FM10K_SW_SRRCTL_PSR_TYPE_lsb 16
+#define FM10K_SW_SRRCTL_PSR_TYPE_msb 29
+#define FM10K_SW_SRRCTL_LOOPBACK_SUPPRESS (1 << 30)
+#define FM10K_SW_SRRCTL_BUFFER_CHAINING_EN (1 << 31)
+#define FM10K_SW_QPRC(q_) \
+ FM10K_SW_REG_OFF(0x400A + 0x40 * (q_))
+#define FM10K_SW_QPRDC(q_) \
+ FM10K_SW_REG_OFF(0x400B + 0x40 * (q_))
+#define FM10K_SW_QBRC_L(q_) \
+ FM10K_SW_REG_OFF(0x400C + 0x40 * (q_))
+#define FM10K_SW_QBRC_H(q_) \
+ FM10K_SW_REG_OFF(0x400D + 0x40 * (q_))
+#define FM10K_SW_RX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x400E + 0x40 * (q_))
+#define FM10K_SW_TDBAL(q_) \
+ FM10K_SW_REG_OFF(0x8000 + 0x40 * (q_))
+#define FM10K_SW_TDBAH(q_) \
+ FM10K_SW_REG_OFF(0x8001 + 0x40 * (q_))
+#define FM10K_SW_TDLEN(q_) \
+ FM10K_SW_REG_OFF(0x8002 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x8003 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_TXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_TXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_TXCTRL_DATA_READ_RO_EN (1 << 13)
+#define FM10K_SW_TDH(q_) \
+ FM10K_SW_REG_OFF(0x8004 + 0x40 * (q_))
+#define FM10K_SW_TDT(q_) \
+ FM10K_SW_REG_OFF(0x8005 + 0x40 * (q_))
+#define FM10K_SW_TDT_TDT_lsb 0
+#define FM10K_SW_TDT_TDT_msb 15
+#define FM10K_SW_TXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x8006 + 0x40 * (q_))
+#define FM10K_SW_TXDCTL_PTHRESH_lsb 0
+#define FM10K_SW_TXDCTL_PTHRESH_msb 6
+#define FM10K_SW_TXDCTL_HTHRESH_lsb 7
+#define FM10K_SW_TXDCTL_HTHRESH_msb 13
+#define FM10K_SW_TXDCTL_ENABLE (1 << 14)
+#define FM10K_SW_TXDCTL_MAX_TIME_lsb 16
+#define FM10K_SW_TXDCTL_MAX_TIME_msb 27
+#define FM10K_SW_TXDCTL_PUSH_DESC (1 << 28)
+#define FM10K_SW_TXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x8007 + 0x40 * (q_))
+#define FM10K_SW_TXQCTL_VF_lsb 0
+#define FM10K_SW_TXQCTL_VF_msb 5
+#define FM10K_SW_TXQCTL_OWNED_BY_VF (1 << 6)
+#define FM10K_SW_TXQCTL_PC_lsb 7
+#define FM10K_SW_TXQCTL_PC_msb 9
+#define FM10K_SW_TXQCTL_TC_lsb 10
+#define FM10K_SW_TXQCTL_TC_msb 15
+#define FM10K_SW_TXQCTL_VID_lsb 16
+#define FM10K_SW_TXQCTL_VID_msb 27
+#define FM10K_SW_TXQCTL_UNLIMITED_BW (1 << 28)
+#define FM10K_SW_TXQCTL_PUSH_MODE_DIS (1 << 29)
+#define FM10K_SW_TXINT(q_) \
+ FM10K_SW_REG_OFF(0x8008 + 0x40 * (q_))
+#define FM10K_SW_TXINT_INT_lsb 0
+#define FM10K_SW_TXINT_INT_msb 7
+#define FM10K_SW_TXINT_INT_TIMER_lsb 8
+#define FM10K_SW_TXINT_INT_TIMER_msb 9
+#define FM10K_SW_QPTC(q_) \
+ FM10K_SW_REG_OFF(0x8009 + 0x40 * (q_))
+#define FM10K_SW_QBTC_L(q_) \
+ FM10K_SW_REG_OFF(0x800A + 0x40 * (q_))
+#define FM10K_SW_QBTC_H(q_) \
+ FM10K_SW_REG_OFF(0x800B + 0x40 * (q_))
+#define FM10K_SW_TQDLOC(q_) \
+ FM10K_SW_REG_OFF(0x800C + 0x40 * (q_))
+#define FM10K_SW_TQDLOC_BASE_lsb 0
+#define FM10K_SW_TQDLOC_BASE_msb 15
+#define FM10K_SW_TQDLOC_SIZE_lsb 16
+#define FM10K_SW_TQDLOC_SIZE_msb 19
+#define FM10K_SW_TQDLOC_SIZE_32 5
+#define FM10K_SW_TQDLOC_SIZE_64 6
+#define FM10K_SW_TQDLOC_SIZE_128 7
+#define FM10K_SW_TX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x800D + 0x40 * (q_))
+#define FM10K_SW_TX_SGLORT_SGLORT_lsb 0
+#define FM10K_SW_TX_SGLORT_SGLORT_msb 15
+#define FM10K_SW_PFVTCTL(q_) \
+ FM10K_SW_REG_OFF(0x800E + 0x40 * (q_))
+#define FM10K_SW_TX_DESC(q_, d_, w_) \
+ FM10K_SW_REG_OFF(0x40000 + \
+ 0x400 * (q_) + 0x4 * (d_) + (w_))
+#define FM10K_SW_PBACL(n_) \
+ FM10K_SW_REG_OFF(0x10000 + (n_))
+#define FM10K_SW_INT_MAP(n_) \
+ FM10K_SW_REG_OFF(0x10080 + (n_))
+#define FM10K_SW_INT_MAP_ENTRIES 8
+#define FM10K_SW_INT_MAP_INDEX_MAILBOX 0
+#define FM10K_SW_INT_MAP_INDEX_FAULT 1
+#define FM10K_SW_INT_MAP_INDEX_SWITCH_UP_DOWN 2
+#define FM10K_SW_INT_MAP_INDEX_SWITCH 3
+#define FM10K_SW_INT_MAP_INDEX_SRAM 4
+#define FM10K_SW_INT_MAP_INDEX_VFLR 5
+#define FM10K_SW_INT_MAP_INDEX_MAX_HOLD_TIME 6
+#define FM10K_SW_INT_MAP_INT_lsb 0
+#define FM10K_SW_INT_MAP_INT_msb 7
+#define FM10K_SW_INT_MAP_INT_TIMER_lsb 8
+#define FM10K_SW_INT_MAP_INT_TIMER_msb 9
+#define FM10K_SW_MSIX_VECTOR(v_) \
+ FM10K_SW_REG_OFF(0x11000 + 0x4 * (v_))
+#define FM10K_SW_INT_CTRL \
+ FM10K_SW_REG_OFF(0x12000)
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_lsb 0
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_msb 9
+#define FM10K_SW_INT_CTRL_ENABLE_MODERATOR (1 << 10)
+#define FM10K_SW_ITR(v_) \
+ FM10K_SW_REG_OFF(0x12400 + (v_))
+
+/*
+ * Interrupt throttle timer intervals in microseconds. These provide the
+ * direct values for programming the ITR interval field when using a Gen3
+ * PCLK, otherwise they need to be scaled appropriately.
+ */
+#define FM10K_SW_ITR_INTERVAL_20K 50
+#define FM10K_SW_ITR_INTERVAL_40K 25
+#define FM10K_SW_ITR_INTERVAL_0_lsb 0
+#define FM10K_SW_ITR_INTERVAL_0_msb 11
+#define FM10K_SW_ITR_INTERVAL_1_lsb 12
+#define FM10K_SW_ITR_INTERVAL_1_msb 23
+#define FM10K_SW_ITR_TIMER_0_EXPIRED (1 << 24)
+#define FM10K_SW_ITR_TIMER_1_EXPIRED (1 << 25)
+#define FM10K_SW_ITR_PENDING_0 (1 << 26)
+#define FM10K_SW_ITR_PENDING_1 (1 << 27)
+#define FM10K_SW_ITR_PENDING_2 (1 << 28)
+#define FM10K_SW_ITR_AUTO_MASK (1 << 29)
+#define FM10K_SW_ITR_MASK_lsb 30
+#define FM10K_SW_ITR_MASK_msb 31
+#define FM10K_SW_ITR_MASK_R_ENABLED 0
+#define FM10K_SW_ITR_MASK_R_BLOCKED 1
+#define FM10K_SW_ITR_MASK_W_KEEP 0
+#define FM10K_SW_ITR_MASK_W_BLOCK 1
+#define FM10K_SW_ITR_MASK_W_ENABLE 2
+#define FM10K_SW_ITR2(v_) \
+ FM10K_SW_REG_OFF(0x12800 + 0x2 * (v_))
+#define FM10K_SW_IP \
+ FM10K_SW_REG_OFF(0x13000)
+#define FM10K_SW_IP_HOT_RESET (1 << 0)
+#define FM10K_SW_IP_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IP_MAILBOX (1 << 2)
+#define FM10K_SW_IP_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IP_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IP_PFLR (1 << 5)
+#define FM10K_SW_IP_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IP_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IP_NOT_IN_RESET (1 << 8)
+#define FM10K_SW_IP_TIMEOUT (1 << 9)
+#define FM10K_SW_IP_VFLR (1 << 10)
+#define FM10K_SW_IM \
+ FM10K_SW_REG_OFF(0x13001)
+#define FM10K_SW_IM_ALL \
+ FM10K_SW_MASK32(10, 0)
+#define FM10K_SW_IM_HOT_RESET (1 << 0)
+#define FM10K_SW_IM_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IM_MAILBOX (1 << 2)
+#define FM10K_SW_IM_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IM_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IM_PFLR (1 << 5)
+#define FM10K_SW_IM_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IM_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IM_TIMEOUT (1 << 9)
+#define FM10K_SW_IM_VFLR (1 << 10)
+#define FM10K_SW_IB FM10K_SW_REG_OFF(0x13002)
+#define FM10K_SW_SRAM_IP FM10K_SW_REG_OFF(0x13003)
+#define FM10K_SW_SRAM_IM FM10K_SW_REG_OFF(0x13004)
+#define FM10K_SW_VLAN_TABLE(f_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_))
+#define FM10K_SW_VLAN_TABLE_ENTRIES 128
+#define FM10K_SW_VLAN_TABLE_ENTRY(f_, n_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + (n_))
+#define FM10K_SW_VLAN_TABLE_VLAN_ENTRY(f_, vl_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + ((vl_) >> 5))
+#define FM10K_SW_VLAN_TABLE_VLAN_BIT(vl_) (1 << ((vl_) & 0x1f))
+#define FM10K_SW_MBMEM(n_) \
+ FM10K_SW_REG_OFF(0x18000 + (n_))
+#define FM10K_SW_MBX(vf_) \
+ FM10K_SW_REG_OFF(0x18800 + (vf_))
+#define FM10K_SW_MBICR(vf_) \
+ FM10K_SW_REG_OFF(0x18840 + ((vf_) >> 5))
+#define FM10K_SW_GMBX FM10K_SW_REG_OFF(0x18842)
+#define FM10K_SW_GMBX_GLOBAL_REQ (1 << 1)
+#define FM10K_SW_GMBX_GLOBAL_ACK (1 << 2)
+#define FM10K_SW_GMBX_PF_REQ_INTERRUPT (1 << 3)
+#define FM10K_SW_GMBX_PF_ACK_INTERRUPT (1 << 4)
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_lsb 5
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_msb 6
+#define FM10K_SW_GMBX_INTERRUPT_NO_CHANGE 0
+#define FM10K_SW_GMBX_INTERRUPT_ENABLE 1
+#define FM10K_SW_GMBX_INTERRUPT_DISABLE 2
+#define FM10K_SW_GMBX_PF_REQ (1 << 7)
+#define FM10K_SW_GMBX_PF_ACK (1 << 8)
+#define FM10K_SW_GMBX_GLOBAL_REQ_INTERRUPT (1 << 9)
+#define FM10K_SW_GMBX_GLOBAL_ACK_INTERRUPT (1 << 10)
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_lsb 11
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_msb 12
+#define FM10K_SW_PFVFLRE FM10K_SW_REG_OFF(0x18844)
+#define FM10K_SW_PFVFLREC FM10K_SW_REG_OFF(0x18846)
+#define FM10K_SW_TEST_CFG0 FM10K_SW_REG_OFF(0x18849)
+#define FM10K_SW_INT_SRAM_CTRL FM10K_SW_REG_OFF(0x18850)
+#define FM10K_SW_FUM_SRAM_CTRL FM10K_SW_REG_OFF(0x18852)
+#define FM10K_SW_PCA_SRAM_CTRL FM10K_SW_REG_OFF(0x18854)
+#define FM10K_SW_PP_SRAM_CTRL FM10K_SW_REG_OFF(0x18858)
+#define FM10K_SW_PCW_SRAM_CTRL FM10K_SW_REG_OFF(0x1885C)
+#define FM10K_SW_RHI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18872)
+#define FM10K_SW_RHI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18860)
+#define FM10K_SW_THI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18864)
+#define FM10K_SW_THI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18868)
+#define FM10K_SW_TIMEOUT_CFG FM10K_SW_REG_OFF(0x1886B)
+#define FM10K_SW_LVMMC FM10K_SW_REG_OFF(0x18880)
+#define FM10K_SW_LVMMI FM10K_SW_REG_OFF(0x18881)
+#define FM10K_SW_HOST_MISC FM10K_SW_REG_OFF(0x19000)
+#define FM10K_SW_HOST_LANE_CTRL FM10K_SW_REG_OFF(0x19001)
+#define FM10K_SW_SERDES_CTRL(l_) \
+ FM10K_SW_REG_OFF(0x19010 + 0x2 * (l_))
+
+/*
+ * BAR4 registers
+ */
+
+/*
+ * Access to non-master PEP registers via BAR4
+ */
+#define FM10K_SW_PCIE_GLOBAL(p_, r_) \
+ (FM10K_SW_REG_OFF(((p_) + 1) << 20) | FM10K_SW_##r_)
+
+/*
+ * SBUS register fields for use with both the PCIE and EPL SBUS interfaces.
+ */
+#define FM10K_SW_SBUS_CFG_SBUS_CONTROLLER_RESET (1 << 0)
+#define FM10K_SW_SBUS_CFG_ROM_ENABLE (1 << 1)
+#define FM10K_SW_SBUS_CFG_ROM_BUSY (1 << 2)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_PASS (1 << 3)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_FAIL (1 << 4)
+
+#define FM10K_SW_SBUS_COMMAND_REGISTER_lsb 0
+#define FM10K_SW_SBUS_COMMAND_REGISTER_msb 7
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_lsb 8
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_msb 15
+#define FM10K_SW_SBUS_COMMAND_OP_lsb 16
+#define FM10K_SW_SBUS_COMMAND_OP_msb 23
+#define FM10K_SW_SBUS_OP_RESET 0x20
+#define FM10K_SW_SBUS_OP_WRITE 0x21
+#define FM10K_SW_SBUS_OP_READ 0x22
+#define FM10K_SW_SBUS_COMMAND_EXECUTE (1 << 24)
+#define FM10K_SW_SBUS_COMMAND_BUSY (1 << 25)
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_lsb 26
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_msb 28
+#define FM10K_SW_SBUS_RESULT_RESET 0x00
+#define FM10K_SW_SBUS_RESULT_WRITE 0x01
+#define FM10K_SW_SBUS_RESULT_READ 0x04
+
+#define FM10K_SW_SBUS_ADDR_EPL_RMON2 1
+#define FM10K_SW_SBUS_ADDR_EPL_LANE(e_, l_) (4 * (e_) + (l_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_SERDES(s_) ((s_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_RMON3 38
+#define FM10K_SW_SBUS_ADDR_EPL_PMRO 39
+
+/* Common to both EPL and PCIE
+ */
+#define FM10K_SW_SBUS_ADDR_SPICO 253
+#define FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER 254
+#define FM10K_SW_SBUS_ADDR_BROADCAST 0xFF
+
+
+#define FM10K_SW_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x000000)
+#define FM10K_SW_MGMT_REG(wo_) \
+ (FM10K_SW_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_FATAL_CODE FM10K_SW_MGMT_REG(0x0)
+#define FM10K_SW_LAST_FATAL_CODE FM10K_SW_MGMT_REG(0x1)
+#define FM10K_SW_FATAL_COUNT FM10K_SW_MGMT_REG(0x2)
+#define FM10K_SW_SOFT_RESET FM10K_SW_MGMT_REG(0x3)
+#define FM10K_SW_SOFT_RESET_COLD_RESET (1 << 0)
+#define FM10K_SW_SOFT_RESET_EPL_RESET (1 << 1)
+#define FM10K_SW_SOFT_RESET_SWITCH_RESET (1 << 2)
+#define FM10K_SW_SOFT_RESET_SWITCH_READY (1 << 3)
+#define FM10K_SW_SOFT_RESET_PCIE_RESET(p_) (1 << ((p_) + 4))
+#define FM10K_SW_SOFT_RESET_PCIE_ACTIVE(p_) (1 << ((p_) + 13))
+#define FM10K_SW_DEVICE_CFG FM10K_SW_MGMT_REG(0x4)
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS 4
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(p_) (1 << (p_))
+#define FM10K_SW_DEVICE_CFG_PCIE_100G_DIS (1 << 4)
+#define FM10K_SW_DEVICE_CFG_FEATURE_lsb 5
+#define FM10K_SW_DEVICE_CFG_FEATURE_msb 6
+#define FM10K_SW_DEVICE_CFG_PCIE_FULL 0
+#define FM10K_SW_DEVICE_CFG_PCIE_HALF 1
+#define FM10K_SW_DEVICE_CFG_PCIE_BASIC 2
+#define FM10K_SW_DEVICE_CFG_PCIE_EN(p_) (1 << (7 + (p_)))
+#define FM10K_SW_RESET_CFG FM10K_SW_MGMT_REG(0x5)
+#define FM10K_SW_WATCHDOG_CFG FM10K_SW_MGMT_REG(0x6)
+#define FM10K_SW_MGMT_SCRATCH(n_) FM10K_SW_MGMT_REG(0x8 + (n_))
+#define FM10K_SW_VITAL_PRODUCT_DATA FM10K_SW_MGMT_REG(0x304)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x400)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_lsb64 0
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_msb64 8
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_lsb64 9
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_msb64 17
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_lsb64 18
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_msb64 26
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_lsb64 27
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_msb64 28
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CORE (1ULL << 29)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SOFTWARE (1ULL << 30)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_GPIO (1ULL << 31)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C (1ULL << 32)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_MDIO (1ULL << 33)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CRM (1ULL << 34)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_TAIL (1ULL << 35)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_HEAD (1ULL << 36)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PINS (1ULL << 39)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FIBM (1ULL << 40)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_BSM (1ULL << 41)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_INT \
+ FM10K_SW_MGMT_REG(0x402)
+#define FM10K_SW_INTERRUPT_MASK_PCIE(p_) \
+ FM10K_SW_MGMT_REG(0x420 + 0x2 * (p_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_lsb64 0
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_msb64 8
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_lsb64 9
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_msb64 17
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_lsb64 18
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_msb64 26
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_lsb64 27
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_msb64 28
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CORE (1ULL << 29)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SOFTWARE (1ULL << 30)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_GPIO (1ULL << 31)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_I2C (1ULL << 32)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_MDIO (1ULL << 33)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CRM (1ULL << 34)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_TAIL (1ULL << 35)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_HEAD (1ULL << 36)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PINS (1ULL << 39)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FIBM (1ULL << 40)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_BSM (1ULL << 41)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_ALL FM10K_SW_MASK64(42, 0)
+#define FM10K_SW_INTERRUPT_MASK_FIBM FM10K_SW_MGMT_REG(0x440)
+#define FM10K_SW_INTERRUPT_MASK_BSM FM10K_SW_MGMT_REG(0x442)
+#define FM10K_SW_CORE_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x444)
+#define FM10K_SW_CORE_INTERRUPT_MASK FM10K_SW_MGMT_REG(0x445)
+#define FM10K_SW_SRAM_ERR_IP FM10K_SW_MGMT_REG(0x446)
+#define FM10K_SW_SRAM_ERR_IM FM10K_SW_MGMT_REG(0x448)
+#define FM10K_SW_PINS_STAT FM10K_SW_MGMT_REG(0x44A)
+#define FM10K_SW_PINS_IP FM10K_SW_MGMT_REG(0x44B)
+#define FM10K_SW_PINS_IM FM10K_SW_MGMT_REG(0x44C)
+#define FM10K_SW_SW_IP FM10K_SW_MGMT_REG(0x44D)
+#define FM10K_SW_SW_IM FM10K_SW_MGMT_REG(0x44E)
+#define FM10K_SW_SW_TEST_AND_SET FM10K_SW_MGMT_REG(0x44F)
+#define FM10K_SW_LSM_CLKOBS_CTRL FM10K_SW_MGMT_REG(0x450)
+#define FM10K_SW_CHIP_VERSION FM10K_SW_MGMT_REG(0x452)
+#define FM10K_SW_BSM_SCRATCH(n_) FM10K_SW_MGMT_REG(0x800 + (n_))
+#define FM10K_SW_BSM_CTRL FM10K_SW_MGMT_REG(0xC00)
+#define FM10K_SW_BSM_ARGS FM10K_SW_MGMT_REG(0xC01)
+#define FM10K_SW_BSM_ADDR_OFFSET(n_) FM10K_SW_MGMT_REG(0xC04 + (n_))
+#define FM10K_SW_BSM_COUNTER(n_) FM10K_SW_MGMT_REG(0xC08 + (n_))
+#define FM10K_SW_BSM_SRAM_CTRL FM10K_SW_MGMT_REG(0xC0A)
+#define FM10K_SW_BSM_IP FM10K_SW_MGMT_REG(0xC0B)
+#define FM10K_SW_BSM_IM FM10K_SW_MGMT_REG(0xC0C)
+#define FM10K_SW_PIN_STRAP_STAT FM10K_SW_MGMT_REG(0xC0D)
+#define FM10K_SW_FUSE_DATA_0 FM10K_SW_MGMT_REG(0xC0E)
+#define FM10K_SW_FUSE_SKU_lsb 11
+#define FM10K_SW_FUSE_SKU_msb 15
+#define FM10K_SW_FUSE_SKU_FM10840 0
+#define FM10K_SW_FUSE_SKU_FM10420 1
+#define FM10K_SW_FUSE_SKU_FM10064 2
+#define FM10K_SW_FUSE_DATA_1 FM10K_SW_MGMT_REG(0xC0F)
+#define FM10K_SW_BIST_CTRL FM10K_SW_MGMT_REG(0xC10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_MODE_EPL (1ULL << 41)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC (1ULL << 42)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL (1ULL << 43)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_BSM (1ULL << 44)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_CRM (1ULL << 45)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FIBM (1ULL << 46)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_SBM (1ULL << 47)
+#define FM10K_SW_REI_CTRL FM10K_SW_MGMT_REG(0xC12)
+#define FM10K_SW_REI_STAT FM10K_SW_MGMT_REG(0xC13)
+#define FM10K_SW_GPIO_CFG FM10K_SW_MGMT_REG(0xC15)
+#define FM10K_SW_GPIO_DATA FM10K_SW_MGMT_REG(0xC16)
+#define FM10K_SW_GPIO_IP FM10K_SW_MGMT_REG(0xC17)
+#define FM10K_SW_GPIO_IM FM10K_SW_MGMT_REG(0xC18)
+#define FM10K_SW_I2C_CFG FM10K_SW_MGMT_REG(0xC19)
+#define FM10K_SW_I2C_CFG_SLAVE_ENABLE (1 << 0)
+#define FM10K_SW_I2C_CFG_ADDR_lsb 1
+#define FM10K_SW_I2C_CFG_ADDR_msb 7
+#define FM10K_SW_I2C_CFG_DIVIDER_lsb 8
+#define FM10K_SW_I2C_CFG_DIVIDER_msb 19
+#define FM10K_SW_I2C_CFG_DIVIDER_100_KHZ 52
+#define FM10K_SW_I2C_CFG_DIVIDER_400_KHZ 10
+#define FM10K_SW_I2C_CFG_INTERRUPT_MASK (1 << 20)
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_lsb 21
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_msb 27
+#define FM10K_SW_I2C_DATA(n_) FM10K_SW_MGMT_REG(0xC1C + (n_))
+#define FM10K_SW_I2C_CTRL FM10K_SW_MGMT_REG(0xC20)
+#define FM10K_SW_I2C_CTRL_ADDR_lsb 0
+#define FM10K_SW_I2C_CTRL_ADDR_msb 7
+#define FM10K_SW_I2C_CTRL_COMMAND_lsb 8
+#define FM10K_SW_I2C_CTRL_COMMAND_msb 9
+#define FM10K_SW_I2C_COMMAND_NULL 0
+#define FM10K_SW_I2C_COMMAND_WR 1
+#define FM10K_SW_I2C_COMMAND_WR_RD 2
+#define FM10K_SW_I2C_COMMAND_RD 3
+#define FM10K_SW_I2C_CTRL_LENGTH_W_lsb 10
+#define FM10K_SW_I2C_CTRL_LENGTH_W_msb 13
+#define FM10K_SW_I2C_CTRL_LENGTH_R_lsb 14
+#define FM10K_SW_I2C_CTRL_LENGTH_R_msb 17
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_lsb 18
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_msb 21
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_lsb 22
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_msb 25
+#define FM10K_SW_I2C_COMPLETION_RUNNING 0
+#define FM10K_SW_I2C_COMPLETION_NORMAL 1
+#define FM10K_SW_I2C_COMPLETION_PREMATURE 2
+#define FM10K_SW_I2C_COMPLETION_NO_DEVICE 3
+#define FM10K_SW_I2C_COMPLETION_TIMEOUT 4
+#define FM10K_SW_I2C_COMPLETION_LOST_ARB 5
+#define FM10K_SW_I2C_COMPLETION_BUS_WAIT 6
+#define FM10K_SW_I2C_COMPLETION_INVALID 7
+#define FM10K_SW_I2C_CTRL_INTERRUPT_PENDING (1 << 26)
+#define FM10K_SW_MDIO_CFG FM10K_SW_MGMT_REG(0xC22)
+#define FM10K_SW_MDIO_DATA FM10K_SW_MGMT_REG(0xC23)
+#define FM10K_SW_MDIO_CTRL FM10K_SW_MGMT_REG(0xC24)
+#define FM10K_SW_SPI_TX_DATA FM10K_SW_MGMT_REG(0xC26)
+#define FM10K_SW_SPI_RX_DATA FM10K_SW_MGMT_REG(0xC27)
+#define FM10K_SW_SPI_HEADER FM10K_SW_MGMT_REG(0xC28)
+#define FM10K_SW_SPI_CTRL FM10K_SW_MGMT_REG(0xC29)
+#define FM10K_SW_LED_CFG FM10K_SW_MGMT_REG(0xC2B)
+#define FM10K_SW_SCAN_DATA_IN FM10K_SW_MGMT_REG(0xC2D)
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_lsb 0
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_msb 24
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
+#define FM10K_SW_SCAN_DATA_IN_INJECT (1 << 28)
+#define FM10K_SW_SCAN_DATA_IN_DRAIN (1 << 29)
+#define FM10K_SW_SCAN_DATA_IN_PASSTHRU (1 << 30)
+#define FM10K_SW_SCAN_DATA_IN_SINGLE (1 << 31)
+#define FM10K_SW_CRM_DATA(m_, n_) \
+ FM10K_SW_MGMT_REG(0x1000 + 0x2 * (m_) + (n_))
+#define FM10K_SW_CRM_CTRL FM10K_SW_MGMT_REG(0x2000)
+#define FM10K_SW_CRM_STATUS FM10K_SW_MGMT_REG(0x2001)
+#define FM10K_SW_CRM_TIME FM10K_SW_MGMT_REG(0x2002)
+#define FM10K_SW_CRM_SRAM_CTRL FM10K_SW_MGMT_REG(0x2004)
+#define FM10K_SW_CRM_IP FM10K_SW_MGMT_REG(0x2008)
+#define FM10K_SW_CRM_IM FM10K_SW_MGMT_REG(0x200C)
+#define FM10K_SW_CRM_COMMAND(n_) FM10K_SW_MGMT_REG(0x2080 + 0x2 * (n_))
+#define FM10K_SW_CRM_REGISTER(n_) FM10K_SW_MGMT_REG(0x2100 + 0x2 * (n_))
+#define FM10K_SW_CRM_PERIOD(n_) FM10K_SW_MGMT_REG(0x2180 + 0x2 * (n_))
+#define FM10K_SW_CRM_PARAM(n_) FM10K_SW_MGMT_REG(0x2200 + (n_))
+#define FM10K_SW_PLL_PCIE_CTRL FM10K_SW_MGMT_REG(0x2241)
+#define FM10K_SW_PLL_PCIE_STAT FM10K_SW_MGMT_REG(0x2242)
+#define FM10K_SW_SBUS_PCIE_CFG FM10K_SW_MGMT_REG(0x2243)
+#define FM10K_SW_SBUS_PCIE_COMMAND FM10K_SW_MGMT_REG(0x2244)
+#define FM10K_SW_SBUS_PCIE_REQUEST FM10K_SW_MGMT_REG(0x2245)
+#define FM10K_SW_SBUS_PCIE_RESPONSE FM10K_SW_MGMT_REG(0x2246)
+#define FM10K_SW_SBUS_PCIE_SPICO_IN FM10K_SW_MGMT_REG(0x2247)
+#define FM10K_SW_SBUS_PCIE_SPICO_OUT FM10K_SW_MGMT_REG(0x2248)
+#define FM10K_SW_SBUS_PCIE_IP FM10K_SW_MGMT_REG(0x2249)
+#define FM10K_SW_SBUS_PCIE_IM FM10K_SW_MGMT_REG(0x224A)
+#define FM10K_SW_MGMT_SYSTIME_CFG FM10K_SW_MGMT_REG(0x224C)
+#define FM10K_SW_MGMT_SYSTIME FM10K_SW_MGMT_REG(0x224E)
+#define FM10K_SW_MGMT_SYSTIME0 FM10K_SW_MGMT_REG(0x2250)
+#define FM10K_SW_SYSTIME_PULSE_(n_) FM10K_SW_MGMT_REG(0x2252 + (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_LO(n_) FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_HI(n_) \
+ FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_) + 0x1)
+#define FM10K_SW_PCIE_XPLL_CTRL FM10K_SW_MGMT_REG(0x3000)
+#define FM10K_SW_PCIE_CLK_CTRL FM10K_SW_MGMT_REG(0x3001)
+#define FM10K_SW_PCIE_CLK_CTRL2 FM10K_SW_MGMT_REG(0x3002)
+#define FM10K_SW_PCIE_CLKMON_RATIO_CFG FM10K_SW_MGMT_REG(0x3003)
+#define FM10K_SW_PCIE_CLKMON_TOLERANCE_CFG FM10K_SW_MGMT_REG(0x3004)
+#define FM10K_SW_PCIE_CLKMON_DEADLINES_CFG FM10K_SW_MGMT_REG(0x3005)
+#define FM10K_SW_PCIE_CLK_STAT FM10K_SW_MGMT_REG(0x3006)
+#define FM10K_SW_PCIE_CLK_IP FM10K_SW_MGMT_REG(0x3007)
+#define FM10K_SW_PCIE_CLK_IM FM10K_SW_MGMT_REG(0x3008)
+#define FM10K_SW_PCIE_WARM_RESET_DELAY FM10K_SW_MGMT_REG(0x3009)
+#define FM10K_SW_EPL_BASE FM10K_SW_REG_OFF(0x0E0000)
+#define FM10K_SW_EPL_PORT_REG(p_, r_) \
+ (FM10K_SW_EPL_BASE + FM10K_SW_REG_OFF(0x400 * (p_) + (r_)))
+#define FM10K_SW_EPL_LANE_REG(p_, l_, r_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x80 * (l_) + (r_))
+
+/* EPL enumerated types
+ */
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX 0
+#define FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1 1
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_L4 2
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_L4_XX 3
+#define FM10K_SW_EPL_QPL_MODE_XX_L4_XX_XX 4
+#define FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX 5
+#define FM10K_SW_EPL_PCS_SEL_DISABLE 0
+#define FM10K_SW_EPL_PCS_SEL_AN_73 1
+#define FM10K_SW_EPL_PCS_SEL_SGMII_10 2
+#define FM10K_SW_EPL_PCS_SEL_SGMII_100 3
+#define FM10K_SW_EPL_PCS_SEL_SGMII_1000 4
+#define FM10K_SW_EPL_PCS_SEL_1000BASEX 5
+#define FM10K_SW_EPL_PCS_SEL_10GBASER 6
+#define FM10K_SW_EPL_PCS_SEL_40GBASER 7
+#define FM10K_SW_EPL_PCS_SEL_100GBASER 8
+
+#define FM10K_SW_LANE_OVERRIDE_NORMAL 0
+#define FM10K_SW_LANE_OVERRIDE_FORCE_GOOD 1
+#define FM10K_SW_LANE_OVERRIDE_FORCE_BAD 2
+
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSTHRU 0
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSSTHRU_CHECK 1
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_GOOD 2
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_BAD 3
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL 4
+
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_DRAIN 0
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL 1
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_NORMAL 2
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_HOLD 3
+
+#define FM10K_SW_TX_MAC_FAULT_MODE_NORMAL 0
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_IDLE 1
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LOCAL_FAULT 2
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_REMOTE_FAULT 3
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LINK_INTERRUPTION 4
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_OK 5
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_ERROR 6
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_USER_VAL 7
+
+#define FM10K_SW_EPL_IP(p_) FM10K_SW_EPL_PORT_REG((p_), 0x300)
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_lsb 0
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_msb 3
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT(n_) (1 << ((n_) + 0))
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_lsb 4
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_msb 7
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT(n_) (1 << ((n_) + 4))
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_lsb 8
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_msb 11
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT(n_) (1 << ((n_) + 8))
+#define FM10K_SW_EPL_IP_ERROR_INTERRUPT (1 << 12)
+#define FM10K_SW_EPL_ERROR_IP(p_) FM10K_SW_EPL_PORT_REG((p_), 0x301)
+#define FM10K_SW_EPL_ERROR_IM(p_) FM10K_SW_EPL_PORT_REG((p_), 0x302)
+#define FM10K_SW_EPL_ERROR_IM_ALL FM10K_SW_MASK32(8, 0)
+#define FM10K_SW_EPL_BIST_STATUS(p_) FM10K_SW_EPL_PORT_REG((p_), 0x303)
+#define FM10K_SW_EPL_CFG_A(p_) FM10K_SW_EPL_PORT_REG((p_), 0x304)
+#define FM10K_SW_EPL_CFG_A_SPEED_UP (1 << 0)
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_lsb 1
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_msb 6
+#define FM10K_SW_EPL_CFG_A_ACTIVE(p_) (1 << ((p_) + 7))
+#define FM10K_SW_EPL_CFG_A_ACTIVE_QUAD (0xf << 7)
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_lsb 11
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_msb 16
+#define FM10K_SW_EPL_CFG_B(p_) FM10K_SW_EPL_PORT_REG((p_), 0x305)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_lsb(p_) ((p_) * 4)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_msb(p_) (((p_) * 4) + 3)
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_lsb 16
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_msb 18
+#define FM10K_SW_EPL_LED_STATUS(p_) FM10K_SW_EPL_PORT_REG((p_), 0x306)
+#define FM10K_SW_EPL_LED_STATUS_PORT_RESET(n_) (1 << ((n_) * 6 + 0))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP(n_) (1 << ((n_) * 6 + 1))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LOCAL_FAULT(n_) \
+ (1 << ((n_) * 6 + 2))
+#define FM10K_SW_EPL_LED_STATUS_PORT_REMOTE_FAULT(n_) \
+ (1 << ((n_) * 6 + 3))
+#define FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING(n_) \
+ (1 << ((n_) * 6 + 4))
+#define FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING(n_) \
+ (1 << ((n_) * 6 + 5))
+#define FM10K_SW_EPL_FIFO_ERROR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x307)
+#define FM10K_SW_EPL_TX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x308)
+#define FM10K_SW_EPL_TX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x309)
+#define FM10K_SW_EPL_TX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30A)
+#define FM10K_SW_EPL_TX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30B)
+#define FM10K_SW_EPL_TX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30C)
+#define FM10K_SW_EPL_TX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30D)
+#define FM10K_SW_EPL_RX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30E)
+#define FM10K_SW_EPL_RX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30F)
+#define FM10K_SW_EPL_RX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x310)
+#define FM10K_SW_EPL_RX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x311)
+#define FM10K_SW_EPL_RX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x312)
+#define FM10K_SW_EPL_RX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x313)
+#define FM10K_SW_PCS_ML_BASER_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x314)
+#define FM10K_SW_PCS_ML_BASER_RX_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x315)
+#define FM10K_SW_PCS_100GBASER_BIP_0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x317)
+#define FM10K_SW_PCS_100GBASER_BIP_1(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x318)
+#define FM10K_SW_PCS_100GBASER_BIP_2(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x319)
+#define FM10K_SW_PCS_100GBASER_BIP_3(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31A)
+#define FM10K_SW_PCS_100GBASER_BIP_4(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31B)
+#define FM10K_SW_PCS_100GBASER_BIP_5(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31C)
+#define FM10K_SW_PCS_100GBASER_BIP_6(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31D)
+#define FM10K_SW_PCS_100GBASER_BIP_7(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31E)
+#define FM10K_SW_PCS_100GBASER_BIP_8(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31F)
+#define FM10K_SW_PCS_100GBASER_BIP_9(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x320)
+#define FM10K_SW_PCS_100GBASER_BIP_10(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x321)
+#define FM10K_SW_PCS_100GBASER_BIP_11(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x322)
+#define FM10K_SW_PCS_100GBASER_BIP_12(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x323)
+#define FM10K_SW_PCS_100GBASER_BIP_13(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x324)
+#define FM10K_SW_PCS_100GBASER_BIP_14(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x325)
+#define FM10K_SW_PCS_100GBASER_BIP_15(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x326)
+#define FM10K_SW_PCS_100GBASER_BIP_16(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x327)
+#define FM10K_SW_PCS_100GBASER_BIP_17(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x328)
+#define FM10K_SW_PCS_100GBASER_BIP_18(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x329)
+#define FM10K_SW_PCS_100GBASER_BIP_19(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32A)
+#define FM10K_SW_PCS_100GBASER_BLOCK_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32B)
+#define FM10K_SW_PCS_100GBASER_AMPS_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32C)
+#define FM10K_SW_RS_FEC_UNCORRECTED(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x33D)
+#define FM10K_SW_RS_FEC_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32E)
+#define FM10K_SW_RS_FEC_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x330)
+#define FM10K_SW_EPL_SYSTIME(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x331)
+#define FM10K_SW_EPL_SYSTIME0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x332)
+#define FM10K_SW_EPL_SYSTIME_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x333)
+#define FM10K_SW_PORT_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x0)
+#define FM10K_SW_AN_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1)
+#define FM10K_SW_AN_IM_ALL FM10K_SW_MASK32(18, 0)
+#define FM10K_SW_LINK_IM(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2)
+#define FM10K_SW_LINK_IM_ALL FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_AN_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3)
+#define FM10K_SW_LINK_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4)
+#define FM10K_SW_MP_CFG(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_73_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_LINK_RULES(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xC)
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_lsb 0
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_msb 3
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_lsb 4
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_msb 8
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_lsb 9
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_msb 12
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_lsb 13
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_msb 17
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_lsb 18
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_msb 21
+#define FM10K_SW_MAC_CFG(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x10)
+#define FM10K_SW_MAC_CFG_ARRAY_SIZE 8
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_lsb 0
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_msb 5
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_lsb 6
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_msb 9
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_lsb 10
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_msb 17
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_lsb 18
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_msb 25
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_lsb 26
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_msb 31
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_lsb 32
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_msb 47
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE_bit 48
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_lsb 49
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_msb 51
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_lsb 52
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_msb 55
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_lsb 56
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_msb 63
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_lsb 64
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_msb 65
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_lsb 66
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_msb 71
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_lsb 72
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_msb 83
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_lsb 84
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_msb 95
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_lsb 96
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_msb 107
+#define FM10K_SW_MAC_CFG_TX_LP_IDLE_REQUEST_bit 108
+#define FM10K_SW_MAC_CFG_TX_LPI_AUTOMATIC_bit 109
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_lsb 110
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_msb 117
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_lsb 118
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_msb 119
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_lsb 120
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_msb 127
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_lsb 128
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_msb 129
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_lsb 130
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_msb 132
+#define FM10K_SW_MAC_CFG_TX_OBEY_LINT_bit 133
+#define FM10K_SW_MAC_CFG_TX_IDLE_ENABLE_DIC_bit 134
+#define FM10K_SW_MAC_CFG_CJPAT_ENABLE_bit 135
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_lsb 136
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_msb 143
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_lsb 144
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_msb 159
+#define FM10K_SW_MAC_CFG_START_CHAR_D_lsb 160
+#define FM10K_SW_MAC_CFG_START_CHAR_D_msb 167
+#define FM10K_SW_MAC_CFG_IEEE_1588_ENABLE_bit 168
+#define FM10K_SW_MAC_CFG_FCS_START_bit 169
+#define FM10K_SW_MAC_CFG_PREAMBLE_MODE_bit 170
+#define FM10K_SW_MAC_CFG_COUNTER_WRAP_bit 171
+#define FM10K_SW_MAC_CFG_LINK_FAULT_DISABLE_bit 172
+#define FM10K_SW_MAC_CFG_RX_DRAIN_bit 173
+#define FM10K_SW_MAC_CFG_RX_FCS_FORCE_BAD_bit 174
+#define FM10K_SW_MAC_CFG_RX_IGNORE_CODE_ERRORS_bit 175
+#define FM10K_SW_MAC_CFG_RX_IGNORE_UNDERSIZE_ERRORS_bit 176
+#define FM10K_SW_MAC_CFG_RX_IGNORE_OVERSIZE_ERRORS_bit 177
+#define FM10K_SW_MAC_CFG_RX_IGNORE_FCS_ERRORS_bit 178
+#define FM10K_SW_MAC_CFG_RX_IGNORE_PREAMBLE_ERRORS_bit 179
+#define FM10K_SW_MAC_CFG_RX_IGNORE_IFG_ERRORS_bit 180
+#define FM10K_SW_MAC_CFG_ERR_WRITE_lsb 181
+#define FM10K_SW_MAC_CFG_ERR_WRITE_msb 182
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_lsb 183
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_msb 186
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_lsb 187
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_msb 191
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_lsb 192
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_msb 196
+#define FM10K_SW_TX_SEQUENCE(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1A)
+#define FM10K_SW_RX_SEQUENCE(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1C)
+#define FM10K_SW_MAC_1588_STATUS(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1E)
+#define FM10K_SW_WAKE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x20)
+#define FM10K_SW_MAC_OVERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x21)
+#define FM10K_SW_MAC_JABBER_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x22)
+#define FM10K_SW_MAC_UNDERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x23)
+#define FM10K_SW_MAC_RUNT_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x24)
+#define FM10K_SW_MAC_OVERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x25)
+#define FM10K_SW_MAC_UNDERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x26)
+#define FM10K_SW_MAC_CODE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x27)
+#define FM10K_SW_EPL_TX_FRAME_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x28)
+#define FM10K_SW_MAC_LINK_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x29)
+#define FM10K_SW_PCS_1000BASEX_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2A)
+#define FM10K_SW_PCS_1000BASEX_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2B)
+#define FM10K_SW_PCS_1000BASEX_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2C)
+#define FM10K_SW_PCS_10GBASER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2D)
+#define FM10K_SW_PCS_10GBASER_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2E)
+#define FM10K_SW_PCS_10GBASER_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2F)
+#define FM10K_SW_AN_37_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x30)
+#define FM10K_SW_AN_37_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_37_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_SGMII_AN_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_SGMII_AN_TX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_TX_CONFIG_LOOPBACK(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_RX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x32)
+#define FM10K_SW_AN_73_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x33)
+#define FM10K_SW_AN_73_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_73_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x36)
+#define FM10K_SW_AN_73_TX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x38)
+#define FM10K_SW_AN_73_RX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3A)
+#define FM10K_SW_PCSL_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3C)
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_lsb 0
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_msb 7
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE (1 << 8)
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL (1 << 9)
+#define FM10K_SW_PCSL_CFG_RX_GB_NARROW (1 << 10)
+#define FM10K_SW_PCSL_CFG_TX_GB_NARROW (1 << 11)
+#define FM10K_SW_MP_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_1000BASEX_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_10GBASER_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_MP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_40GBASER_RX_BIP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_10GBASER_RX_BER_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_DISPARITY_ERROR_8B10B(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_LANE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x45)
+#define FM10K_SW_LANE_SERDES_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x46)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x47)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_msb 1
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT (1 << 5)
+#define FM10K_SW_LANE_ACTIVITY_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x48)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x49)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_msb 1
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT (1 << 5)
+#define FM10K_SW_LANE_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4A)
+#define FM10K_SW_LANE_SERDES_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4B)
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_lsb 0
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_msb 7
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_lsb 8
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_msb 23
+#define FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK (1 << 12)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_RDY (1 << 24)
+#define FM10K_SW_LANE_SERDES_STATUS_TX_RDY (1 << 25)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_IDLE_DETECT (1 << 26)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_ACTIVITY (1 << 27)
+#define FM10K_SW_SERDES_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4C)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IM_CORE_STATUS_4
+#define FM10K_SW_SERDES_IM_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IM_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IM_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IM_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IM_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IM_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IM_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IM_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IM_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IM_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IM_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_SERDES_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4D)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IP_CORE_STATUS_4
+#define FM10K_SW_SERDES_IP_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IP_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IP_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IP_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IP_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IP_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IP_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IP_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IP_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IP_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IP_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_LANE_ANALOG_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4E)
+#define FM10K_SW_LANE_ANALOG_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4F)
+#define FM10K_SW_LANE_SAI_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x50)
+#define FM10K_SW_LANE_SAI_CFG_CODE_lsb64 0
+#define FM10K_SW_LANE_SAI_CFG_CODE_msb64 15
+#define FM10K_SW_LANE_SAI_CFG_DATA_lsb64 16
+#define FM10K_SW_LANE_SAI_CFG_DATA_msb64 31
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_lsb64 32
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_msb64 47
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_lsb64 48
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_msb64 49
+#define FM10K_SW_LANE_SAI_CFG_REQUEST (1ULL << 50)
+#define FM10K_SW_LANE_SAI_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x52)
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_lsb 0
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_msb 15
+#define FM10K_SW_LANE_SAI_STATUS_COMPLETE (1 << 16)
+#define FM10K_SW_LANE_SAI_STATUS_ACCESS_REQUEST (1 << 17)
+#define FM10K_SW_LANE_SAI_STATUS_IN_PROGRESS (1 << 18)
+#define FM10K_SW_LANE_SAI_STATUS_BUSY (1 << 19)
+#define FM10K_SW_LANE_SAI_STATUS_REQUEST_ERROR (1 << 20)
+
+#define FM10K_SW_PORTS_MGMT_BASE FM10K_SW_REG_OFF(0x0E8000)
+#define FM10K_SW_PORTS_MGMT_REG(wo_) \
+ (FM10K_SW_PORTS_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_PLL_EPL_CTRL FM10K_SW_PORTS_MGMT_REG(0x0)
+#define FM10K_SW_PLL_EPL_STAT FM10K_SW_PORTS_MGMT_REG(0x1)
+#define FM10K_SW_PLL_FABRIC_CTRL FM10K_SW_PORTS_MGMT_REG(0x2)
+#define FM10K_SW_PLL_FABRIC_REFDIV_lsb 3
+#define FM10K_SW_PLL_FABRIC_REFDIV_msb 8
+#define FM10K_SW_PLL_FABRIC_FBDIV4_lsb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV4_msb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV255_lsb 10
+#define FM10K_SW_PLL_FABRIC_FBDIV255_msb 17
+#define FM10K_SW_PLL_FABRIC_OUTDIV_lsb 18
+#define FM10K_SW_PLL_FABRIC_OUTDIV_msb 23
+#define FM10K_SW_PLL_FABRIC_STAT FM10K_SW_PORTS_MGMT_REG(0x3)
+#define FM10K_SW_PLL_FABRIC_LOCK FM10K_SW_PORTS_MGMT_REG(0x4)
+#define FM10K_SW_PLL_FABRIC_FREQSEL_lsb 4
+#define FM10K_SW_PLL_FABRIC_FREQSEL_msb 7
+#define FM10K_SW_PLL_FABRIC_FREQSEL_CTRL 0
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F600 1
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F500 2
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F400 3
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F300 4
+#define FM10K_SW_SBUS_EPL_CFG FM10K_SW_PORTS_MGMT_REG(0x5)
+#define FM10K_SW_SBUS_EPL_COMMAND FM10K_SW_PORTS_MGMT_REG(0x6)
+#define FM10K_SW_SBUS_EPL_REQUEST FM10K_SW_PORTS_MGMT_REG(0x7)
+#define FM10K_SW_SBUS_EPL_RESPONSE FM10K_SW_PORTS_MGMT_REG(0x8)
+#define FM10K_SW_SBUS_EPL_SPICO_IN FM10K_SW_PORTS_MGMT_REG(0x9)
+#define FM10K_SW_SBUS_EPL_SPICO_OUT FM10K_SW_PORTS_MGMT_REG(0xA)
+#define FM10K_SW_SBUS_EPL_IP FM10K_SW_PORTS_MGMT_REG(0xB)
+#define FM10K_SW_SBUS_EPL_IM FM10K_SW_PORTS_MGMT_REG(0xC)
+#define FM10K_SW_ETHCLK_CFG(n_) FM10K_SW_PORTS_MGMT_REG(0xE + (n_))
+#define FM10K_SW_ETHCLK_RATIO(n_) FM10K_SW_PORTS_MGMT_REG(0x10 + (n_))
+#define FM10K_SW_PM_CLKOBS_CTRL FM10K_SW_PORTS_MGMT_REG(0x12)
+
+#define FM10K_SW_PCIE_CFG_BASE FM10K_SW_REG_OFF(0x120000)
+#define FM10K_SW_PCIE_CFG_REG(wo_) \
+ (FM10K_SW_PCIE_CFG_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * These register offsets can also be passed to fm10k_read_config(), which
+ * will mask off the upper bits
+ */
+#define FM10K_SW_PCIE_CFG_ID FM10K_SW_PCIE_CFG_REG(0x0)
+#define FM10K_SW_PCIE_CFG_CMD FM10K_SW_PCIE_CFG_REG(0x1)
+#define FM10K_SW_PCIE_CFG_1 FM10K_SW_PCIE_CFG_REG(0x2)
+#define FM10K_SW_PCIE_CFG_2 FM10K_SW_PCIE_CFG_REG(0x3)
+#define FM10K_SW_PCIE_CFG_BAR0 FM10K_SW_PCIE_CFG_REG(0x4)
+#define FM10K_SW_PCIE_CFG_BAR1 FM10K_SW_PCIE_CFG_REG(0x5)
+#define FM10K_SW_PCIE_CFG_BAR2 FM10K_SW_PCIE_CFG_REG(0x6)
+#define FM10K_SW_PCIE_CFG_BAR3 FM10K_SW_PCIE_CFG_REG(0x7)
+#define FM10K_SW_PCIE_CFG_BAR4 FM10K_SW_PCIE_CFG_REG(0x8)
+#define FM10K_SW_PCIE_CFG_BAR5 FM10K_SW_PCIE_CFG_REG(0x9)
+#define FM10K_SW_PCIE_CFG_CARDBUS FM10K_SW_PCIE_CFG_REG(0xA)
+#define FM10K_SW_PCIE_CFG_SUBID FM10K_SW_PCIE_CFG_REG(0xB)
+#define FM10K_SW_PCIE_CFG_EXP_ROM FM10K_SW_PCIE_CFG_REG(0xC)
+#define FM10K_SW_PCIE_CFG_CAP_PTR FM10K_SW_PCIE_CFG_REG(0xD)
+#define FM10K_SW_PCIE_CFG_RSVD FM10K_SW_PCIE_CFG_REG(0xE)
+#define FM10K_SW_PCIE_CFG_INT FM10K_SW_PCIE_CFG_REG(0xF)
+#define FM10K_SW_PCIE_CFG_PM_CAP FM10K_SW_PCIE_CFG_REG(0x10)
+#define FM10K_SW_PCIE_CFG_PM_CTRL FM10K_SW_PCIE_CFG_REG(0x11)
+#define FM10K_SW_PCIE_CFG_PCIE_CAP FM10K_SW_PCIE_CFG_REG(0x1C)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP FM10K_SW_PCIE_CFG_REG(0x1D)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL FM10K_SW_PCIE_CFG_REG(0x1E)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CAP FM10K_SW_PCIE_CFG_REG(0x1F)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL FM10K_SW_PCIE_CFG_REG(0x20)
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_lsb 16
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_msb 19
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_2P5 1
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_5 2
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_8 3
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_lsb 20
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_msb 24
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP2 FM10K_SW_PCIE_CFG_REG(0x25)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL2 FM10K_SW_PCIE_CFG_REG(0x26)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL2 FM10K_SW_PCIE_CFG_REG(0x28)
+#define FM10K_SW_PCIE_CFG_MSIX_CAP FM10K_SW_PCIE_CFG_REG(0x2C)
+#define FM10K_SW_PCIE_CFG_MSIX_TABLE_OFFSET FM10K_SW_PCIE_CFG_REG(0x2D)
+#define FM10K_SW_PCIE_CFG_MSIX_PBA FM10K_SW_PCIE_CFG_REG(0x2E)
+#define FM10K_SW_PCIE_CFG_VPD_CAP FM10K_SW_PCIE_CFG_REG(0x34)
+#define FM10K_SW_PCIE_CFG_VPD_DATA FM10K_SW_PCIE_CFG_REG(0x35)
+#define FM10K_SW_PCIE_CFG_AER_HDR FM10K_SW_PCIE_CFG_REG(0x40)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_STATUS FM10K_SW_PCIE_CFG_REG(0x41)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_MASK FM10K_SW_PCIE_CFG_REG(0x42)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_SEVERITY FM10K_SW_PCIE_CFG_REG(0x43)
+#define FM10K_SW_PCIE_CFG_AER_COERR_STATUS FM10K_SW_PCIE_CFG_REG(0x44)
+#define FM10K_SW_PCIE_CFG_AER_COERR_MASK FM10K_SW_PCIE_CFG_REG(0x45)
+#define FM10K_SW_PCIE_CFG_AER_CTRL FM10K_SW_PCIE_CFG_REG(0x46)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG0 FM10K_SW_PCIE_CFG_REG(0x47)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG1 FM10K_SW_PCIE_CFG_REG(0x48)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG2 FM10K_SW_PCIE_CFG_REG(0x49)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG3 FM10K_SW_PCIE_CFG_REG(0x4A)
+#define FM10K_SW_PCIE_CFG_SPD_HDR FM10K_SW_PCIE_CFG_REG(0x52)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_L FM10K_SW_PCIE_CFG_REG(0x53)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_H FM10K_SW_PCIE_CFG_REG(0x54)
+#define FM10K_SW_PCIE_CFG_ARI_HDR FM10K_SW_PCIE_CFG_REG(0x56)
+#define FM10K_SW_PCIE_CFG_ARI_CTRL FM10K_SW_PCIE_CFG_REG(0x57)
+#define FM10K_SW_PCIE_CFG_SPCIE_HDR FM10K_SW_PCIE_CFG_REG(0x5A)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_CTRL3 FM10K_SW_PCIE_CFG_REG(0x5B)
+#define FM10K_SW_PCIE_CFG_SPCIE_ERR_STS FM10K_SW_PCIE_CFG_REG(0x5C)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ01 FM10K_SW_PCIE_CFG_REG(0x5D)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ23 FM10K_SW_PCIE_CFG_REG(0x5E)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ45 FM10K_SW_PCIE_CFG_REG(0x5F)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ67 FM10K_SW_PCIE_CFG_REG(0x60)
+#define FM10K_SW_PCIE_CFG_SRIOV_HDR FM10K_SW_PCIE_CFG_REG(0x62)
+#define FM10K_SW_PCIE_CFG_SRIOV_CAP FM10K_SW_PCIE_CFG_REG(0x63)
+#define FM10K_SW_PCIE_CFG_SRIOV_CTRL FM10K_SW_PCIE_CFG_REG(0x64)
+#define FM10K_SW_PCIE_CFG_SRIOV_CFG FM10K_SW_PCIE_CFG_REG(0x65)
+#define FM10K_SW_PCIE_CFG_SRIOV_NUM FM10K_SW_PCIE_CFG_REG(0x66)
+#define FM10K_SW_PCIE_CFG_SRIOV_MAP FM10K_SW_PCIE_CFG_REG(0x67)
+#define FM10K_SW_PCIE_CFG_SRIOV_DEVID FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_SUP FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_CFG FM10K_SW_PCIE_CFG_REG(0x6A)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR0 FM10K_SW_PCIE_CFG_REG(0x6B)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR1 FM10K_SW_PCIE_CFG_REG(0x6C)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR2 FM10K_SW_PCIE_CFG_REG(0x6D)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR3 FM10K_SW_PCIE_CFG_REG(0x6E)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR4 FM10K_SW_PCIE_CFG_REG(0x6F)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR5 FM10K_SW_PCIE_CFG_REG(0x70)
+#define FM10K_SW_PCIE_CFG_SRIOV_MIG FM10K_SW_PCIE_CFG_REG(0x71)
+#define FM10K_SW_PCIE_CFG_TPH_HDR FM10K_SW_PCIE_CFG_REG(0x72)
+#define FM10K_SW_PCIE_CFG_TPH_CAP FM10K_SW_PCIE_CFG_REG(0x73)
+#define FM10K_SW_PCIE_CFG_TPH_CTRL FM10K_SW_PCIE_CFG_REG(0x74)
+#define FM10K_SW_PCIE_CFG_ACS_HDR FM10K_SW_PCIE_CFG_REG(0x76)
+#define FM10K_SW_PCIE_CFG_ACS_CAP FM10K_SW_PCIE_CFG_REG(0x77)
+#define FM10K_SW_PCIE_PORTLOGIC FM10K_SW_PCIE_CFG_REG(0x1C0)
+#define FM10K_SW_PCIE_PORTLOGIC_LINK_STATE FM10K_SW_PCIE_CFG_REG(0x1CA)
+
+
+#define FM10K_SW_FFU_BASE FM10K_SW_REG_OFF(0xC00000)
+#define FM10K_SW_FFU_REG(wo_) (FM10K_SW_FFU_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_FFU_NUM_SLICES 32
+#define FM10K_SW_FFU_NUM_SCENARIOS 32
+
+/* FFU enumerated types */
+#define FM10K_SW_FFU_MUX_SEL_MAP_DIP_MAP_SIP 0
+#define FM10K_SW_FFU_MUX_SEL_MAP_DMAC_MAP_SMAC 1
+#define FM10K_SW_FFU_MUX_SEL_MAP_PROT_MAP_LENGTH 2
+#define FM10K_SW_FFU_MUX_SEL_MAP_SRC_MAP_TYPE 3
+#define FM10K_SW_FFU_MUX_SEL_USER 4
+#define FM10K_SW_FFU_MUX_SEL_FTYPE_SWPRI 5
+#define FM10K_SW_FFU_MUX_SEL_IPMISC 6
+#define FM10K_SW_FFU_MUX_SEL_TOS 7
+#define FM10K_SW_FFU_MUX_SEL_PROT 8
+#define FM10K_SW_FFU_MUX_SEL_TTL 9
+#define FM10K_SW_FFU_MUX_SEL_SRC_PORT 10
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID_11_8 11
+#define FM10K_SW_FFU_MUX_SEL_VID_7_0 12
+#define FM10K_SW_FFU_MUX_SEL_RXTAG 13
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_15_0 14
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_31_16 15
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_47_32 16
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_15_0 17
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_31_16 18
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_47_32 19
+#define FM10K_SW_FFU_MUX_SEL_DGLORT 20
+#define FM10K_SW_FFU_MUX_SEL_SGLORT 21
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID 22
+#define FM10K_SW_FFU_MUX_SEL_VPRI2_VID2 23
+#define FM10K_SW_FFU_MUX_SEL_L2_TYPE 24
+#define FM10K_SW_FFU_MUX_SEL_L4_DST 25
+#define FM10K_SW_FFU_MUX_SEL_L4_SRC 26
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_DST 27
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_SRC 28
+#define FM10K_SW_FFU_MUX_SEL_L4A 29
+#define FM10K_SW_FFU_MUX_SEL_L4B 30
+#define FM10K_SW_FFU_MUX_SEL_L4C 31
+#define FM10K_SW_FFU_MUX_SEL_L4D 32
+#define FM10K_SW_FFU_MUX_SEL_MAP_VPRI1_VID1 33
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_31_0 34
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_63_32 35
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_95_64 36
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_127_96 37
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_31_0 38
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_63_32 39
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64 40
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96 41
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_msb 19
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_TYPE_EXP (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_FLOOD_SET (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_msb 7
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_lsb 8
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_msb 20
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_msb 11
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_msb 17
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_PRI (1 << 19)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_msb 5
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_DSCP (1 << 17)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_PRI (1 << 19)
+
+/* FFU Registers */
+#define FM10K_SW_FFU_SLICE_TCAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x2000 * (sl_) + 0x4 * (n_))
+#define FM10K_SW_FFU_SLICE_TCAM_ENTRIES 1024
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_lsb64 0
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_msb64 31
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_lsb64 32
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_msb64 39
+#define FM10K_SW_FFU_SLICE_SRAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x1000 + 0x2000 * (sl_) + 0x2 * (n_))
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_lsb64 21
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_msb64 22
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_ARP 0
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT 1
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_BIT_SET 2
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET 3
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_lsb64 23
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_msb64 34
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_lsb64 35
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_msb64 36
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_lsb64 37
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_msb64 39
+#define FM10K_SW_FFU_SLICE_VALID(sl_) \
+ FM10K_SW_FFU_REG(0x1800 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_VALID_SCENARIO(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_FFU_SLICE_CASCADE_ACTION(sl_) \
+ FM10K_SW_FFU_REG(0x1804 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG(sl_, scen_) \
+ FM10K_SW_FFU_REG(0x1840 + 0x2 * (scen_) + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_lsb64 0
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_msb64 5
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_lsb64 6
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_msb64 11
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_lsb64 12
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_msb64 17
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_lsb64 18
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_msb64 23
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_lsb64 24
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_msb64 29
+#define FM10K_SW_FFU_SLICE_CFG_START_COMPARE (1ULL << 30)
+#define FM10K_SW_FFU_SLICE_CFG_START_ACTION (1ULL << 31)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_LOW (1ULL << 32)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_HIGH (1ULL << 33)
+#define FM10K_SW_FFU_SLICE_CFG_CASE_lsb64 34
+#define FM10K_SW_FFU_SLICE_CFG_CASE_msb64 37
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_lsb64 38
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_msb64 39
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_NOT_MAPPED 0
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_LOW_NIBBLE 1
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_HIGH_NIBBLE 2
+#define FM10K_SW_FFU_MASTER_VALID FM10K_SW_FFU_REG(0x40000)
+#define FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_SLICES_VALID FM10K_SW_MASK64(31, 0)
+#define FM10K_SW_FFU_MASTER_VALID_CHUNK_VALID(c_) (1ULL << ((c_) + 32))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_CHUNKS_VALID FM10K_SW_MASK64(63, 32)
+
+#define FM10K_SW_L2LOOKUP_BASE FM10K_SW_REG_OFF(0xC80000)
+#define FM10K_SW_L2LOOKUP_REG(wo_) \
+ (FM10K_SW_L2LOOKUP_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_MA_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x10000 * (t_) + 0x4 * (n_))
+#define FM10K_SW_INGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x20000 + 0x4 * (v_))
+#define FM10K_SW_INGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative
+ * to the start of the upper 64-bit word
+ */
+#define FM10K_SW_INGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_msb64 (89 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_REFLECT (1ULL << (90 - 64))
+#define FM10K_SW_INGRESS_VID_TABLE_TRAP_IGMP (1ULL << (91 - 64))
+#define FM10K_SW_EGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x24000 + 0x4 * (v_))
+#define FM10K_SW_EGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative to
+ * the start of the upper 64-bit word
+ */
+#define FM10K_SW_EGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_msb64 (86 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_lsb64 (87 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_msb64 (92 - 64)
+#define FM10K_SW_MA_USED_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28000 + 0x200 * (t_) + (n_))
+#define FM10K_SW_INGRESS_MST_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28400 + 0x200 * (t_) + 0x2 * (n_))
+#define FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE 24
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) \
+ (((l_) % FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) * 2)
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_msb64(l_) \
+ (FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) + 1)
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_DISABLE 0
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LISTENING 1
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LEARNING 2
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD 3
+#define FM10K_SW_EGRESS_MST_TABLE(t_) \
+ FM10K_SW_L2LOOKUP_REG(0x28800 + 0x2 * (t_))
+#define FM10K_SW_EGRESS_MST_TABLE_FORWARDING(l_) (1ULL << (l_))
+#define FM10K_SW_MA_TABLE_CFG_1 FM10K_SW_L2LOOKUP_REG(0x28A00)
+#define FM10K_SW_MA_TABLE_CFG_2 FM10K_SW_L2LOOKUP_REG(0x28A02)
+#define FM10K_SW_MTU_TABLE(n_) FM10K_SW_L2LOOKUP_REG(0x28A08 + (n_))
+#define FM10K_SW_IEEE_RESERVED_MAC_ACTION FM10K_SW_L2LOOKUP_REG(0x28A10)
+#define FM10K_SW_IEEE_RESERVED_MAC_TRAP_PRIORITY FM10K_SW_L2LOOKUP_REG(0x28A14)
+#define FM10K_SW_IEEE_RESERVED_MAC_CFG FM10K_SW_L2LOOKUP_REG(0x28A16)
+
+
+#define FM10K_SW_GLORT_BASE FM10K_SW_REG_OFF(0xCE0000)
+#define FM10K_SW_GLORT_REG(wo_) \
+ (FM10K_SW_GLORT_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_GLORT_CAM_ENTRIES 256
+#define FM10K_SW_GLORT_DEST_TABLE(n_) FM10K_SW_GLORT_REG(0x0 + 0x2 * (n_))
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_lsb64 0
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_msb64 47
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_lsb64 48
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_msb64 59
+#define FM10K_SW_GLORT_CAM(n_) FM10K_SW_GLORT_REG(0x2000 + (n_))
+#define FM10K_SW_GLORT_CAM_MATCH_ANY 0x00000000
+#define FM10K_SW_GLORT_CAM_MATCH_NONE FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_GLORT_CAM_KEY_lsb 0
+#define FM10K_SW_GLORT_CAM_KEY_msb 15
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_lsb 16
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_msb 31
+#define FM10K_SW_GLORT_RAM(n_) FM10K_SW_GLORT_REG(0x2200 + 0x2 * (n_))
+#define FM10K_SW_GLORT_RAM_STRICT_lsb64 0
+#define FM10K_SW_GLORT_RAM_STRICT_msb64 1
+#define FM10K_SW_GLORT_RAM_STRICT_FTYPE 0
+#define FM10K_SW_GLORT_RAM_STRICT_RSVD 1
+#define FM10K_SW_GLORT_RAM_STRICT_HASHED 2
+#define FM10K_SW_GLORT_RAM_STRICT_STRICT 3
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_lsb64 2
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_msb64 13
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_lsb64 14
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_msb64 21
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_lsb64 22
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_msb64 29
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_lsb64 30
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_msb64 33
+#define FM10K_SW_GLORT_RAM_HASH_ROTATION (1ULL << 34)
+
+
+#define FM10K_SW_PARSER_BASE FM10K_SW_REG_OFF(0xCF0000)
+#define FM10K_SW_PARSER_REG(wo_) \
+ (FM10K_SW_PARSER_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_PARSER_PORT_CFG_1(p_) \
+ FM10K_SW_PARSER_REG(0x0 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_1_FTAG (1ULL << 0)
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(v_) (1ULL << (1 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(v_) (1ULL << (5 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_FIRST (1ULL << 9)
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_lsb 10
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_msb 21
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_lsb 22
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_lsb 26
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_msb 37
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_lsb 38
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_msb 41
+#define FM10K_SW_PARSER_PORT_CFG_1_USE_DEFAULT_VLAN (1ULL << 42)
+#define FM10K_SW_PARSER_PORT_CFG_2(p_) \
+ FM10K_SW_PARSER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_lsb 0
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_msb 3
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_lsb 4
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_msb 7
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_MPLS (1ULL << 8)
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_lsb 9
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_msb 11
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3 (1ULL << 12)
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4 (1ULL << 13)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV4_OPTIONS (1ULL << 14)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_HOP_BY_HOP (1ULL << 15)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_ROUTING (1ULL << 16)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_FRAG (1ULL << 17)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_DEST (1ULL << 18)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_AUTH (1ULL << 19)
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_lsb 20
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_TAGGED (1ULL << 26)
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_UNTAGGED (1ULL << 27)
+#define FM10K_SW_PARSER_PORT_CFG_2_USE_DEFAULT_DSCP (1ULL << 28)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_VLAN (1ULL << 29)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_DSCP (1ULL << 30)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_ISL (1ULL << 31)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_PREFER_DSCP (1ULL << 32)
+#define FM10K_SW_PARSER_PORT_CFG_3(p_) FM10K_SW_PARSER_REG(0x100 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_ISL(p_) FM10K_SW_PARSER_REG(0x180 + (p_))
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_lsb 0
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_msb 15
+#define FM10K_SW_PORT_CFG_ISL_USR_lsb 16
+#define FM10K_SW_PORT_CFG_ISL_USR_msb 23
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_lsb 24
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_msb 27
+#define FM10K_SW_PARSER_VLAN_TAG(n_) FM10K_SW_PARSER_REG(0x1C0 + (n_))
+#define FM10K_SW_PARSER_CUSTOM_TAG(n_) FM10K_SW_PARSER_REG(0x1C4 + (n_))
+#define FM10K_SW_PARSER_MPLS_TAG FM10K_SW_PARSER_REG(0x1C8)
+#define FM10K_SW_PARSER_DI_CFG(n_) FM10K_SW_PARSER_REG(0x1D0 + 0x2 * (n_))
+#define FM10K_SW_RX_VPRI_MAP(p_) FM10K_SW_PARSER_REG(0x200 + 0x2 * (p_))
+#define FM10K_SW_DSCP_PRI_MAP(pri_) FM10K_SW_PARSER_REG(0x280 + (pri_))
+#define FM10K_SW_VPRI_PRI_MAP(pri_) FM10K_SW_PARSER_REG(0x2C0 + (pri_))
+
+
+#define FM10K_SW_HANDLER_BASE FM10K_SW_REG_OFF(0xD50000)
+#define FM10K_SW_HANDLER_REG(wo_) \
+ (FM10K_SW_HANDLER_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_SYS_CFG_1 FM10K_SW_HANDLER_REG(0x0)
+#define FM10K_SW_SYS_CFG_1_DROP_PAUSE (1 << 0)
+#define FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS (1 << 1)
+#define FM10K_SW_SYS_CFG_1_ENABLE_TRAP_PLUS_LOG (1 << 2)
+#define FM10K_SW_SYS_CFG_1_DROP_INVALID_SMAC (1 << 3)
+#define FM10K_SW_SYS_CFG_1_DROP_MAC_CTRL_ETHERTYPE (1 << 4)
+#define FM10K_SW_CPU_MAC FM10K_SW_HANDLER_REG(0x2)
+#define FM10K_SW_SYS_CFG_ROUTER FM10K_SW_HANDLER_REG(0x4)
+#define FM10K_SW_L34_HASH_CFG FM10K_SW_HANDLER_REG(0x5)
+#define FM10K_SW_L34_HASH_CFG_SYMMETRIC (1 << 0)
+#define FM10K_SW_L34_HASH_CFG_USE_SIP (1 << 1)
+#define FM10K_SW_L34_HASH_CFG_USE_DIP (1 << 2)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT (1 << 3)
+#define FM10K_SW_L34_HASH_CFG_USE_TCP (1 << 4)
+#define FM10K_SW_L34_HASH_CFG_USE_UDP (1 << 5)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT1 (1 << 6)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT2 (1 << 7)
+#define FM10K_SW_L34_HASH_CFG_USE_L4SRC (1 << 8)
+#define FM10K_SW_L34_HASH_CFG_USE_L4DST (1 << 9)
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_lsb 10
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_msb 11
+#define FM10K_SW_L34_HASH_CFG_PROT1_lsb 16
+#define FM10K_SW_L34_HASH_CFG_PROT1_msb 23
+#define FM10K_SW_L34_HASH_CFG_PROT2_lsb 24
+#define FM10K_SW_L34_HASH_CFG_PROT2_msb 31
+#define FM10K_SW_L34_FLOW_HASH_CFG_1 FM10K_SW_HANDLER_REG(0x6)
+#define FM10K_SW_L34_FLOW_HASH_CFG_2 FM10K_SW_HANDLER_REG(0x7)
+#define FM10K_SW_L234_HASH_CFG FM10K_SW_HANDLER_REG(0x8)
+#define FM10K_SW_L234_HASH_CFG_USE_L2_IF_IP (1 << 0)
+#define FM10K_SW_L234_HASH_CFG_USE_L34 (1 << 1)
+#define FM10K_SW_L234_HASH_CFG_SYMMETRIC (1 << 2)
+#define FM10K_SW_L234_HASH_CFG_USE_DMAC (1 << 3)
+#define FM10K_SW_L234_HASH_CFG_USE_SMAC (1 << 4)
+#define FM10K_SW_L234_HASH_CFG_USE_TYPE (1 << 5)
+#define FM10K_SW_L234_HASH_CFG_USE_VPRI (1 << 6)
+#define FM10K_SW_L234_HASH_CFG_USE_VID (1 << 8)
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_lsb 8
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_msb 9
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_lsb 10
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_msb 11
+#define FM10K_SW_CPU_TRAP_MASK_FH FM10K_SW_HANDLER_REG(0xA)
+#define FM10K_SW_TRAP_GLORT FM10K_SW_HANDLER_REG(0xC)
+#define FM10K_SW_RX_MIRROR_CFG FM10K_SW_HANDLER_REG(0xD)
+#define FM10K_SW_LOG_MIRROR_PROFILE FM10K_SW_HANDLER_REG(0xE)
+#define FM10K_SW_FH_MIRROR_PROFILE_TABLE(n_) FM10K_SW_HANDLER_REG(0x40 + (n_))
+#define FM10K_SW_PORT_CFG_2(p_) FM10K_SW_HANDLER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_3(p_) FM10K_SW_HANDLER_REG(0x100 + (p_))
+#define FM10K_SW_FH_LOOPBACK_SUPPRESS(p_) FM10K_SW_HANDLER_REG(0x140 + (p_))
+#define FM10K_SW_FH_HEAD_IP FM10K_SW_HANDLER_REG(0x180)
+#define FM10K_SW_FH_HEAD_IM FM10K_SW_HANDLER_REG(0x182)
+#define FM10K_SW_PARSER_EARLY_SRAM_CTRL FM10K_SW_HANDLER_REG(0x184)
+#define FM10K_SW_PARSER_LATE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x185)
+#define FM10K_SW_MAPPER_SRAM_CTRL FM10K_SW_HANDLER_REG(0x186)
+#define FM10K_SW_FFU_SRAM_CTRL(n_) FM10K_SW_HANDLER_REG(0x1A0 + 0x4 * (n_))
+#define FM10K_SW_ARP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C0)
+#define FM10K_SW_VLAN_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C1)
+#define FM10K_SW_MA_TABLE_SRAM_CTRL(n_) FM10K_SW_HANDLER_REG(0x1C4 + 0x2 * (n_))
+#define FM10K_SW_FID_GLORT_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C8)
+#define FM10K_SW_GLORT_RAM_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CA)
+#define FM10K_SW_GLORT_TABLE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CB)
+#define FM10K_SW_FH_HEAD_OUTPUT_FIFO_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CC)
+
+
+#define FM10K_SW_LAG_BASE FM10K_SW_REG_OFF(0xD90000)
+#define FM10K_SW_LAG_REG(wo_) (FM10K_SW_LAG_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_LAG_CFG(l_) FM10K_SW_LAG_REG(l_)
+#define FM10K_SW_LAG_CFG_LAG_SIZE_lsb 0
+#define FM10K_SW_LAG_CFG_LAG_SIZE_msb 3
+#define FM10K_SW_LAG_CFG_INDEX_lsb 4
+#define FM10K_SW_LAG_CFG_INDEX_msb 7
+#define FM10K_SW_LAG_CFG_HASH_ROTATION (1 << 8)
+#define FM10K_SW_LAG_CFG_IN_LAG (1 << 9)
+#define FM10K_SW_CANONICAL_GLORT_CAM(n_) FM10K_SW_LAG_REG((n_) + 0x40)
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_lsb 0
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_msb 15
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_lsb 16
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_msb 19
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_lsb 20
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_msb 22
+
+
+#define FM10K_SW_RX_STATS_BASE FM10K_SW_REG_OFF(0xE00000)
+#define FM10K_SW_RX_STATS_REG(wo_) \
+ (FM10K_SW_RX_STATS_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_RX_STATS_BANK(b_, i_) \
+ FM10K_SW_RX_STATS_REG(0x1000 * (b_) + 0x4 * (i_))
+#define FM10K_SW_RX_STATS_BANK_1_INDEX 0
+#define FM10K_SW_RX_STATS_BANK_2_INDEX 1
+#define FM10K_SW_RX_STATS_BANK_3_INDEX 2
+#define FM10K_SW_RX_STATS_BANK_4_INDEX 3
+#define FM10K_SW_RX_STATS_BANK_5_INDEX 4
+#define FM10K_SW_RX_STATS_BANK_6_INDEX 5
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_UCAST 0
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_MCAST 1
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_BCAST 2
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_UCAST 3
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_MCAST 4
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_BCAST 5
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_UCAST 6
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_MCAST 7
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_BCAST 8
+#define FM10K_SW_RX_STATS_BANK_1_IEEE802_3_PAUSE 9
+#define FM10K_SW_RX_STATS_BANK_1_CLASS_BASED_PAUSE 10
+#define FM10K_SW_RX_STATS_BANK_1_FRAMING_ERR 11
+#define FM10K_SW_RX_STATS_BANK_1_FCS_ERR 12
+#define FM10K_SW_RX_STATS_BANK_2_LEN_LT_64 0
+#define FM10K_SW_RX_STATS_BANK_2_LEN_EQ_64 1
+#define FM10K_SW_RX_STATS_BANK_2_LEN_65_127 2
+#define FM10K_SW_RX_STATS_BANK_2_LEN_128_255 3
+#define FM10K_SW_RX_STATS_BANK_2_LEN_256_511 4
+#define FM10K_SW_RX_STATS_BANK_2_LEN_512_1023 5
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1024_1522 6
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1523_2047 7
+#define FM10K_SW_RX_STATS_BANK_2_LEN_2048_4095 8
+#define FM10K_SW_RX_STATS_BANK_2_LEN_4096_8191 9
+#define FM10K_SW_RX_STATS_BANK_2_LEN_8192_10239 10
+#define FM10K_SW_RX_STATS_BANK_2_LEN_GE_10240 11
+#define FM10K_SW_RX_STATS_BANK_3_PRI(p_) (p_)
+#define FM10K_SW_RX_STATS_BANK_4_FID_FORWARDED 0
+#define FM10K_SW_RX_STATS_BANK_4_FLOOD_FORWARDED 1
+#define FM10K_SW_RX_STATS_BANK_4_SPECIALLY_HANDLED 2
+#define FM10K_SW_RX_STATS_BANK_4_PARSER_ERROR_DROP 3
+#define FM10K_SW_RX_STATS_BANK_4_ECC_ERROR_DROP 4
+#define FM10K_SW_RX_STATS_BANK_4_TRAPPED 5
+#define FM10K_SW_RX_STATS_BANK_4_PAUSE_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_4_STP_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_4_SECURITY_VIOLATIONS 8
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_TAG_DROPS 9
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_INGRESS_DROPS 10
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_EGRESS_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_4_GLORT_MISS_DROPS 12
+#define FM10K_SW_RX_STATS_BANK_4_FFU_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_4_TRIGGER_DROPS 14
+#define FM10K_SW_RX_STATS_BANK_5_POLICER_DROPS 0
+#define FM10K_SW_RX_STATS_BANK_5_TTL_DROPS 1
+#define FM10K_SW_RX_STATS_BANK_5_CM_PRIV_DROPS 2
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP0_DROPS 3
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP1_DROPS 4
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG0_DROPS 5
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG1_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG0_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG1_DROPS 8
+#define FM10K_SW_RX_STATS_BANK_5_TRIGGER_REDIRECTS 10
+#define FM10K_SW_RX_STATS_BANK_5_FLOOD_CONTROL_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_5_GLORT_FORWARDED 12
+#define FM10K_SW_RX_STATS_BANK_5_LOOPBACK_SUPP_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_5_OTHER_DROPS 14
+#define FM10K_SW_RX_PORT_STAT(b_, s_, p_, f_) \
+ (FM10K_SW_RX_STATS_BANK(FM10K_SW_RX_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_RX_STATS_##b_##_##s_) + \
+ ((f_) ? 0 : FM10K_SW_REG_OFF(2)))
+#define FM10K_SW_RX_STATS_CFG(p_) FM10K_SW_RX_STATS_REG(0x10000 + (p_))
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_lsb 0
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_msb 7
+#define FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS 0x00003f00
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_1 (1 << 8)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_2 (1 << 9)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_3 (1 << 10)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_4 (1 << 11)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_5 (1 << 12)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_6 (1 << 13)
+#define FM10K_SW_RX_STATS_CFG_SWITCH_PRI (1 << 14)
+
+
+#define FM10K_SW_HANDLER_TAIL_BASE FM10K_SW_REG_OFF(0xE30000)
+#define FM10K_SW_HANDLER_TAIL_REG(wo_) \
+ (FM10K_SW_HANDLER_TAIL_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_SAF_MATRIX(l_) FM10K_SW_HANDLER_TAIL_REG(0x2 * (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF(l_) (1ULL << (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS FM10K_SW_MASK64(47, 0)
+#define FM10K_SW_SAF_MATRIX_IGNORE_ERROR (1ULL << 50)
+#define FM10K_SW_FRAME_TIME_OUT FM10K_SW_HANDLER_TAIL_REG(0x80)
+#define FM10K_SW_SAF_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x81)
+#define FM10K_SW_EGRESS_PAUSE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x82)
+#define FM10K_SW_RX_STATS_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x84)
+#define FM10K_SW_POLICER_USAGE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x88)
+#define FM10K_SW_TCN_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x8C)
+#define FM10K_SW_FH_TAIL_IP FM10K_SW_HANDLER_TAIL_REG(0x8D)
+#define FM10K_SW_FH_TAIL_IM FM10K_SW_HANDLER_TAIL_REG(0x8E)
+#define FM10K_SW_TAIL_PERMIT_MGMT FM10K_SW_HANDLER_TAIL_REG(0x8F)
+#define FM10K_SW_TAIL_FORCE_IDLE FM10K_SW_HANDLER_TAIL_REG(0x90)
+
+
+#define FM10K_SW_CM_USAGE_BASE FM10K_SW_REG_OFF(0xE60000)
+#define FM10K_SW_CM_USAGE_REG(wo_) \
+ (FM10K_SW_CM_USAGE_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_CM_SWEEPER_SWITCH_PRI_TO_TC FM10K_SW_CM_USAGE_REG(0x0)
+#define FM10K_SW_CM_SWEEPER_TC_TO_SMP FM10K_SW_CM_USAGE_REG(0x2)
+#define FM10K_SW_CM_TX_TC_PRIVATE_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x200 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_TX_TC_HOG_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x400 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_PAUSE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x600 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_PRIVATE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x680 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_HOG_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x700 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_PAUSE_RESEND_INTERVAL(l_) \
+ FM10K_SW_CM_USAGE_REG(0x780 + (l_))
+#define FM10K_SW_CM_PAUSE_BASE_FREQ \
+ FM10K_SW_CM_USAGE_REG(0x7C0)
+#define FM10K_SW_CM_PAUSE_CFG(l_) \
+ FM10K_SW_CM_USAGE_REG(0x800 + (l_))
+#define FM10K_SW_CM_SHARED_WM(p_) \
+ FM10K_SW_CM_USAGE_REG(0x840 + (p_))
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_WM(s_) \
+ FM10K_SW_CM_USAGE_REG(0x850 + (s_))
+#define FM10K_SW_CM_GLOBAL_WM FM10K_SW_CM_USAGE_REG(0x852)
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_lsb 0
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_msb 14
+#define FM10K_SW_CM_GLOBAL_CFG FM10K_SW_CM_USAGE_REG(0x853)
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_lsb 0
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_msb 7
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_ON (1 << 8)
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_OFF (1 << 9)
+#define FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN (1 << 10)
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN (1 << 11)
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN (1 << 12)
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_lsb 13
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_msb 18
+#define FM10K_SW_CM_TC_PC_MAP(l_) FM10K_SW_CM_USAGE_REG(0x880 + (l_))
+#define FM10K_SW_CM_PC_SMP_MAP(l_) FM10K_SW_CM_USAGE_REG(0x8C0 + (l_))
+#define FM10K_SW_CM_SOFTDROP_WM(p_) FM10K_SW_CM_USAGE_REG(0x900 + (p_))
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_CFG(s_) \
+ FM10K_SW_CM_USAGE_REG(0x910 + 0x2 * (s_))
+#define FM10K_SW_TX_RATE_LIM_CFG(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xA00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_TX_RATE_LIM_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xC00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_BSG_MAP(l_) \
+ FM10K_SW_CM_USAGE_REG(0xE00 + (l_))
+#define FM10K_SW_CM_TX_TC_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x1000 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_USAGE(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x1200 + 0x2 * (l_) + (s_))
+#define FM10K_SW_MCAST_EPOCH_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1280 + (s_))
+#define FM10K_SW_CM_SHARED_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1282 + (s_))
+#define FM10K_SW_CM_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1284 + (s_))
+#define FM10K_SW_CM_GLOBAL_USAGE FM10K_SW_CM_USAGE_REG(0x1286)
+#define FM10K_SW_CM_PAUSE_GEN_STATE(l_) FM10K_SW_CM_USAGE_REG(0x12C0 + (l_))
+#define FM10K_SW_CM_PAUSE_RCV_TIMER FM10K_SW_CM_USAGE_REG(0x1300)
+#define FM10K_SW_CM_PAUSE_RCV_PORT_TIMER(l_) \
+ FM10K_SW_CM_USAGE_REG(0x1340 + (l_))
+
+/* FM10K_SW_CM_EGRESS_PAUSE_COUNT is also
+ * known as FM10K_SW_CM_PAUSE_RCV_STATE
+ */
+#define FM10K_SW_CM_EGRESS_PAUSE_COUNT(l_, dw_) \
+ FM10K_SW_CM_USAGE_REG(0x1400 + 0x4 * (l_) + 0x2 * (dw_))
+
+#define FM10K_SW_MOD_BASE FM10K_SW_REG_OFF(0xE80000)
+#define FM10K_SW_MOD_REG(wo_) \
+ (FM10K_SW_MOD_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_MOD_MAX_MGMT_WAIT_CYCLE FM10K_SW_MOD_REG(0xF8)
+#define FM10K_SW_MOD_IP FM10K_SW_MOD_REG(0x106)
+#define FM10K_SW_MOD_IM FM10K_SW_MOD_REG(0x108)
+#define FM10K_SW_MOD_SRAM_BIST_OUT FM10K_SW_MOD_REG(0x10A)
+#define FM10K_SW_MOD_SRAM_ERROR_WRITE FM10K_SW_MOD_REG(0x10C)
+#define FM10K_SW_MOD_PAUSE_SMAC FM10K_SW_MOD_REG(0x10E)
+#define FM10K_SW_MOD_ROUTER_SMAC(n_) FM10K_SW_MOD_REG(0x120 + 0x2 * (n_))
+#define FM10K_SW_MOD_MCAST_VLAN_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x10000 + 0x2 * (n_))
+#define FM10K_SW_MOD_VLAN_TAG_VID1_MAP(v_) \
+ FM10K_SW_MOD_REG(0x20000 + 0x2 * (v_))
+#define FM10K_SW_MOD_VID2_MAP(v_) \
+ FM10K_SW_MOD_REG(0x22000 + 0x2 * (v_))
+#define FM10K_SW_MOD_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x24000 + 0x2 * (n_))
+#define FM10K_SW_MOD_PER_PORT_CFG_1(p_) \
+ FM10K_SW_MOD_REG(0x24080 + 0x2 * (p_))
+#define FM10K_SW_MOD_PER_PORT_CFG_2(p_) \
+ FM10K_SW_MOD_REG(0x24100 + 0x2 * (p_))
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_lsb64 0
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_msb64 5
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP1_UPDATE (1ULL << 6)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP2_UPDATE (1ULL << 7)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI1_UPDATE (1ULL << 8)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI2_UPDATE (1ULL << 9)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_lsb64 10
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_msb64 11
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_lsb64 12
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_msb64 13
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DMAC_ROUTING (1ULL << 14)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_SMAC_ROUTING (1ULL << 15)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_TTL_DECREMENT (1ULL << 16)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DSCP_MODIFICATION (1ULL << 17)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_FTAG (1ULL << 18)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VID2_FIRST (1ULL << 19)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_lsb64 20
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_msb64 22
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_lsb64 23
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_msb64 30
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_TYPE (1ULL << 31)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_lsb64 32
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_msb64 47
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIN_FRAME_SIZE (1ULL << 48)
+#define FM10K_SW_MOD_VPRI1_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24180 + 0x2 * (p_))
+#define FM10K_SW_MOD_VPRI2_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24200 + 0x2 * (p_))
+#define FM10K_SW_MOD_VLAN_ETYPE(n_) \
+ FM10K_SW_MOD_REG(0x24280 + 0x2 * (n_))
+#define FM10K_SW_MOD_STATS_CFG(p_) \
+ FM10K_SW_MOD_REG(0x24300 + 0x2 * (p_))
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 (1 << 0)
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8 (1 << 1)
+#define FM10K_SW_MOD_STATS_BANK_FRAME(b_, i_) \
+ FM10K_SW_MOD_REG(0x25000 + 0x800 * (b_) + 0x2 * (i_))
+#define FM10K_SW_MOD_STATS_BANK_BYTE(b_, i_) \
+ FM10K_SW_MOD_REG(0x26000 + 0x800 * (b_) + 0x2 * (i_))
+#define FM10K_SW_MOD_STATS_BANK_7_INDEX 0
+#define FM10K_SW_MOD_STATS_BANK_8_INDEX 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_UCAST 0
+#define FM10K_SW_MOD_STATS_BANK_7_L2_MCAST 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_BCAST 2
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR 3
+#define FM10K_SW_MOD_STATS_BANK_7_TIMEOUT_DROP 4
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR_DROP 5
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ECC_DROP 6
+#define FM10K_SW_MOD_STATS_BANK_7_LOOPBACK_DROP 7
+#define FM10K_SW_MOD_STATS_BANK_7_TTL1_DROP 8
+#define FM10K_SW_MOD_STATS_BANK_7_PAUSE 9
+#define FM10K_SW_MOD_STATS_BANK_7_CB_PAUSE 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_LT_64 0
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_EQ_64 1
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_65_127 2
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_128_255 3
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_256_511 4
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_512_1023 5
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1024_1522 6
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1523_2047 7
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_2048_4095 8
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_4096_8191 9
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_8192_10239 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_GE_10240 11
+#define FM10K_SW_TX_PORT_STAT(b_, s_, p_, f_, p1_) \
+ ((f_) ? \
+ FM10K_SW_MOD_STATS_BANK_FRAME(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_MOD_STATS_##b_##_##s_) : \
+ FM10K_SW_MOD_STATS_BANK_BYTE(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p1_) + FM10K_SW_MOD_STATS_##b_##_##s_))
+
+#define FM10K_SW_SCHED_BASE FM10K_SW_REG_OFF(0xF00000)
+#define FM10K_SW_SCHED_REG(wo_) \
+ (FM10K_SW_SCHED_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * Schedule fields for both FM10K_SW_SCHED_RX_SCHEDULE and
+ * FM10K_SW_SCHED_TX_SCHEDULE
+ */
+#define FM10K_SW_SCHED_SCHEDULE_ENTRY(p_, l_, q_) \
+ (FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_PHYS_PORT, (p_)) | \
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_LOG_PORT, (l_)) | \
+ ((q_) ? FM10K_SW_SCHED_SCHEDULE_QUAD : 0))
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_lsb 0
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_msb 7
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_lsb 8
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_msb 13
+#define FM10K_SW_SCHED_SCHEDULE_QUAD (1 << 14)
+#define FM10K_SW_SCHED_SCHEDULE_COLOR (1 << 15)
+#define FM10K_SW_SCHED_SCHEDULE_IDLE (1 << 16)
+
+#define FM10K_SW_SCHED_RX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20000 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_TX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20400 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_SCHEDULE_CTRL FM10K_SW_SCHED_REG(0x20800)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE (1 << 0)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_PAGE (1 << 1)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_lsb 2
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_msb 10
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE (1 << 11)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_PAGE (1 << 12)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_lsb 13
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_msb 21
+#define FM10K_SW_SCHED_CONFIG_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x20801)
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x20840 + (l_))
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_lsb 0
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_msb 15
+#define FM10K_SW_SCHED_SSCHED_SRAM_CTRL FM10K_SW_SCHED_REG(0x30050)
+#define FM10K_SW_SCHED_ESCHED_CFG_1(l_) FM10K_SW_SCHED_REG(0x30080 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_2(l_) FM10K_SW_SCHED_REG(0x300C0 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_3(l_) FM10K_SW_SCHED_REG(0x30100 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_ESCHED FM10K_SW_SCHED_REG(0x30240)
+#define FM10K_SW_SCHED_ESCHED_SRAM_CTRL FM10K_SW_SCHED_REG(0x30242)
+#define FM10K_SW_SCHED_FREELIST_INIT FM10K_SW_SCHED_REG(0x30244)
+#define FM10K_SW_SCHED_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_FREELIST_SRAM_CTRL FM10K_SW_SCHED_REG(0x30246)
+#define FM10K_SW_SCHED_MONITOR_DRR_Q_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x30400 + (q_))
+#define FM10K_SW_SCHED_MONITOR_DRR_CFG_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x30600 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_MONITOR FM10K_SW_SCHED_REG(0x30A40)
+#define FM10K_SW_SCHED_MONITOR_SRAM_CTRL FM10K_SW_SCHED_REG(0x30A41)
+#define FM10K_SW_SCHED_MCAST_LEN_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x34000 + (n_))
+#define FM10K_SW_SCHED_MCAST_DEST_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x38000 + 0x2 * (n_))
+#define FM10K_SW_SCHED_MCAST_LIMITED_SKEW_MULTICAST \
+ FM10K_SW_SCHED_REG(0x3A000)
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(n_) \
+ FM10K_SW_SCHED_REG(0x60400 + 0x2 * (n_))
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES 8
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_lsb 0
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_msb 9
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_lsb 10
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_msb 19
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_lsb 20
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_msb 24
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_lsb 25
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_msb 29
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_lsb 30
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_msb 39
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT FM10K_SW_SCHED_REG(0x60410)
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES 1024
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_msb 9
+#define FM10K_SW_SCHED_MGMT_TIMER_RXQ FM10K_SW_SCHED_REG(0x60412)
+#define FM10K_SW_SCHED_MGMT_TIMER_LG FM10K_SW_SCHED_REG(0x60413)
+#define FM10K_SW_SCHED_RXQ_SRAM_CTRL FM10K_SW_SCHED_REG(0x60414)
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60600 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60800 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60A00 + (q_))
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES 384
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_lsb 0
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_msb 15
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT FM10K_SW_SCHED_REG(0x62000)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_TXQ_SRAM_CTRL FM10K_SW_SCHED_REG(0x62002)
+#define FM10K_SW_SCHED_FIFO_SRAM_CTRL FM10K_SW_SCHED_REG(0x62004)
+#define FM10K_SW_SCHED_IP FM10K_SW_SCHED_REG(0x62006)
+#define FM10K_SW_SCHED_IM FM10K_SW_SCHED_REG(0x62007)
+
+#endif /* _FM10K_REGS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.c b/drivers/net/fm10k/switch/fm10k_sbus.c
new file mode 100644
index 0000000..d7d656e
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.c
@@ -0,0 +1,292 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+#define FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX 4
+#define FM10K_SW_SBUS_COMMAND_TIMEOUT_US 500000
+#define FM10K_SW_SBUS_RING_DIVIDER 0x01
+
+static int fm10k_sbus_init(struct fm10k_sbus *);
+static int fm10k_sbus_exec(struct fm10k_sbus *, struct fm10k_sbus_req *);
+static int fm10k_sbus_reset_all(struct fm10k_sbus *);
+static int fm10k_sbus_sbm_reset(struct fm10k_sbus *);
+
+struct fm10k_sbus *
+fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name,
+ unsigned int cfg_reg)
+{
+ struct fm10k_sbus *sb;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: attaching", name);
+
+ sb = (struct fm10k_sbus *)rte_zmalloc
+ ("fm10k_sbus", sizeof(struct fm10k_sbus), 0);
+ if (sb == NULL) {
+ FM10K_SW_INFO("sbus %s: failed to allocate context", name);
+ goto fail;
+ }
+
+ sb->sw = sw;
+ sb->name = name;
+ pthread_mutex_init(&sb->lock, NULL);
+
+ sb->cfg_reg = cfg_reg;
+ sb->cmd_reg = cfg_reg + FM10K_SW_REG_OFF(1);
+ sb->req_reg = cfg_reg + FM10K_SW_REG_OFF(2);
+ sb->resp_reg = cfg_reg + FM10K_SW_REG_OFF(3);
+
+ error = fm10k_sbus_init(sb);
+ if (error)
+ goto fail;
+
+ FM10K_SW_TRACE("sbus %s: attach successful", name);
+ return sb;
+fail:
+ if (sb)
+ fm10k_sbus_detach(sb);
+ return NULL;
+}
+
+void
+fm10k_sbus_detach(struct fm10k_sbus *sb)
+{
+ FM10K_SW_TRACE("sbus %s: detaching", sb->name);
+
+ rte_free(sb);
+}
+
+static int
+fm10k_sbus_init(struct fm10k_sbus *sb)
+{
+ uint32_t data;
+ int error;
+
+ error = fm10k_sbus_sbm_reset(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_reset_all(sb);
+ if (error)
+ goto done;
+
+ /* set clock to REFCLK/2 */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ FM10K_SW_SBUS_RING_DIVIDER);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to set ring divider", sb->name);
+ goto done;
+ }
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ &data);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to read back ring divider",
+ sb->name);
+ goto done;
+ }
+ if (data != FM10K_SW_SBUS_RING_DIVIDER) {
+ FM10K_SW_ERR("sbus %s: ring divider "
+ "verify failed (expected %u, got %u)",
+ sb->name, FM10K_SW_SBUS_RING_DIVIDER, data);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return error;
+}
+
+static int
+fm10k_sbus_exec(struct fm10k_sbus *sb, struct fm10k_sbus_req *req)
+{
+ struct fm10k_switch *sw = sb->sw;
+ unsigned int total_usecs;
+ unsigned int drop_lock;
+ unsigned int delay_time;
+ int error = 0;
+ uint32_t data;
+ uint8_t expected_result;
+
+ FM10K_SW_SBUS_LOCK(sb);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (data & FM10K_SW_SBUS_COMMAND_BUSY) {
+ FM10K_SW_INFO("sbus %s: bus busy (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+
+ switch (req->op) {
+ case FM10K_SW_SBUS_OP_RESET:
+ FM10K_SW_TRACE("sbus %s: RESET dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_RESET;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_WRITE:
+ FM10K_SW_TRACE("sbus %s: WRITE dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_WRITE;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_READ:
+ FM10K_SW_TRACE("sbus %s: READ dev=0x%02x reg=0x%02x",
+ sb->name, req->dev, req->reg);
+ expected_result = FM10K_SW_SBUS_RESULT_READ;
+ req->data = 0;
+ break;
+ default:
+ FM10K_SW_INFO("sbus %s: invalid opcode 0x%02x",
+ sb->name, req->op);
+ error = -1;
+ goto done;
+ }
+
+ /* Clear the execute bit */
+ fm10k_write_switch_reg(sw, sb->cmd_reg, 0);
+
+ data =
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_OP, req->op) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_ADDRESS, req->dev) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_REGISTER, req->reg) |
+ FM10K_SW_SBUS_COMMAND_EXECUTE;
+ fm10k_write_switch_reg(sw, sb->cmd_reg, data);
+ fm10k_write_flush(sw);
+
+ total_usecs = 0;
+ delay_time = 1;
+ drop_lock = 0;
+ do {
+ if (drop_lock)
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_udelay(delay_time);
+ if (drop_lock)
+ FM10K_SW_SWITCH_LOCK(sw);
+ total_usecs += delay_time;
+ if (total_usecs >= FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX) {
+ drop_lock = 1;
+ delay_time <<= 1;
+ }
+
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (!(data & FM10K_SW_SBUS_COMMAND_BUSY)) {
+ if (FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE)
+ != expected_result) {
+ FM10K_SW_INFO("sbus %s: expected "
+ "result code %u, got %u", sb->name,
+ expected_result,
+ FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE));
+ error = -1;
+ goto done;
+ }
+ if (req->op == FM10K_SW_SBUS_OP_READ) {
+ req->data =
+ fm10k_read_switch_reg(sw, sb->resp_reg);
+ FM10K_SW_TRACE("sbus %s: READ data=0x%02x",
+ sb->name, req->data);
+ }
+ goto done;
+ }
+
+ } while (total_usecs < FM10K_SW_SBUS_COMMAND_TIMEOUT_US);
+
+ error = -1;
+ FM10K_SW_INFO("sbus %s: command timed out after %u us "
+ "(op=0x%02x dev=0x%02x reg=0x%02x data=0x%08x)", sb->name,
+ total_usecs, req->op, req->dev, req->reg, req->data);
+
+done:
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ FM10K_SW_SBUS_UNLOCK(sb);
+ return error;
+}
+
+static int
+fm10k_sbus_reset_all(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_BROADCAST;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: broadcast reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+static int
+fm10k_sbus_sbm_reset(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_SPICO;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: SBM reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+int
+fm10k_sbus_read(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t *data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_READ;
+ req.dev = dev;
+ req.reg = reg;
+
+ error = fm10k_sbus_exec(sb, &req);
+ *data = error ? 0 : req.data;
+
+ return error;
+}
+
+int
+fm10k_sbus_write(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_WRITE;
+ req.dev = dev;
+ req.reg = reg;
+ req.data = data;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ return error;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.h b/drivers/net/fm10k/switch/fm10k_sbus.h
new file mode 100644
index 0000000..e67967f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SBUS_H_
+#define _FM10K_SW_SBUS_H_
+
+#include <pthread.h>
+
+
+struct fm10k_sbus {
+ struct fm10k_switch *sw;
+ const char *name;
+ pthread_mutex_t lock;
+ unsigned int cfg_reg;
+ unsigned int cmd_reg;
+ unsigned int req_reg;
+ unsigned int resp_reg;
+};
+
+#define FM10K_SW_SBUS_LOCK(sb_) pthread_mutex_lock(&((sb_)->lock))
+#define FM10K_SW_SBUS_UNLOCK(sb_) pthread_mutex_unlock(&((sb_)->lock))
+
+struct fm10k_sbus_req {
+ uint8_t op;
+ uint8_t dev;
+ uint8_t reg;
+ uint32_t data;
+};
+
+
+struct fm10k_sbus *fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name, unsigned int cfg_reg);
+void fm10k_sbus_detach(struct fm10k_sbus *sb);
+int fm10k_sbus_read(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t *data);
+int fm10k_sbus_write(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t data);
+
+#endif /* _FM10K_SW_SBUS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
new file mode 100644
index 0000000..d69b51c
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -0,0 +1,336 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SWITCH_H_
+#define _FM10K_SW_SWITCH_H_
+
+#include <rte_spinlock.h>
+#include <pthread.h>
+#include <sys/time.h>
+#include <semaphore.h>
+
+#include "../fm10k.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+
+
+/*
+ * EPL
+ */
+#define FM10K_SW_EXT_PORTS_MAX 4
+
+#define FM10K_SW_EPLS_MAX 9
+#define FM10K_SW_EPLS_SUPPORTED 2
+#define FM10K_SW_EPL_LANES 4
+#define FM10K_SW_EPLS_PEP_MAX 2
+
+/*
+ * PEP
+ */
+#define FM10K_SW_PEP_GROUP_MAX 10
+#define FM10K_SW_LOGICAL_PORTS_MAX 48
+
+#define FM10K_SW_PEPS_MAX 9
+#define FM10K_SW_PEPS_SUPPORTED 4
+#define FM10K_SW_PEP_PORTS_MAX 4
+
+/*
+ * GLORT
+ */
+#define FM10K_SW_GROUP_MAX 10
+#define FM10K_SW_VFS_MAX 64
+#define FM10K_SW_PFS_GLORT_START 0x1000
+#define FM10K_SW_VF_GLORT_START 0x2000
+#define FM10K_SW_MULTI_GLORT_START 0x3000
+#define FM10K_SW_DPORT_MASK(lport) (1ULL << (lport))
+
+
+/*
+ * CONFIG
+ */
+#define FM10K_SW_CONFIG_MAX 1000
+#define FM10K_SW_FFU_RULE_MAX 256
+#define FM10K_SW_FFU_RULE_ITEM_MAX 10
+
+/*
+ * FFU COUNT
+ */
+#define FM10K_SW_FFU_CNT_BANK 3
+#define FM10K_SW_POLICER_LAST 19
+#define FM10K_SW_FFU_CNT_START (FM10K_SW_POLICER_LAST + 1)
+#define FM10K_SW_FFU_CNT_MAX 100
+
+#define FM10K_SW_PEP_QUEUES_MAX 256
+#define FM10K_SW_VF_QUEUES_MAX \
+ (FM10K_SW_PEP_QUEUES_MAX - 1) /* Need 1 queue for PF */
+
+#define FM10K_SW_FTAG_SIZE 8
+#define FM10K_SW_PACKET_SIZE_MIN 17
+#define FM10K_SW_PACKET_SIZE_MAX (15 * 1024)
+#define FM10K_SW_MTU_MAX \
+ (FM10K_SW_PACKET_SIZE_MAX - ETHER_HDR_LEN - ETHER_VLAN_ENCAP_LEN)
+#define FM10K_SW_SEG_SIZE_MAX (16 * 1024)
+#define FM10K_SW_TSO_SIZE_MAX (256 * 1024 - 1)
+
+#define FM10K_SW_MEM_POOL_SEG_SIZE 192
+#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
+#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+
+
+
+#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
+#define FM10K_SW_CARD(vname_, dname_) \
+ FM10K_SW_CARD_ID(FM10K_SW_VENDOR_ID_##vname_, \
+ FM10K_SW_DEV_ID_##dname_)
+
+/*
+ * All IDs that may appear in the vendor ID or subsystem vendor ID.
+ */
+#define FM10K_SW_VENDOR_ID_INTEL 0x8086
+#define FM10K_SW_VENDOR_ID_SILICOM 0x1374
+#define FM10K_SW_VENDOR_ID_SILICOM_RB 0x1B2E
+
+
+/*
+ * All IDs that may appear in the device ID or subsystem device ID.
+ */
+#define FM10K_SW_DEV_ID_FM10K 0x15a4
+
+/* Silicom cards */
+#define FM10K_SW_DEV_ID_PE310G4DBIR_T 0x01B0
+#define FM10K_SW_DEV_ID_PE310G4DBIR_SRD 0x01B1
+#define FM10K_SW_DEV_ID_PE310G4DBIR_LRD 0x01B2
+#define FM10K_SW_DEV_ID_PE310G4DBIR_ER 0x01B3
+#define FM10K_SW_DEV_ID_PE310G4DBIR_DA 0x01B4
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS41 0x01B8
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS43 0x01B9
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QL4 0x01BA
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4 0x01C0
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4_REV_2 0x01C4
+#define FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4 0x01C1
+#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
+#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
+/*
+ * SWITCH
+ */
+
+struct fm10k_device_info {
+ uint16_t subvendor;
+ uint16_t subdevice;
+ const char *desc;
+ uint8_t num_ext_ports;
+ uint8_t ext_port_speed;
+ uint8_t num_epls;
+ uint8_t num_peps;
+};
+
+
+struct fm10k_i2c;
+struct fm10k_sbus;
+struct fm10k_hw;
+struct fm10k_ext_ports;
+struct fm10k_dpdk_cfg;
+
+struct fm10k_sw_port_map {
+ uint32_t glort;
+ uint16_t logical_port; /* logical port number */
+ uint16_t physical_port; /* */
+};
+
+struct fm10k_switch {
+ uint32_t *hw_addr;
+ uint32_t *sw_addr;
+ struct fm10k_hw *master_hw;
+ struct fm10k_device_info *info;
+ pthread_mutex_t lock;
+ struct fm10k_i2c *i2c;
+ struct fm10k_sbus *epl_sbus;
+ struct fm10k_ext_ports *ext_ports;
+ sem_t intr_tq;
+ pthread_t intr_task;
+ pthread_t led_task;
+ pthread_t stats_task;
+ uint32_t detaching;
+ uint32_t glort_cam_ram_idx;
+ uint32_t glort_dest_table_idx;
+ uint32_t mcast_dest_table_idx;
+ uint32_t mcast_len_table_idx;
+ uint32_t mcast_vlan_table_idx;
+ uint32_t epl_serdes_code_version_build_id;
+ uint16_t pep_mask; /* mask of non-master peps */
+ uint8_t pepno;
+ uint8_t serdes_loopback;
+ uint8_t epla_no;
+ uint8_t eplb_no;
+ uint8_t mac_addr[6];
+ struct fm10k_dpdk_cfg *dpdk_cfg;
+ struct fm10k_sw_port_map *pep_map;
+ struct fm10k_sw_port_map *epl_map;
+ int inited;
+};
+
+#define FM10K_SW_SWITCH_LOCK(sw_) pthread_mutex_lock(&((sw_)->lock))
+#define FM10K_SW_SWITCH_UNLOCK(sw_) pthread_mutex_unlock(&((sw_)->lock))
+
+#define FM10K_SW_HOWMANY(x, y, y1) (((x) + (y) - 1) / (y1))
+
+
+static inline uint64_t
+fm10k_uptime_us(void)
+{
+ struct timeval tv;
+
+ gettimeofday(&tv, NULL);
+ return ((uint64_t)tv.tv_sec * 1000000 + tv.tv_usec);
+}
+
+
+static inline uint32_t
+fm10k_read_switch_reg(struct fm10k_switch *sw, uint32_t reg)
+{
+ return ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+}
+
+static inline uint64_t
+fm10k_read_switch_reg64(struct fm10k_switch *sw, uint32_t reg)
+{
+ uint64_t temp, result;
+
+ result = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+ temp = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1];
+ result |= temp << 32;
+
+ return result;
+}
+
+static inline void
+fm10k_read_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *results, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ results[i] =
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i];
+}
+
+
+static inline void
+fm10k_write_switch_reg(struct fm10k_switch *sw, uint32_t reg, uint32_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val;
+}
+
+static inline void
+fm10k_write_switch_reg64(struct fm10k_switch *sw, uint32_t reg, uint64_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] = val >> 32;
+}
+
+static inline void
+fm10k_write_switch_reg128(struct fm10k_switch *sw,
+ uint32_t reg, uint64_t val_hi, uint64_t val_lo)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] =
+ val_lo & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] =
+ val_lo >> 32;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 2] =
+ val_hi & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 3] =
+ val_hi >> 32;
+}
+
+static inline void
+fm10k_write_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *data, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i] =
+ data[i];
+}
+
+static inline uint32_t
+fm10k_write_flush(struct fm10k_switch *sw)
+{
+ return ((volatile uint32_t *)sw->master_hw->hw_addr)[FM10K_SW_CTRL];
+}
+
+static inline void
+fm10k_gpio_output_set(struct fm10k_switch *sw, int gpio_pin, int value)
+{
+ uint32_t data;
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_CFG);
+ data |= 1 << gpio_pin;
+ data &= ~(1 << (gpio_pin + 16));
+
+ /* set gpio output */
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_CFG, data);
+
+ /*
+ * Wait 1 msec (PCA spec specifies reset pulse width = 4 ns and
+ * and reset time = 100 ns)
+ */
+ usec_delay(1000);
+ /* set reset */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_DATA);
+ if (value == 0)
+ data &= ~(1 << gpio_pin);
+ else
+ data |= 1 << gpio_pin;
+
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
+}
+
+#define fm10k_udelay usec_delay
+typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
+
+unsigned int fm10k_switch_eplidx_to_eplno
+ (struct fm10k_switch *sw, unsigned int eplidx);
+void fm10k_switch_intr(struct fm10k_hw *hw);
+
+struct fm10k_switch *fm10k_switch_get(void);
+struct fm10k_device_info *fm10k_get_device_info(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func);
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue);
+int fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw,
+ struct fm10k_hw *hw_list[]);
+int fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+void *fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw);
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw);
+void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+
+uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
+uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
+uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
+uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
+uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
+uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
+uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new);
+
+int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
+
+void fm10k_switch_flowset_switchto(const char *name);
+void fm10k_switch_show_port(void);
+void fm10k_switch_show_ffu(void);
+void fm10k_switch_show_bank(void);
+
+void fm10k_flow_list_init(void *flow_list);
+const struct rte_flow_ops *fm10k_flow_ops_get(void);
+
+#endif
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 2/7] net/fm10k: add some modules of port
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 3/7] net/fm10k: add config ffu statistics support Xiaojun Liu
` (5 subsequent siblings)
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
To support switch management, add the following new files:
Add fm10k/switch/fm10k_spico_code.h.
Add fm10k/switch/fm10k_spico_code.c(define spico code for serdes)
Add fm10k/switch/fm10k_serdes.h
Add fm10k/switch/fm10k_serdes.c(support serdes management)
Add fm10k/switch/fm10k_sm.h
Add fm10k/switch/fm10k_sm.c(state machine for external ports)
Add fm10k/switch/fm10k_ext_port.h
Add fm10k/switch/fm10k_ext_port.c(support external ports)
and modify fm10k/Makefile(add fm10k_sm.c,
fm10k_spico_code.c, fm10k_serdes.c and fm10k_ext_port.c).
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 4 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
drivers/net/fm10k/switch/fm10k_sm.h | 78 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
9 files changed, 6146 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index aca6bfb..4ec2a80 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -84,8 +84,12 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.c b/drivers/net/fm10k/switch/fm10k_ext_port.c
new file mode 100644
index 0000000..e838fff
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.c
@@ -0,0 +1,841 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_serdes.h"
+#include "fm10k_sm.h"
+#include "fm10k_switch.h"
+
+
+#define LANE_EVENT(e_) FM10K_SW_EXT_PORT_LANE_EVENT_##e_
+#define LANE_TIMER(t_) FM10K_SW_EXT_PORT_LANE_TIMER_##t_
+#define LANE_TIMEOUT(t_) FM10K_SW_EXT_PORT_LANE_##t_##_TIMEOUT_MS
+#define LANE_POLL(t_) FM10K_SW_EXT_PORT_LANE_##t_##_POLL_MS
+#define LANE_STATE(s_) FM10K_SW_EXT_PORT_LANE_STATE_##s_
+#define LANE_FLAG(f_) FM10K_SW_EXT_PORT_LANE_FLAG_##f_
+#define LANE_FLAGS(f_) FM10K_SW_EXT_PORT_LANE_FLAGS_##f_
+#define LANE_FLAG_SET(l_, f_) ((l_)->flags & LANE_FLAG(f_))
+#define LANE_FLAGS_SET(l_, f_, f1_) \
+ (((l_)->flags & LANE_FLAGS(f_)) == LANE_FLAGS(f1_))
+
+#define PORT_EVENT(e_) FM10K_SW_EXT_PORT_EVENT_##e_
+#define PORT_STATE(s_) FM10K_SW_EXT_PORT_STATE_##s_
+
+static void fm10k_ext_ports_epl_intr
+ (struct fm10k_ext_ports *ports, unsigned int eplno);
+static void fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip);
+static void fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_bring_down(struct fm10k_ext_port *port);
+static void fm10k_ext_port_lane_process_event
+ (struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_lane_process_timer
+ (struct fm10k_sm *sm, uint16_t timer);
+static void fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_phy_enable(struct fm10k_ext_port *port);
+static void fm10k_ext_port_phy_disable(struct fm10k_ext_port *port);
+
+struct fm10k_ext_ports *
+fm10k_ext_ports_attach(struct fm10k_switch *sw)
+{
+// struct fm10k_pep *pep = sw->pep;
+ struct fm10k_ext_ports *ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int is_quad, lane_speed;
+ unsigned int i, j;
+ unsigned int eplidx, laneno;
+
+ FM10K_SW_TRACE("attaching external ports\n");
+
+ ports = (struct fm10k_ext_ports *)rte_zmalloc
+ ("fm10k_ext_ports", sizeof(struct fm10k_ext_ports), 0);
+ if (ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports\n");
+ goto fail;
+ }
+
+ ports->ports = (struct fm10k_ext_port *)rte_zmalloc
+ ("fm10k_ext_port_list",
+ sizeof(struct fm10k_ext_port) * cfg->num_ext_ports, 0);
+ if (ports->ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports list\n");
+ goto fail;
+ }
+ ports->sw = sw;
+ ports->num_ports = cfg->num_ext_ports;
+ ports->ports_per_epl = cfg->num_ext_ports / cfg->num_epls;
+ ports->epl_mask = 0;
+
+ switch (cfg->ext_port_speed) {
+ default:
+ case 10:
+ is_quad = 0;
+ lane_speed = 10;
+ break;
+ case 25:
+ is_quad = 0;
+ lane_speed = 25;
+ break;
+ case 40:
+ is_quad = 1;
+ lane_speed = 10;
+ break;
+ case 100:
+ is_quad = 1;
+ lane_speed = 25;
+ break;
+ }
+
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ port->ports = ports;
+ fm10k_ext_port_eplidx_lane(ports, i, &eplidx, &laneno);
+ port->portno = i;
+ port->eplno = fm10k_switch_eplidx_to_eplno(sw, eplidx);
+ snprintf(port->name, sizeof(port->name), "EPL[%u][%u]",
+ port->eplno, port->first_lane);
+ port->first_lane = laneno;
+ port->is_quad = is_quad;
+ port->num_lanes = port->is_quad ? 4 : 1;
+ port->lane_speed = lane_speed;
+ port->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_NUM_EVENTS, 0,
+ fm10k_ext_port_process_event, NULL,
+ (void *)port);
+ if (port->sm == NULL)
+ goto fail;
+ port->sm->portno = port->portno;
+ port->sm->laneno = 0;
+ port->an_im = FM10K_SW_AN_IM_ALL;
+ port->link_im = FM10K_SW_LINK_IM_ALL;
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ lane->port = port;
+ lane->abs_laneno = port->eplno * FM10K_SW_EPL_LANES + j;
+ lane->rel_laneno = j;
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+ lane->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_LANE_NUM_EVENTS,
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS,
+ fm10k_ext_port_lane_process_event,
+ fm10k_ext_port_lane_process_timer,
+ (void *)lane);
+ if (lane->sm == NULL)
+ goto fail;
+ lane->sm->portno = lane->port->portno;
+ lane->sm->laneno = lane->rel_laneno;
+ }
+
+ ports->epl_mask |= (1 << port->eplno);
+ }
+
+ return (ports);
+fail:
+ if (ports)
+ fm10k_ext_ports_detach(ports);
+ return NULL;
+}
+
+void
+fm10k_ext_ports_detach(struct fm10k_ext_ports *ports)
+{
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i, j;
+
+ FM10K_SW_TRACE("detaching external ports\n");
+
+ if (ports->ports) {
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+
+ if (port->sm)
+ fm10k_sm_detach(port->sm);
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ if (lane->sm)
+ fm10k_sm_detach(lane->sm);
+ }
+
+ fm10k_ext_port_phy_disable(port);
+ }
+ rte_free(ports->ports);
+ }
+
+ rte_free(ports);
+}
+
+unsigned int
+fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports, uint64_t gid)
+{
+ unsigned int i;
+ uint16_t epl_mask;
+
+ /*
+ * Process EPL interrupts for all EPLs that are in use and have
+ * their bit set in global interrupt detect.
+ */
+ epl_mask = ports->epl_mask &
+ FM10K_SW_REG_FIELD64(gid, GLOBAL_INTERRUPT_DETECT_EPL);
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++)
+ if (epl_mask & (1 << i))
+ fm10k_ext_ports_epl_intr(ports, i);
+
+ return ports->epl_mask;
+}
+
+static void
+fm10k_ext_ports_epl_intr(struct fm10k_ext_ports *ports, unsigned int eplno)
+{
+ struct fm10k_switch *sw = ports->sw;
+ struct fm10k_ext_port *port;
+ uint64_t gid;
+ unsigned int i, j;
+ uint32_t epl_ip;
+
+ /*
+ * The global interrupt detect architecture makes it somewhat
+ * challenging to ensure events are not lost and spurious interrupts
+ * are not generated.
+ *
+ * The EPLs have an internal interrupt hierarchy whose reporting is
+ * rooted at EPL_IP. Whenever an EPL transitions from having no
+ * unmasked interrupts pending to at least one unmasked interrupt
+ * pending or from having at least one unmasked interrupt pending to
+ * no unmasked interrupts pending, it sends a vector to the
+ * interrupt controller over a serial bus indicating the
+ * at-least-one/none pending state, and the controller updates the
+ * bit for that EPL in the global interrupt detect register
+ * accordingly.
+ *
+ * Because of the update-vector-in-flight potential, there is a race
+ * between re-enabling the EPL PCIe interrupt mask after servicing
+ * the EPL interrupts and the all-clear vector from the EPL reaching
+ * the interrupt controller - it's possible for the software to
+ * clear all the EPL-internal interrupt sources during servicing and
+ * then re-enable the EPL interrupt in the PCIe mask before the
+ * all-clear vector reaches the interrupt controller and clears the
+ * EPL pending bit in the global interrupt detect register. This
+ * would result in a spurious EPL interrupt, potentially for every
+ * necessary EPL interrupt.
+ *
+ * The approach taken here to avoid spurious interrupts is to first
+ * mask off all EPL-internal sources, then poll the EPL bits in the
+ * global interrupt detect register until they clear (indicating the
+ * update vector from the EPL has reached the interrupt controller).
+ * Then software interrupt processing proceeds, and the EPL-internal
+ * masks are restored to their desired state at the end. This
+ * ensures that when the EPL PCIe interrupt mask is re-enabled, the
+ * EPL pending bits in the global interrupt detect register will
+ * only be set if there are new, unprocessed EPL events of interest.
+ *
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* Grab epl_ip before it is cleared by the masks below */
+ epl_ip = fm10k_read_switch_reg(sw, FM10K_SW_EPL_IP(eplno));
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_AN_IM(eplno, port->first_lane),
+ FM10K_SW_AN_IM_ALL);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IM(eplno, port->first_lane),
+ FM10K_SW_LINK_IM_ALL);
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(eplno, j),
+ FM10K_SW_SERDES_IM_ALL);
+ }
+ }
+
+ /*
+ * Wait for the all-clear vector to propagate over the internal
+ * interrupt serial bus to the global interrupt detect.
+ */
+ do {
+ gid = fm10k_read_switch_reg64
+ (sw, FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ } while (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(eplno));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ /*
+ * Proceed with interrupt processing per-external-port.
+ */
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+ fm10k_ext_port_intr(port, epl_ip);
+ }
+}
+
+static void
+fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ uint32_t serdes_ip, data = 0;
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (epl_ip & FM10K_SW_EPL_IP_SERDES_INTERRUPT(i)) {
+ FM10K_SW_SWITCH_LOCK(sw);
+ serdes_ip = fm10k_read_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i));
+ serdes_ip &= ~lane->im;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i), serdes_ip);
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK)
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_STATUS(port->eplno,
+ i));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK) {
+ fm10k_sm_send_event(lane->sm,
+ (data &
+ FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK) ?
+ LANE_EVENT(SIGNAL_OK) : LANE_EVENT(SIGNAL_NOK));
+ }
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RX_PLL_LOCK));
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_TX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(TX_PLL_LOCK));
+ } else {
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RESTORE_IM));
+ }
+ }
+}
+
+void
+fm10k_ext_port_up(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_UP));
+}
+
+void
+fm10k_ext_port_down(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_DOWN));
+}
+
+unsigned int
+fm10k_ext_port_isup(struct fm10k_ext_port *port)
+{
+ /* no locked needed as we are just inspecting
+ * the current value of an int
+ */
+ unsigned int port_state = port->sm->state;
+
+ return (port_state == FM10K_SW_EXT_PORT_STATE_UP);
+}
+
+static void
+fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port *port = sm->ctx;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ unsigned int lanes_up;
+
+ switch (sm->state) {
+ case PORT_STATE(DOWN):
+ if (event == PORT_EVENT(BRING_UP)) {
+ fm10k_ext_port_phy_enable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event
+ (lane->sm, LANE_EVENT(BRING_UP));
+ }
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ }
+ break;
+
+ case PORT_STATE(WAITING_FOR_LANES):
+ if (event == PORT_EVENT(LANE_UP)) {
+ lanes_up = 0;
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (lane->sm->state != LANE_STATE(UP))
+ break;
+ lanes_up++;
+ }
+ if (lanes_up == port->num_lanes) {
+ sm->state = PORT_STATE(UP);
+ FM10K_SW_INFO
+ ("Port %d is LINK UP", port->portno);
+ }
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ }
+ break;
+
+ case PORT_STATE(UP):
+ if (event == PORT_EVENT(LANE_DOWN)) {
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ }
+ }
+}
+
+static void
+fm10k_ext_port_bring_down(struct fm10k_ext_port *port)
+{
+ struct fm10k_sm *sm = port->sm;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+
+ sm->state = PORT_STATE(DOWN);
+
+ fm10k_ext_port_phy_disable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event(lane->sm, LANE_EVENT(BRING_DOWN));
+ }
+}
+
+static void
+fm10k_ext_port_lane_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ struct fm10k_ext_port *port = lane->port;
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ int error;
+
+ switch (sm->state) {
+ case LANE_STATE(DOWN):
+ if (event == LANE_EVENT(BRING_UP)) {
+ lane->flags &= ~LANE_FLAGS(PLLS_LOCKED);
+
+ /*
+ * Clear all pending interrupts.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, lane->rel_laneno),
+ FM10K_SW_SERDES_IP_ALL);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->im &= ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+ }
+ break;
+
+ case LANE_STATE(WAIT_PLL_LOCK):
+ if (event == LANE_EVENT(RX_PLL_LOCK) ||
+ event == LANE_EVENT(TX_PLL_LOCK)) {
+ if (event == LANE_EVENT(RX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(RX_LOCKED);
+ if (event == LANE_EVENT(TX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(TX_LOCKED);
+ if (LANE_FLAGS_SET(lane, PLLS_LOCKED, PLLS_LOCKED)) {
+ error = fm10k_epl_serdes_continue_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_SIGNAL_OK);
+ /* unmask RX signal OK interrupt */
+ lane->im |=
+ (FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ lane->im &=
+ ~FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ }
+ }
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_SIGNAL_OK):
+ if (event == LANE_EVENT(SIGNAL_OK)) {
+ sm->state = LANE_STATE(WAIT_DFE_ICAL);
+
+ fm10k_epl_serdes_configure_for_dfe_tuning(lane);
+ fm10k_epl_serdes_start_dfe_ical(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ sm->state = LANE_STATE(WAIT_DFE_PCAL);
+
+ fm10k_epl_serdes_start_dfe_pcal(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+ sm->state = LANE_STATE(UP);
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_UP));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT)) ||
+ (event == LANE_EVENT(SIGNAL_NOK))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(UP):
+ if (event == LANE_EVENT(SIGNAL_NOK))
+ fm10k_ext_port_restart_lane(lane);
+ else if (event == LANE_EVENT(BRING_DOWN))
+ fm10k_ext_port_disable_lane(lane);
+ break;
+ }
+
+ /* Always restore the interrupt mask */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(port->eplno,
+ lane->rel_laneno),
+ lane->im);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_ext_port_lane_process_timer(struct fm10k_sm *sm, uint16_t timer)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ uint32_t status;
+
+ switch (sm->state) {
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_ICAL);
+ fm10k_epl_serdes_dfe_ical_status(lane, &status);
+
+ if (status == FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_ICAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else if (status == FM10K_SW_SERDES_DFE_ICAL_CONVERGED)
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_FAILED));
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_PCAL);
+
+ fm10k_epl_serdes_dfe_pcal_status(lane, &status);
+ if (status == FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_PCAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else {
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ }
+ }
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+ struct fm10k_ext_port *port = lane->port;
+ int error;
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->flags &= ~FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED;
+ lane->im |= FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ lane->im = ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_DOWN));
+}
+
+static void
+fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+
+ fm10k_sm_timer_cancel(sm, LANE_TIMER(DFE_TUNING));
+ fm10k_epl_serdes_disable(lane);
+
+ /*
+ * Disable all interrupts.
+ */
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+
+ sm->state = LANE_STATE(DOWN);
+}
+
+void
+fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane)
+{
+ FM10K_SW_ASSERT(ext_port_idx < ports->num_ports,
+ ("ext_port_idx out of range"));
+
+ *eplidx = ext_port_idx / ports->ports_per_epl;
+ *lane = ext_port_idx % ports->ports_per_epl;
+}
+
+
+static void
+fm10k_ext_port_phy_enable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the
+ * PCA9505 that the QSFP control lines are connected
+ * to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get
+ * at the PCA9505 that the QSFP control lines
+ * are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x10);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x10);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to enable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_phy_disable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x60);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x61);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to disable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.h b/drivers/net/fm10k/switch/fm10k_ext_port.h
new file mode 100644
index 0000000..7b0e94f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_EXT_PORT_H_
+#define _FM10K_SW_EXT_PORT_H_
+
+#include "fm10k_regs.h"
+#include "fm10k_sm.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+
+struct fm10k_sbus;
+struct fm10k_switch;
+
+#define FM10K_SW_TRACE_ENABLE 0
+
+enum fm10k_ext_port_lane_state {
+ FM10K_SW_EXT_PORT_LANE_STATE_DOWN,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_ICAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_PCAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_UP
+};
+
+enum fm10k_ext_port_lane_event {
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_TX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_NOK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_COMPLETE,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_FAILED,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_TIMED_OUT,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RESTORE_IM,
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_DOWN,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_EVENTS
+};
+
+enum fm10k_ext_port_lane_timer {
+ FM10K_SW_EXT_PORT_LANE_TIMER_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_DFE_TUNING,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS
+};
+
+#define FM10K_SW_EXT_PORT_LANE_PLL_LOCK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_SIGNAL_OK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_POLL_MS 200
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_TIMEOUT_MS 3000
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_POLL_MS 100
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_TIMEOUT_MS 2000
+
+
+enum fm10k_ext_port_state {
+ FM10K_SW_EXT_PORT_STATE_DOWN,
+ FM10K_SW_EXT_PORT_STATE_WAITING_FOR_LANES,
+ FM10K_SW_EXT_PORT_STATE_UP
+};
+
+enum fm10k_ext_port_event {
+ FM10K_SW_EXT_PORT_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_EVENT_BRING_DOWN,
+ FM10K_SW_EXT_PORT_EVENT_LANE_UP,
+ FM10K_SW_EXT_PORT_EVENT_LANE_DOWN,
+
+ FM10K_SW_EXT_PORT_NUM_EVENTS
+};
+
+struct fm10k_ext_port;
+
+struct fm10k_ext_port_lane {
+ struct fm10k_ext_port *port;
+ struct fm10k_sm *sm;
+ uint32_t im;
+ uint16_t dfe_poll_time_ms;
+ uint8_t abs_laneno;
+ uint8_t rel_laneno;
+ uint8_t flags;
+};
+
+#define FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED 0x01
+#define FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED 0x02
+
+#define FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED \
+ (FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED | \
+ FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED)
+
+struct fm10k_ext_ports;
+
+struct fm10k_ext_port {
+ struct fm10k_ext_ports *ports;
+ struct fm10k_sm *sm;
+ struct fm10k_ext_port_lane lanes[FM10K_SW_EPL_LANES];
+ char name[10];
+ uint8_t portno;
+ uint8_t eplno;
+ uint8_t lport; /* filled in by switch */
+ uint8_t first_lane; /* range is [0..3] */
+ uint8_t num_lanes;
+ uint8_t is_quad;
+ uint8_t lane_speed;
+ uint8_t last_led_flags;
+ uint32_t an_im;
+ uint32_t link_im;
+};
+
+#define FM10K_SW_EXT_PORT_LED_FLAG_UP 0x01
+#define FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE 0x02
+
+struct fm10k_ext_ports {
+ struct fm10k_switch *sw;
+ struct fm10k_ext_port *ports;
+ uint16_t epl_mask;
+ uint8_t num_ports;
+ uint8_t ports_per_epl;
+};
+
+#define FM10K_SW_EXT_PORTS_EPL_USED(p_, n_) ((p_)->epl_mask & (1 << (n_)))
+
+struct fm10k_ext_ports *fm10k_ext_ports_attach(struct fm10k_switch *sw);
+void fm10k_ext_ports_detach(struct fm10k_ext_ports *ports);
+unsigned int fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports,
+ uint64_t gid);
+void fm10k_ext_port_up(struct fm10k_ext_port *port);
+void fm10k_ext_port_down(struct fm10k_ext_port *port);
+unsigned int fm10k_ext_port_isup(struct fm10k_ext_port *port);
+void fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane);
+
+#endif /* _FM10K_SW_EXT_PORT_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.c b/drivers/net/fm10k/switch/fm10k_serdes.c
new file mode 100644
index 0000000..8f4f2c6
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.c
@@ -0,0 +1,1886 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_serdes.h"
+#include "fm10k_spico_code.h"
+#include "fm10k_switch.h"
+
+
+#define FM10K_SW_SERDES_BIST_TIMEOUT_MS 5000
+#define FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS 5000
+#define FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS 3000
+#define FM10K_SW_SERDES_INT02_TIMEOUT_MS 500
+
+#define FM10K_SW_SERDES_CONFIG_DELAY_US 20
+#define FM10K_SW_SERDES_RESET_DELAY_US 22
+#define FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US 200
+
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_25G 0xa5
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_10G 0x42
+
+#define FM10K_SW_SERDES_WIDTH_40 3 /* 25G */
+#define FM10K_SW_SERDES_WIDTH_20 1 /* 10G */
+
+#define FM10K_SW_SERDES_EQ_SEL_PRECUR 0
+#define FM10K_SW_SERDES_EQ_SEL_ATTEN 1
+#define FM10K_SW_SERDES_EQ_SEL_POSTCUR 2
+
+#define FM10K_SW_SERDES_DFE_DEFAULT_HF 0x00
+#define FM10K_SW_SERDES_DFE_DEFAULT_LF 0x0c
+#define FM10K_SW_SERDES_DFE_DEFAULT_DC 0x38
+#define FM10K_SW_SERDES_DFE_DEFAULT_BW 0x0f
+
+#define FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES 10
+#define FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES 500
+
+/*
+ * These values correspond to IES FM_PORT_LINK_OPTIMIZATION_BALANCE
+ * settings.
+ */
+#define FM10K_SW_LINK_OPT_PARAM_A10G 0x4800
+#define FM10K_SW_LINK_OPT_PARAM_B10G 0xA000
+#define FM10K_SW_LINK_OPT_PARAM_A25G 0x10001
+#define FM10K_SW_LINK_OPT_PARAM_B25G 0x10001
+
+#define FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD 10
+
+
+static int fm10k_spico_ram_bist(struct fm10k_sbus *sb);
+static int fm10k_load_epl_spico_code(struct fm10k_switch *sw);
+static int fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id);
+static int fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb);
+static int fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id);
+static int fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_swap_image_check_crc(struct fm10k_sbus *sb,
+ unsigned int crc_code);
+static int fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words);
+static int fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id);
+static int fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id);
+static int fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider);
+static int fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static void fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static int fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold);
+static int fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq);
+static int fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms);
+static int fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data,
+ uint32_t mask, uint32_t *result);
+static int fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result);
+static int fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param);
+static void fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane);
+static int fm10k_epl_serdes_configure_near_loopback
+ (struct fm10k_ext_port_lane *lane);
+static int fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param);
+static int fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status);
+static int fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged);
+
+
+int
+fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: initializing EPL serdes", sb->name);
+
+ error = fm10k_spico_ram_bist(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_load_epl_spico_code(sw);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_spico_ram_bist(struct fm10k_sbus *sb)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: starting SPICO RAM BIST", sb->name);
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x00, 0x03);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x00, 0x05);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_BIST_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO, 0x00, &data);
+ if (error)
+ goto done;
+
+ if (data & 0x18) {
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO, 0x00, 0x00);
+ if (error)
+ goto done;
+ if ((data & 0x18) != 0x08) {
+ FM10K_SW_INFO("sbus %s: SPICO RAM "
+ "BIST failed with bad status (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+ goto done;
+ }
+ }
+ fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x00, 0x00);
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SPICO RAM BIST timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_load_epl_spico_code(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ const uint16_t *sbm_code_image;
+ uint32_t sbm_code_size;
+ uint32_t sbm_code_version_build_id;
+ const uint16_t *serdes_code_image;
+ uint32_t serdes_code_size;
+ uint32_t serdes_code_version_build_id;
+ const uint16_t *swap_code_image;
+ uint32_t swap_code_size;
+ unsigned int swap_crc_code;
+ int error = 0;
+
+ FM10K_SW_TRACE("sbus %s: loading SPICO code", sb->name);
+
+ sbm_code_image = fm10000_sbus_master_code_prd;
+ sbm_code_size = fm10000_sbus_master_code_size_prd;
+ sbm_code_version_build_id =
+ fm10000_sbus_master_code_version_build_id_prd;
+
+ FM10K_SW_TRACE("sbus %s: sbm code version=%4.4x build=%4.4x",
+ sb->name, sbm_code_version_build_id >> 16,
+ sbm_code_version_build_id & 0xffff);
+
+ serdes_code_image = fm10000_serdes_spico_code_prd2;
+ serdes_code_size = fm10000_serdes_spico_code_size_prd2;
+ serdes_code_version_build_id =
+ fm10000_serdes_spico_code_version_build_id_prd2;
+ sw->epl_serdes_code_version_build_id = serdes_code_version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: serdes code version=%4.4x build=%4.4x",
+ sb->name, serdes_code_version_build_id >> 16,
+ serdes_code_version_build_id & 0xffff);
+
+ swap_code_image = fm10000_serdes_swap_code_prd2;
+ swap_code_size = fm10000_serdes_swap_code_size_prd2;
+
+ error = fm10k_sbm_spico_upload_image(sb, sbm_code_image,
+ sbm_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_check_crc_version_build_id(sb,
+ sbm_code_version_build_id);
+ if (error)
+ goto done;
+
+ if (swap_code_size > 0) {
+ if ((sbm_code_version_build_id & 0x00008000) == 0) {
+ error = fm10k_serdes_swap_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x1a;
+ } else {
+ error = fm10k_serdes_swap_alt_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x04;
+ }
+ if (error)
+ goto done;
+
+ error = fm10k_swap_image_check_crc(sb, swap_crc_code);
+ if (error)
+ goto done;
+ }
+
+ error = fm10k_serdes_spico_upload_image(sb,
+ FM10K_SW_SBUS_ADDR_BROADCAST,
+ serdes_code_image, serdes_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_check_crc_version_build_id(sw,
+ serdes_code_version_build_id);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+static int
+fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM SPICO image "
+ "(%u words)", sb->name, num_words);
+
+ data = (1 << 6) | (1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, data);
+ if (error)
+ goto done;
+
+ data |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ data = 0x80000000 | ((uint32_t)image[i] << 16) | i;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO, 0x03, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM image upload time %u ms",
+ sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ data = (1 << 6);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, data);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x16, data);
+ if (error)
+ goto done;
+
+ data = (1 << 6) | (1 << 8);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id)
+{
+ int error;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SBM SPICO code CRC "
+ "and version", sb->name);
+
+ error = fm10k_sbm_spico_do_crc(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_get_version_build_id(sb, &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO code version "
+ "compare failed (expected 0x%08x, got 0x%08x)",
+ sb->name, expected_version_build_id, version_build_id);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM SPICO image "
+ "version=0x%4.4x build_id=0x%4.4x", sb->name,
+ version_build_id >> 16,
+ version_build_id & 0xffff);
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM SPICO CRC "
+ "check", sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x02, 0, &crc);
+ if (error)
+ goto done;
+
+ crc = crc >> 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO CRC check failed "
+ "(CRC interrupt returned 0xffff)", sb->name);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM SPICO CRC check "
+ "passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM SPICO CRC "
+ "check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id)
+{
+ int error;
+ uint32_t result;
+
+ FM10K_SW_TRACE("sbus %s: getting SBM code version",
+ sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x00, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id = result & 0xffff0000;
+ error = fm10k_sbm_spico_int(sb, 0x01, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id |= (result >> 16) & 0xffff;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ error = fm10k_sbm_spico_int(sb, 0x1c, 0, &addr);
+ if (error)
+ goto done;
+
+ addr >>= 16;
+ if (addr == 0xffff) {
+ FM10K_SW_INFO("sbus %s: invalid build - cannot "
+ "upload SBM swap image", sb->name);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, reg01);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x03, addr);
+ if (error)
+ goto done;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO, 0x03, 0x80000000 | addr);
+ if (error)
+ goto done;
+
+ for (i = 0; i < num_words - 2; i += 3) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x14,
+ 0xc0000000 |
+ image[i] |
+ ((uint32_t)image[i + 1] << 10) |
+ ((uint32_t)image[i + 2] << 20));
+ if (error)
+ goto done;
+ }
+
+ if (num_words - i == 2) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x14,
+ 0x80000000 | image[i] | ((uint32_t)image[i + 1] << 10));
+ if (error)
+ goto done;
+ } else if (num_words - i == 1) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x14,
+ 0x40000000 | image[i]);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image upload time "
+ "%u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~(1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: alt-uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 10) | (1 << 11);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, reg01);
+ if (error)
+ goto done;
+
+ addr = 0x0400;
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x04,
+ 0x8000 |
+ ((uint32_t)image[i + 1] << 16) |
+ (addr + i));
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image alt-upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~((1 << 10) | (1 << 11));
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+static int
+fm10k_swap_image_check_crc(struct fm10k_sbus *sb, unsigned int crc_code)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM swap image CRC "
+ "check", sb->name);
+
+ error =
+ fm10k_sbm_spico_int(sb, crc_code, 0, &crc);
+ if (error)
+ goto done;
+
+ crc >>= 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM swap image CRC check "
+ "failed (CRC interrupt 0x%02x returned 0xffff)",
+ sb->name, crc_code);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM swap image CRC "
+ "check passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM swap image "
+ "CRC check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ uint32_t reg07;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading serdes SPICO image "
+ "to bus address 0x%02x (%u words)", sb->name, dev, num_words);
+
+ reg07 = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, dev, 0x07, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, 0x07, reg07);
+ if (error)
+ goto done;
+
+ data = (1 << 30);
+ error = fm10k_sbus_write(sb, dev, 0x00, data);
+ if (error)
+ goto done;
+
+ data = (1 << 4) | (1 << 5);
+ error = fm10k_sbus_write(sb, dev, 0x08, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i += 3) {
+ data = 0xc0000000 | image[i];
+ if (i + 1 < num_words) {
+ data |= (uint32_t)image[i + 1] << 10;
+
+ if (i + 2 < num_words)
+ data |= (uint32_t)image[i + 2] << 20;
+ }
+ error = fm10k_sbus_write(sb, dev, 0x0a, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: serdes SPICO image upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ error = fm10k_sbus_write(sb, dev, 0x00, 0);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev, 0x01, 0x20000000);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, dev, 0x0b, data);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 0);
+ error = fm10k_sbus_write(sb, dev, 0x07, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, 0x07, reg07);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 1);
+ error = fm10k_sbus_write(sb, dev, 0x07, reg07);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev, 0x08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ unsigned int first = 0;
+ unsigned int last = FM10K_SW_EPLS_MAX * FM10K_SW_EPL_LANES - 1;
+ unsigned int i;
+ int error;
+ uint32_t data;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SPICO code CRC and "
+ "version for serdes %u through %u", sb->name, first, last);
+
+ for (i = first; i <= last; i++) {
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(i), 0xff, &data);
+ if (error || data != 0x01) {
+ FM10K_SW_TRACE("sbus %s: skipping "
+ "serdes %u (error=%d, data=0x%08x)", sb->name, i,
+ error, data);
+ continue;
+ }
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, i);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_get_version_build_id(sw, i,
+ &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SERDES SPICO code "
+ "version compare failed (expected 0x%08x, got "
+ "0x%08x)", sb->name, expected_version_build_id,
+ version_build_id);
+ error = -1;
+ goto done;
+ }
+ }
+ FM10K_SW_TRACE("sbus %s: serdes %u through %u are OK",
+ sb->name, first, last);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: performing SPICO "
+ "CRC check", sb->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3c, 0, &crc);
+ if (error)
+ goto done;
+
+ if (crc != 0) {
+ FM10K_SW_INFO("sbus %s: serdes %u: SPICO CRC check "
+ "failed (CRC interrupt returned 0x%08x)",
+ sb->name, serdes, crc);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t data;
+ uint8_t addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: resetting SPICO",
+ sb->name, serdes);
+
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_sbus_write(sb, addr, 0, 0x00);
+ if (error)
+ goto done;
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+
+ data = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, addr, 0x07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, addr, 0x0b, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 4);
+ error = fm10k_sbus_write(sb, addr, 0x07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 1);
+ error = fm10k_sbus_write(sb, addr, 0x07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, addr, 0x08, 0);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: getting code "
+ "version", sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x00, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id = data << 16;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3f, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id |= data & 0xffff;
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: SBM interrupt 0x%02x "
+ "(param=0x%08x)", sb->name, int_num, param);
+
+ /* int write */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x02,
+ (param << 16) | int_num);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x07, &data);
+ if (error)
+ goto done;
+
+ data |= (1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x07, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO, 0x07, data);
+ if (error)
+ goto done;
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO, 0x08, &data);
+ if (error)
+ goto done;
+
+ if ((data & 0x8000) || ((data & 0x3ff) == 0)) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ FM10K_SW_TRACE("sbus %s: SBM "
+ "interrupt 0x%02x (param=0x%08x, result=0x%08x)",
+ sb->name, int_num, param, data);
+ *result = data;
+ goto done;
+ }
+ }
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SBM interrupt timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ uint64_t start_time_us, elapsed_time_ms;
+ unsigned int sbus_addr;
+ int error;
+ uint32_t data, intr;
+
+ sbus_addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: (sbus addr 0x%02x) "
+ "interrupt 0x%04x (param=0x%08x)", sb->name, serdes, sbus_addr,
+ int_num, param);
+
+ /* int write */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ error = fm10k_sbus_write(sb, sbus_addr, 0x03,
+ ((uint32_t)int_num << 16) | param);
+ if (error)
+ goto done;
+ }
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "(2) after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ FM10K_SW_TRACE("sbus %s: serdes %u: interrupt "
+ "0x%04x (param=0x%08x, result=0x%08x)", sb->name, serdes,
+ int_num, param, data);
+ if (result)
+ *result = data;
+ }
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int divider, width_mode;
+ int error = 0;
+ uint32_t data;
+
+ /*
+ * Bit rate and width
+ */
+ if (speed == 25) {
+ /* 25G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_25G;
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_10G;
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ /*
+ * state = CONFIGURED
+ */
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring",
+ sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_bit_rate(sw, serdes, divider);
+ if (error)
+ goto done;
+
+ fm10k_epl_serdes_set_pcsl_width_mode(sw, serdes,
+ width_mode);
+
+ /*
+ * Data select
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX data select",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_spico_int02_retry(sw, serdes, 0x1ff,
+ FM10K_SW_SERDES_INT02_TIMEOUT_MS);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_reg_write(sw, serdes, 0x21, 0x0c00);
+ if (error)
+ goto done;
+
+ /*
+ * Configure TX equalization
+ *
+ * Use defaults - copper and optical are the same.
+ *
+ * DEFAULTS| precursor | cursor | postcursor |
+ * --------+-----------+--------+------------+
+ * copper | 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ * optical| 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursors",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_ATTEN, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_PRECUR, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_POSTCUR, 15);
+ if (error)
+ goto done;
+
+ /*
+ * Configure 'options', which means:
+ * - PLL calibration mode
+ * - Phase slip
+ * - RX termination
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting PLL calibration mode",
+ sw->epl_sbus->name, serdes);
+ /* enable PLL calibration */
+ data = (1 << 0) | (1 << 1);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x11, data);
+ if (error)
+ goto done;
+ /* There appears to be no action to take for phase slip */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring RX termination",
+ sw->epl_sbus->name, serdes);
+ /* RX termination appears to default to AVDD */
+ data = (1 << 0);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x2b, data);
+ if (error)
+ goto done;
+
+ /*
+ * Set TX and RX lane polarity.
+ * Assuming no inversion required.
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring TX/RX lane polarity",
+ sw->epl_sbus->name, serdes);
+ data = 0x0300;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x13, data, NULL);
+ if (error)
+ goto done;
+
+ /*
+ * Force local fault
+ */
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /*
+ * Enable TX/RX
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX/RX",
+ sw->epl_sbus->name, serdes);
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /*
+ * Clear and enable interrupts
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring interrupts",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_sbus_write(sw->epl_sbus,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes), 0x08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int width_mode;
+ int error = 0;
+ int near_loopback = sw->serdes_loopback;
+
+ if (speed == 25) {
+ /* 25G */
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_pcsl_bit_slip(sw, serdes);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_width_mode(sw, serdes, width_mode);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX output",
+ sw->epl_sbus->name, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x07);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_init_signal_ok(sw, serdes, 0);
+ if (error)
+ goto done;
+
+ if (near_loopback) {
+ /* enable near loopback */
+ error = fm10k_epl_serdes_configure_near_loopback(lane);
+ if (error)
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error = 0;
+
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /* XXX is this delay necessary? in some places the IES SDK does this */
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting bit rate",
+ sw->epl_sbus->name, serdes);
+
+ data = divider & 0x7ff;
+ data |= (1 << 12);
+ data |= (1 << 15);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x05, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting width mode to %u",
+ sw->epl_sbus->name, serdes, width_mode);
+
+ data = FM10K_SW_SERDES_WIDTH_20 | (FM10K_SW_SERDES_WIDTH_20 << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+ data = width_mode | (width_mode << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+done:
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ return error;
+}
+
+static int
+fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursor %u to %u",
+ sw->epl_sbus->name, serdes, which, tx_eq);
+
+ data = (tx_eq & 0xff) | (which << 14);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x15, data);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+void
+fm10k_epl_serdes_set_signal_detect(struct fm10k_ext_port_lane *lane,
+ unsigned int override)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int eplno = lane->port->eplno;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal detect to %s",
+ sw->epl_sbus->name, lane->abs_laneno,
+ (override == 0) ? "NORMAL" :
+ (override == 1) ? "FORCE_GOOD" :
+ (override == 2) ? "FORCE_BAD" : "<unknown>");
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno));
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE,
+ override, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno), data);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ unsigned int eplno, lane;
+ uint32_t pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl width mode",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ if (width_mode == FM10K_SW_SERDES_WIDTH_40) {
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ } else {
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ }
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static int
+fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes)
+{
+ unsigned int eplno, lane;
+ int error;
+ uint32_t int_return, pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl bit slip",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x0c, (1 << 7), &int_return);
+ if (error)
+ goto done;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ if (int_return & 0x1)
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0c, (1 << 8));
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold)
+{
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal OK threshold to %u",
+ sw->epl_sbus->name, serdes, threshold);
+
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x20, 0x20, NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_read_modify_write(sw, serdes, 0x080,
+ (threshold & 0xf) << 2, 0x3c, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t result;
+ int error;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < timeout_ms) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x02, data, &result);
+ if (result == 0x02) {
+ error = 0;
+ break;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ }
+
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x8000 | (addr & 0xff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), result);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x1a, 0x00, result);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr,
+ uint32_t data, uint32_t mask, uint32_t *result)
+{
+ int error;
+ uint32_t read_data;
+
+ error = fm10k_epl_serdes_dma_esb_read(sw, serdes, addr, &read_data);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_write(sw, serdes, addr,
+ (data & mask) | (read_data & ~mask));
+ if (error)
+ goto done;
+ if (result)
+ *result = (data & mask) | (read_data & ~mask);
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param)
+{
+ int error;
+ uint32_t result;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, int_num, param, &result);
+ if (error)
+ goto done;
+ if (result != int_num) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+static void
+fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane)
+{
+ if (sw) {
+ *eplno = serdes / FM10K_SW_EPL_LANES;
+ *lane = serdes % FM10K_SW_EPL_LANES;
+ }
+}
+
+static int
+fm10k_epl_serdes_configure_near_loopback(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring near loopback",
+ sw->epl_sbus->name, serdes);
+
+ /* XXX assuming no lane polarity has been set */
+
+ /* set near loopback mode */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ data = (1 << 8) | (1 << 0);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x08, data);
+ if (error)
+ goto done;
+
+ /* disable tx output, leave tx/ex enabled */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /* set signal detect override to normal */
+ fm10k_epl_serdes_set_signal_detect(lane, FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring serdes for dfe tuning",
+ sw->epl_sbus->name, serdes);
+
+ /*
+ * Configure the serdes to run DFE tuning
+ */
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (0 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_HF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (1 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_LF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (2 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_DC & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (3 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_BW & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ /* allow early link up */
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26, 0x5b01, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning initial calibration (ical)
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe ical",
+ sw->epl_sbus->name, serdes);
+
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_A25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_A10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x01);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_ical_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status, converged;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ if ((dfe_status & 0x11) == 0x11) {
+ *status = FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS;
+ } else {
+ error = fm10k_epl_serdes_get_ical_result(sw,
+ serdes, &converged);
+ if (error)
+ goto done;
+ if (converged)
+ *status = FM10K_SW_SERDES_DFE_ICAL_CONVERGED;
+ else
+ *status = FM10K_SW_SERDES_DFE_ICAL_FAILED;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning periodic calibration (pcal), just one
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe pcal",
+ sw->epl_sbus->name, serdes);
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_B25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_B10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x02);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_pcal_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+ if (dfe_status & 0x02)
+ *status = FM10K_SW_SERDES_DFE_PCAL_COMPLETE;
+ else
+ *status = FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_stop_dfe_tuning(struct fm10k_ext_port_lane *lane,
+ unsigned int force_reset)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ unsigned int ical_stopped, pcal_stopped;
+ unsigned int i;
+ uint32_t dfe_status, status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ ical_stopped = 0;
+ pcal_stopped = 0;
+
+ if (!force_reset) {
+ if (dfe_status & 0x3) {
+ /* stop all DFE tuning */
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes,
+ 0x0a, 0);
+ if (error)
+ goto done;
+
+ for (i = 0;
+ i < FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES;
+ i++) {
+ error = fm10k_epl_serdes_dfe_ical_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ ical_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+
+ if (ical_stopped)
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES;
+ i++) {
+ error =
+ fm10k_epl_serdes_dfe_pcal_status
+ (lane, &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 0;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+
+ if (ical_stopped && pcal_stopped && (dfe_status & 0x40)) {
+ error = fm10k_epl_serdes_start_dfe_pcal(lane);
+ if (error)
+ goto done;
+ pcal_stopped = 0;
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES; i++) {
+ error = fm10k_epl_serdes_dfe_pcal_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+ }
+
+ if (!ical_stopped || !pcal_stopped) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x18, 0x07);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param & 0xffff);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param >> 16);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126, (0x0b << 8),
+ status);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: dfe status: "
+ "coarse_active=%u fine_active=%u adaptive=%u",
+ sw->epl_sbus->name, serdes, *status & (1 << 0),
+ *status & (1 << 1), *status & (1 << 6));
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged)
+{
+ uint32_t val1;
+ uint32_t val2;
+ uint32_t diff;
+ int error;
+
+ *converged = 0;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12), &val1);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12) | (1 << 8), &val2);
+ if (error)
+ goto done;
+
+ /* sign extension */
+ if (val1 & 0x8000)
+ val1 |= 0xffff0000;
+ if (val2 & 0x8000)
+ val2 |= 0xffff0000;
+ diff = abs(val2 - val1);
+
+ if (diff >= FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD)
+ *converged = 1;
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.h b/drivers/net/fm10k/switch/fm10k_serdes.h
new file mode 100644
index 0000000..d7c07b1
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SERDES_H_
+#define _FM10K_SW_SERDES_H_
+
+
+#define FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_ICAL_CONVERGED 1
+#define FM10K_SW_SERDES_DFE_ICAL_FAILED 2
+#define FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_PCAL_COMPLETE 1
+
+
+int fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw);
+int fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_ical_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_pcal_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_stop_dfe_tuning
+ (struct fm10k_ext_port_lane *lane, unsigned int force_reset);
+void fm10k_epl_serdes_set_signal_detect
+ (struct fm10k_ext_port_lane *lane, unsigned int override);
+
+#endif /* _FM10K_SW_SERDES_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sm.c b/drivers/net/fm10k/switch/fm10k_sm.c
new file mode 100644
index 0000000..74fb023
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.c
@@ -0,0 +1,182 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <rte_malloc.h>
+
+#include "../base/fm10k_osdep.h"
+#include "fm10k_debug.h"
+#include "fm10k_sm.h"
+
+static void *fm10k_sm_handle_event(void *ptr);
+static void *fm10k_sm_handle_timer(void *ptr);
+static void fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap);
+static uint16_t fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap);
+static void fm10k_sm_event_heap_push
+ (struct fm10k_sm_event_heap *heap, uint16_t id);
+
+
+struct fm10k_sm *
+fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx)
+{
+ struct fm10k_sm *sm;
+ struct fm10k_sm_timer *timer;
+ unsigned int i;
+
+ sm = (struct fm10k_sm *)rte_zmalloc("fm10k_sm",
+ sizeof(struct fm10k_sm), 0);
+ if (sm == NULL)
+ goto fail;
+
+ sm->num_events = num_events;
+ fm10k_sm_event_heap_init(&sm->events);
+ pthread_create(&sm->event_task, NULL, fm10k_sm_handle_event, sm);
+ sm->num_timers = num_timers;
+ if (sm->num_timers > 0) {
+ sm->timers = (struct fm10k_sm_timer *)rte_zmalloc("fm10k_sm_timers",
+ sizeof(struct fm10k_sm_timer) * sm->num_timers, 0);
+ if (sm->timers == NULL)
+ goto fail;
+ }
+
+ sm->event_handler = event_handler;
+ sm->timer_handler = timer_handler;
+ sm->ctx = ctx;
+
+ for (i = 0; i < sm->num_timers; i++) {
+ timer = &sm->timers[i];
+ sem_init(&timer->tq, 0, 0);
+ timer->sm = sm;
+ pthread_create(&timer->t, NULL, fm10k_sm_handle_timer, timer);
+ timer->timer_id = i;
+ }
+
+ return sm;
+
+fail:
+ if (sm)
+ fm10k_sm_detach(sm);
+
+ return NULL;
+}
+
+void
+fm10k_sm_detach(struct fm10k_sm *sm)
+{
+ rte_free(sm);
+}
+
+void
+fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id)
+{
+ fm10k_sm_event_heap_push(&sm->events, event_id);
+}
+
+void
+fm10k_sm_timer_start(struct fm10k_sm *sm, uint16_t timer_id,
+ unsigned int timeout_ms)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+
+ gettimeofday(&timer->start, 0);
+ timer->timeout_ms = timeout_ms;
+ timer->running = 1;
+ sem_post(&timer->tq);
+}
+
+/*
+ * It is possible that an expiration events from this timer will
+ * occur after this is called.
+ */
+void
+fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+ timer->running = 0;
+}
+
+static void *
+fm10k_sm_handle_event(void *ctx)
+{
+ struct fm10k_sm *sm = ctx;
+ uint16_t event_id;
+
+ while (1) {
+ event_id = fm10k_sm_event_heap_pop(&sm->events);
+ sm->event_handler(sm, event_id);
+ }
+ return NULL;
+}
+
+static void *
+fm10k_sm_handle_timer(void *ctx)
+{
+ struct fm10k_sm_timer *timer = ctx;
+ struct fm10k_sm *sm = timer->sm;
+ struct timeval now;
+ struct timeval sleep;
+ uint64_t cost, timeout;
+
+ while (1) {
+ sem_wait(&timer->tq);
+ if (timer->running) {
+ gettimeofday(&now, 0);
+ cost = (now.tv_sec - timer->start.tv_sec) * 1000000;
+ cost += (now.tv_usec - timer->start.tv_usec);
+ if (cost < timer->timeout_ms * 1000) {
+ timeout = timer->timeout_ms * 1000;
+ timeout -= cost;
+ sleep.tv_sec = timeout / 1000000;
+ sleep.tv_usec = timeout % 1000000;
+ select(0, NULL, NULL, NULL, &sleep);
+ }
+ if (timer->running)
+ sm->timer_handler(sm, timer->timer_id);
+ }
+ }
+
+ return NULL;
+}
+
+
+static void
+fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap)
+{
+ heap->count = 0;
+ pthread_mutex_init(&heap->lock, NULL);
+ sem_init(&heap->s, 0, 0);
+}
+
+static uint16_t
+fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap)
+{
+ int i;
+ uint16_t id;
+
+ sem_wait(&heap->s);
+ pthread_mutex_lock(&heap->lock);
+ id = heap->heap[0];
+ for (i = 1; i < heap->count; i++)
+ heap->heap[i - 1] = heap->heap[i];
+ heap->count--;
+ pthread_mutex_unlock(&heap->lock);
+ return id;
+}
+
+static void
+fm10k_sm_event_heap_push(struct fm10k_sm_event_heap *heap, uint16_t id)
+{
+ pthread_mutex_lock(&heap->lock);
+ if (heap->count >= FM10K_SM_ID_HEAP_MAX) {
+ pthread_mutex_unlock(&heap->lock);
+ return;
+ }
+ heap->heap[heap->count] = id;
+ heap->count++;
+ pthread_mutex_unlock(&heap->lock);
+ sem_post(&heap->s);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sm.h b/drivers/net/fm10k/switch/fm10k_sm.h
new file mode 100644
index 0000000..1b6e3a9
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SM_H_
+#define _FM10K_SW_SM_H_
+
+#include <stdint.h>
+
+#include <signal.h>
+#include <sys/time.h>
+#include <pthread.h>
+#include <semaphore.h>
+
+/*
+ * This framework provides state machine implementation support for use in
+ * the rest of the driver.
+ *
+ * The model is that for a given state machine, there are N>0 events
+ * identified by the integers [0..N-1] and there are M>=0 timers identified
+ * by the integers [0..M-1].
+ *
+ * The timers are one-shot timers.
+ *
+ * Since all state machine processing occurs in the context of the
+ * task queue, if that task queue was created with only a single thread, then
+ * no locking is required in the provided event and timer handlers to
+ * prevent overlapping execution.
+ *
+ */
+
+struct fm10k_sm;
+
+struct fm10k_sm_timer {
+ struct fm10k_sm *sm;
+ pthread_t t;
+ sem_t tq;
+ struct timeval start;
+ uint16_t timeout_ms;
+ uint16_t timer_id;
+ uint8_t running;
+};
+
+typedef void (*fm10k_sm_process_event_t)(struct fm10k_sm *sm, uint16_t event);
+typedef void (*fm10k_sm_process_timer_t)(struct fm10k_sm *sm, uint16_t timer);
+
+#define FM10K_SM_ID_HEAP_MAX 128
+struct fm10k_sm_event_heap {
+ pthread_mutex_t lock;
+ sem_t s;
+ uint16_t count;
+ uint16_t heap[FM10K_SM_ID_HEAP_MAX];
+};
+
+struct fm10k_sm {
+ pthread_t event_task;
+ struct fm10k_sm_timer *timers;
+ struct fm10k_sm_event_heap events;
+ fm10k_sm_process_event_t event_handler;
+ fm10k_sm_process_timer_t timer_handler;
+ void *ctx;
+ unsigned int state;
+ uint16_t num_events;
+ uint16_t num_timers;
+ uint8_t portno;
+ uint8_t laneno;
+};
+
+struct fm10k_sm *fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx);
+void fm10k_sm_detach(struct fm10k_sm *sm);
+void fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id);
+void fm10k_sm_timer_start(struct fm10k_sm *sm,
+ uint16_t timer_id, unsigned int timeout_ms);
+void fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id);
+
+#endif /* _FM10K_SW_SM_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.c b/drivers/net/fm10k/switch/fm10k_spico_code.c
new file mode 100644
index 0000000..cfab632
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.c
@@ -0,0 +1,2966 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include "fm10k_spico_code.h"
+
+
+/* Production versions */
+
+
+/* SBus master code (production version)
+ * source file: release_2015_01_14/sbus_master.0x1013_0001.rom
+ */
+
+const uint32_t fm10000_sbus_master_code_version_build_id_prd = 0x10130001;
+const uint32_t fm10000_sbus_master_code_size_prd = 3338;
+
+const uint16_t fm10000_sbus_master_code_prd[] = {
+ 0x132, 0x002, 0x006, 0x000, 0x390, 0x002, 0x080, 0x247,
+ 0x071, 0x247, 0x01f, 0x247, 0x020, 0x247, 0x02e, 0x247,
+ 0x02f, 0x247, 0x03f, 0x247, 0x040, 0x247, 0x042, 0x247,
+ 0x043, 0x247, 0x03d, 0x0c0, 0x02d, 0x247, 0x031, 0x247,
+ 0x032, 0x081, 0x18b, 0x247, 0x039, 0x248, 0x03a, 0x181,
+ 0x247, 0x034, 0x248, 0x036, 0x209, 0x247, 0x033, 0x248,
+ 0x035, 0x305, 0x034, 0x247, 0x037, 0x248, 0x038, 0x190,
+ 0x247, 0x03b, 0x248, 0x03c, 0x3d4, 0x1ff, 0x00c, 0x3d5,
+ 0x100, 0x00c, 0x3d4, 0x2ff, 0x00c, 0x3c7, 0x0fe, 0x018,
+ 0x0c2, 0x019, 0x044, 0x233, 0x000, 0x152, 0x01b, 0x347,
+ 0x01b, 0x030, 0x085, 0x187, 0x247, 0x03e, 0x3c7, 0x100,
+ 0x041, 0x080, 0x247, 0x065, 0x3c4, 0x3fc, 0x010, 0x3d4,
+ 0x22f, 0x00e, 0x347, 0x033, 0x008, 0x247, 0x008, 0x081,
+ 0x189, 0x227, 0x2c7, 0x044, 0x220, 0x2c7, 0x045, 0x220,
+ 0x2c7, 0x046, 0x220, 0x2c7, 0x047, 0x220, 0x2c7, 0x048,
+ 0x220, 0x2c7, 0x049, 0x220, 0x2c7, 0x04a, 0x220, 0x2c7,
+ 0x04b, 0x3a7, 0x05b, 0x1a7, 0x3a5, 0x08d, 0x2c7, 0x04c,
+ 0x0c0, 0x04d, 0x044, 0x07d, 0x001, 0x0df, 0x02c, 0x000,
+ 0x009, 0x322, 0x02c, 0x009, 0x005, 0x347, 0x02d, 0x02c,
+ 0x3d4, 0x3df, 0x00e, 0x307, 0x02c, 0x384, 0x01f, 0x229,
+ 0x220, 0x3b0, 0x0aa, 0x287, 0x199, 0x390, 0x000, 0x041,
+ 0x040, 0x045, 0x020, 0x12c, 0x000, 0x020, 0x163, 0x000,
+ 0x020, 0x175, 0x000, 0x020, 0x27e, 0x000, 0x020, 0x362,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x1de, 0x000, 0x020, 0x330, 0x001, 0x020, 0x270, 0x001,
+ 0x020, 0x2af, 0x001, 0x020, 0x2a1, 0x001, 0x020, 0x338,
+ 0x001, 0x020, 0x285, 0x001, 0x020, 0x2b9, 0x001, 0x020,
+ 0x2a8, 0x001, 0x020, 0x001, 0x002, 0x020, 0x105, 0x001,
+ 0x020, 0x19e, 0x001, 0x020, 0x3e6, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000,
+ 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x044, 0x140, 0x000, 0x054, 0x362, 0x03d,
+ 0x000, 0x001, 0x0b0, 0x364, 0x03d, 0x001, 0x001, 0x0c6,
+ 0x307, 0x02e, 0x044, 0x187, 0x000, 0x055, 0x141, 0x02e,
+ 0x307, 0x030, 0x302, 0x02e, 0x00b, 0x374, 0x0c0, 0x02e,
+ 0x327, 0x02d, 0x000, 0x367, 0x044, 0x140, 0x000, 0x3d4,
+ 0x3df, 0x00e, 0x207, 0x009, 0x00a, 0x04f, 0x04e, 0x055,
+ 0x3d5, 0x200, 0x014, 0x000, 0x35d, 0x054, 0x000, 0x3f8,
+ 0x055, 0x081, 0x364, 0x010, 0x001, 0x001, 0x00d, 0x364,
+ 0x011, 0x002, 0x009, 0x009, 0x3c4, 0x3fe, 0x010, 0x3d4,
+ 0x3bf, 0x00e, 0x080, 0x101, 0x364, 0x010, 0x002, 0x001,
+ 0x00a, 0x364, 0x011, 0x001, 0x001, 0x006, 0x3c4, 0x3fd,
+ 0x010, 0x111, 0x045, 0x347, 0x026, 0x018, 0x347, 0x027,
+ 0x019, 0x347, 0x028, 0x01b, 0x347, 0x029, 0x01c, 0x044,
+ 0x222, 0x000, 0x0a1, 0x000, 0x31e, 0x347, 0x026, 0x018,
+ 0x347, 0x027, 0x019, 0x044, 0x233, 0x000, 0x347, 0x01b,
+ 0x028, 0x347, 0x01c, 0x029, 0x0a2, 0x000, 0x30c, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x000, 0x001, 0x016, 0x362,
+ 0x01b, 0x001, 0x001, 0x013, 0x362, 0x01b, 0x002, 0x001,
+ 0x00d, 0x362, 0x01b, 0x003, 0x001, 0x013, 0x362, 0x01b,
+ 0x004, 0x001, 0x013, 0x04e, 0x045, 0x364, 0x03d, 0x002,
+ 0x001, 0x005, 0x044, 0x01a, 0x001, 0x000, 0x3f7, 0x348,
+ 0x01b, 0x01a, 0x000, 0x3f2, 0x348, 0x01b, 0x01a, 0x000,
+ 0x3ed, 0x101, 0x247, 0x018, 0x3d7, 0x0ff, 0x019, 0x000,
+ 0x074, 0x3c7, 0x100, 0x014, 0x364, 0x004, 0x100, 0x009,
+ 0x00f, 0x044, 0x08d, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x055, 0x0c1,
+ 0x03d, 0x0bf, 0x000, 0x2b7, 0x000, 0x2bc, 0x3c7, 0x3ff,
+ 0x028, 0x087, 0x0b6, 0x0c1, 0x04e, 0x044, 0x37c, 0x000,
+ 0x347, 0x067, 0x029, 0x2c7, 0x02a, 0x322, 0x04e, 0x009,
+ 0x00c, 0x3c2, 0x016, 0x067, 0x009, 0x00c, 0x0c1, 0x028,
+ 0x0a7, 0x000, 0x298, 0x347, 0x033, 0x028, 0x000, 0x3fa,
+ 0x347, 0x034, 0x028, 0x000, 0x3f5, 0x111, 0x009, 0x3ff,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x040,
+ 0x103, 0x051, 0x042, 0x2a7, 0x001, 0x00c, 0x040, 0x280,
+ 0x051, 0x1a7, 0x101, 0x050, 0x225, 0x042, 0x280, 0x045,
+ 0x287, 0x045, 0x364, 0x005, 0x008, 0x009, 0x3fd, 0x347,
+ 0x01b, 0x00a, 0x347, 0x01c, 0x00b, 0x3c7, 0x041, 0x00c,
+ 0x020, 0x260, 0x000, 0x364, 0x005, 0x008, 0x009, 0x3fd,
+ 0x3d5, 0x100, 0x00c, 0x3c7, 0x042, 0x00c, 0x044, 0x260,
+ 0x000, 0x364, 0x005, 0x010, 0x001, 0x3fd, 0x307, 0x005,
+ 0x384, 0x007, 0x262, 0x004, 0x009, 0x009, 0x347, 0x002,
+ 0x01b, 0x347, 0x003, 0x01c, 0x045, 0x262, 0x006, 0x009,
+ 0x3ef, 0x141, 0x020, 0x0c0, 0x01b, 0x0c0, 0x01c, 0x045,
+ 0x347, 0x019, 0x00d, 0x1c7, 0x00d, 0x345, 0x018, 0x00d,
+ 0x3d5, 0x200, 0x00c, 0x364, 0x005, 0x008, 0x001, 0x3fd,
+ 0x3d4, 0x1ff, 0x00c, 0x045, 0x364, 0x005, 0x008, 0x009,
+ 0x3fd, 0x0c0, 0x00c, 0x020, 0x260, 0x000, 0x054, 0x347,
+ 0x032, 0x031, 0x307, 0x031, 0x044, 0x2ad, 0x000, 0x141,
+ 0x031, 0x307, 0x030, 0x302, 0x031, 0x00b, 0x004, 0x0c0,
+ 0x031, 0x2a7, 0x001, 0x00a, 0x347, 0x031, 0x032, 0x347,
+ 0x031, 0x070, 0x000, 0x00b, 0x307, 0x032, 0x302, 0x031,
+ 0x009, 0x3e2, 0x3c7, 0x3ff, 0x070, 0x055, 0x3d4, 0x2ff,
+ 0x00e, 0x0a3, 0x020, 0x091, 0x000, 0x044, 0x1b9, 0x000,
+ 0x362, 0x01b, 0x001, 0x001, 0x009, 0x362, 0x01b, 0x016,
+ 0x001, 0x004, 0x000, 0x00b, 0x0cf, 0x019, 0x044, 0x233,
+ 0x000, 0x307, 0x01b, 0x003, 0x004, 0x0a0, 0x045, 0x304,
+ 0x035, 0x227, 0x1bc, 0x2e2, 0x003, 0x001, 0x093, 0x2e2,
+ 0x002, 0x001, 0x046, 0x2e2, 0x001, 0x001, 0x063, 0x227,
+ 0x001, 0x03d, 0x2e4, 0x001, 0x009, 0x010, 0x2e4, 0x002,
+ 0x009, 0x019, 0x2e4, 0x004, 0x009, 0x029, 0x2e4, 0x008,
+ 0x009, 0x029, 0x000, 0x02b, 0x041, 0x0a2, 0x1a7, 0x3b5,
+ 0x03a, 0x044, 0x1b1, 0x001, 0x043, 0x3a4, 0x3fe, 0x000,
+ 0x3e1, 0x041, 0x327, 0x018, 0x041, 0x044, 0x08d, 0x001,
+ 0x247, 0x01e, 0x043, 0x2c7, 0x018, 0x044, 0x0fb, 0x001,
+ 0x043, 0x3a4, 0x3fd, 0x000, 0x3cd, 0x3a4, 0x3fb, 0x000,
+ 0x3c9, 0x3a4, 0x3f7, 0x000, 0x3c5, 0x0a1, 0x045, 0x227,
+ 0x3a4, 0x0ff, 0x197, 0x384, 0x01f, 0x001, 0x010, 0x262,
+ 0x001, 0x001, 0x00e, 0x262, 0x002, 0x001, 0x00f, 0x262,
+ 0x003, 0x001, 0x00d, 0x000, 0x3ea, 0x000, 0x3e8, 0x044,
+ 0x079, 0x002, 0x000, 0x3e3, 0x000, 0x3e1, 0x000, 0x3df,
+ 0x0bf, 0x1aa, 0x2a8, 0x224, 0x19a, 0x384, 0x003, 0x001,
+ 0x00c, 0x262, 0x001, 0x001, 0x017, 0x262, 0x002, 0x001,
+ 0x015, 0x000, 0x015, 0x0c5, 0x019, 0x247, 0x01b, 0x0a2,
+ 0x1a7, 0x3b5, 0x03a, 0x2c7, 0x01c, 0x044, 0x222, 0x000,
+ 0x000, 0x3bd, 0x000, 0x3bb, 0x000, 0x3b9, 0x000, 0x3b7,
+ 0x000, 0x3b5, 0x054, 0x307, 0x066, 0x191, 0x384, 0x07f,
+ 0x3c7, 0x0fe, 0x018, 0x0c2, 0x019, 0x044, 0x233, 0x000,
+ 0x347, 0x01b, 0x064, 0x0a7, 0x044, 0x0bf, 0x003, 0x055,
+ 0x0a4, 0x020, 0x091, 0x000, 0x186, 0x285, 0x181, 0x0a7,
+ 0x1ab, 0x324, 0x004, 0x285, 0x055, 0x364, 0x010, 0x002,
+ 0x009, 0x026, 0x364, 0x011, 0x001, 0x009, 0x029, 0x247,
+ 0x064, 0x044, 0x0cd, 0x003, 0x3c4, 0x2ff, 0x065, 0x04f,
+ 0x04e, 0x055, 0x364, 0x065, 0x100, 0x001, 0x3fa, 0x327,
+ 0x065, 0x3a4, 0x007, 0x364, 0x065, 0x040, 0x001, 0x007,
+ 0x287, 0x302, 0x04e, 0x009, 0x3ec, 0x045, 0x364, 0x011,
+ 0x001, 0x001, 0x005, 0x3c4, 0x3fd, 0x010, 0x054, 0x04e,
+ 0x000, 0x3cc, 0x309, 0x023, 0x003, 0x024, 0x347, 0x023,
+ 0x04f, 0x00b, 0x01f, 0x3d5, 0x010, 0x00e, 0x08f, 0x187,
+ 0x395, 0x0a0, 0x247, 0x017, 0x080, 0x247, 0x016, 0x0c0,
+ 0x015, 0x3c5, 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4,
+ 0x3fe, 0x014, 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e,
+ 0x347, 0x04f, 0x009, 0x020, 0x20b, 0x002, 0x307, 0x04f,
+ 0x00b, 0x028, 0x3c5, 0x002, 0x03d, 0x3d5, 0x010, 0x00e,
+ 0x387, 0x0b2, 0x187, 0x385, 0x080, 0x247, 0x017, 0x08e,
+ 0x187, 0x385, 0x0e6, 0x247, 0x016, 0x0c0, 0x015, 0x3c5,
+ 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4, 0x3fe, 0x014,
+ 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e, 0x000, 0x006,
+ 0x082, 0x208, 0x244, 0x03d, 0x0df, 0x02c, 0x0b3, 0x020,
+ 0x091, 0x000, 0x307, 0x018, 0x3a7, 0x051, 0x044, 0x061,
+ 0x001, 0x2c7, 0x061, 0x04c, 0x324, 0x061, 0x001, 0x020,
+ 0x040, 0x3b7, 0x126, 0x08b, 0x187, 0x044, 0x1c4, 0x001,
+ 0x009, 0x005, 0x3c7, 0x022, 0x01b, 0x042, 0x364, 0x01b,
+ 0x022, 0x009, 0x027, 0x04c, 0x348, 0x061, 0x061, 0x324,
+ 0x061, 0x04d, 0x151, 0x050, 0x000, 0x01c, 0x327, 0x04f,
+ 0x3a4, 0x0ff, 0x322, 0x050, 0x00f, 0x014, 0x040, 0x0aa,
+ 0x084, 0x044, 0x1c4, 0x001, 0x042, 0x362, 0x01b, 0x00a,
+ 0x009, 0x008, 0x04c, 0x325, 0x061, 0x04d, 0x141, 0x050,
+ 0x045, 0x384, 0x0ff, 0x040, 0x193, 0x220, 0x042, 0x384,
+ 0x00f, 0x041, 0x0a1, 0x111, 0x003, 0x005, 0x2a9, 0x000,
+ 0x3fc, 0x042, 0x045, 0x0a0, 0x0d0, 0x061, 0x04d, 0x101,
+ 0x151, 0x061, 0x009, 0x3fc, 0x045, 0x0c0, 0x04f, 0x0c0,
+ 0x050, 0x387, 0x051, 0x044, 0x073, 0x001, 0x044, 0x209,
+ 0x000, 0x051, 0x101, 0x051, 0x045, 0x307, 0x03f, 0x001,
+ 0x006, 0x247, 0x018, 0x000, 0x01f, 0x0a0, 0x287, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x012, 0x001, 0x012, 0x362,
+ 0x01b, 0x011, 0x009, 0x005, 0x347, 0x018, 0x040, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3eb, 0x0a0, 0x045, 0x347,
+ 0x018, 0x03f, 0x0c0, 0x019, 0x3c7, 0x070, 0x01b, 0x3c5,
+ 0x001, 0x01b, 0x044, 0x222, 0x000, 0x3d4, 0x3fe, 0x01b,
+ 0x044, 0x222, 0x000, 0x307, 0x03e, 0x044, 0x205, 0x000,
+ 0x3c5, 0x002, 0x01b, 0x044, 0x222, 0x000, 0x307, 0x03e,
+ 0x194, 0x044, 0x205, 0x000, 0x0c1, 0x019, 0x044, 0x233,
+ 0x000, 0x327, 0x01b, 0x0c0, 0x019, 0x3c5, 0x001, 0x01b,
+ 0x044, 0x222, 0x000, 0x3b5, 0x200, 0x045, 0x2c7, 0x01e,
+ 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001, 0x007, 0x041,
+ 0x044, 0x0fb, 0x001, 0x043, 0x121, 0x307, 0x030, 0x282,
+ 0x00b, 0x3f1, 0x045, 0x3b7, 0x03b, 0x307, 0x01e, 0x000,
+ 0x0b2, 0x091, 0x020, 0x362, 0x002, 0x054, 0x327, 0x06f,
+ 0x009, 0x009, 0x044, 0x08d, 0x001, 0x001, 0x080, 0x000,
+ 0x00d, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x012, 0x009,
+ 0x076, 0x044, 0x0b2, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x287, 0x001, 0x06b, 0x2c7, 0x01d, 0x327, 0x040,
+ 0x3a4, 0x0ff, 0x2c7, 0x018, 0x0c9, 0x01a, 0x044, 0x2d7,
+ 0x001, 0x044, 0x206, 0x001, 0x324, 0x035, 0x0c0, 0x01a,
+ 0x087, 0x187, 0x385, 0x06c, 0x282, 0x00b, 0x015, 0x0c1,
+ 0x01a, 0x087, 0x187, 0x385, 0x0bc, 0x282, 0x00b, 0x00c,
+ 0x0c2, 0x01a, 0x088, 0x187, 0x08c, 0x282, 0x00b, 0x004,
+ 0x0c3, 0x01a, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x010, 0x041, 0x307, 0x01a, 0x18b, 0x305, 0x042, 0x385,
+ 0x0e0, 0x3b7, 0x33b, 0x044, 0x1b1, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3e8, 0x307, 0x03e, 0x044,
+ 0x205, 0x000, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x00b, 0x041, 0x307, 0x01d, 0x3b7, 0x03b, 0x044, 0x1b1,
+ 0x001, 0x043, 0x121, 0x307, 0x030, 0x282, 0x00b, 0x3ed,
+ 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387, 0x3ff, 0x3c7,
+ 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296, 0x001, 0x347,
+ 0x023, 0x043, 0x092, 0x020, 0x365, 0x002, 0x054, 0x044,
+ 0x1de, 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296,
+ 0x001, 0x040, 0x044, 0x1c9, 0x001, 0x001, 0x00c, 0x042,
+ 0x0c5, 0x019, 0x2c7, 0x01c, 0x247, 0x01b, 0x020, 0x222,
+ 0x000, 0x042, 0x080, 0x045, 0x044, 0x1b1, 0x001, 0x001,
+ 0x3fb, 0x041, 0x3a7, 0x0ff, 0x0c6, 0x019, 0x131, 0x001,
+ 0x00c, 0x044, 0x233, 0x000, 0x30a, 0x01c, 0x002, 0x3f8,
+ 0x081, 0x000, 0x003, 0x080, 0x043, 0x045, 0x307, 0x040,
+ 0x384, 0x0ff, 0x009, 0x017, 0x0a0, 0x287, 0x044, 0x1b9,
+ 0x000, 0x362, 0x01b, 0x011, 0x001, 0x00a, 0x121, 0x307,
+ 0x030, 0x282, 0x00b, 0x3f3, 0x0a0, 0x045, 0x347, 0x018,
+ 0x040, 0x0c0, 0x01a, 0x327, 0x040, 0x044, 0x2c3, 0x001,
+ 0x324, 0x034, 0x009, 0x3f2, 0x227, 0x045, 0x247, 0x01c,
+ 0x2c7, 0x01b, 0x280, 0x001, 0x3e9, 0x0c3, 0x019, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222,
+ 0x000, 0x0c2, 0x01b, 0x044, 0x222, 0x000, 0x347, 0x01e,
+ 0x019, 0x3d0, 0x041, 0x019, 0x307, 0x041, 0x044, 0x205,
+ 0x000, 0x044, 0x233, 0x000, 0x327, 0x01b, 0x00b, 0x3f6,
+ 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x287,
+ 0x045, 0x2c7, 0x01e, 0x0a0, 0x287, 0x044, 0x300, 0x001,
+ 0x001, 0x007, 0x041, 0x044, 0x24e, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3f1, 0x045, 0x09f, 0x18a,
+ 0x0a1, 0x1aa, 0x324, 0x01e, 0x001, 0x006, 0x305, 0x01e,
+ 0x000, 0x005, 0x208, 0x304, 0x01e, 0x300, 0x043, 0x00b,
+ 0x005, 0x080, 0x000, 0x008, 0x272, 0x3e8, 0x00b, 0x004,
+ 0x397, 0x3e8, 0x305, 0x033, 0x3b7, 0x13b, 0x000, 0x343,
+ 0x327, 0x06e, 0x0c0, 0x01a, 0x044, 0x2c3, 0x001, 0x0a9,
+ 0x305, 0x033, 0x247, 0x007, 0x344, 0x035, 0x007, 0x3d4,
+ 0x37f, 0x00e, 0x020, 0x091, 0x000, 0x327, 0x06f, 0x0c0,
+ 0x01a, 0x044, 0x2c3, 0x001, 0x324, 0x034, 0x009, 0x005,
+ 0x347, 0x06f, 0x040, 0x0ad, 0x0c1, 0x008, 0x247, 0x009,
+ 0x345, 0x033, 0x008, 0x344, 0x035, 0x008, 0x020, 0x091,
+ 0x000, 0x327, 0x06e, 0x054, 0x055, 0x0ab, 0x000, 0x3d2,
+ 0x327, 0x06f, 0x054, 0x055, 0x0af, 0x000, 0x3e7, 0x327,
+ 0x06e, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001, 0x0aa, 0x000,
+ 0x3c1, 0x327, 0x06f, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001,
+ 0x0ae, 0x000, 0x3d3, 0x044, 0x2f8, 0x001, 0x362, 0x01b,
+ 0x011, 0x009, 0x00a, 0x044, 0x2d7, 0x001, 0x044, 0x206,
+ 0x001, 0x055, 0x045, 0x307, 0x034, 0x000, 0x3fc, 0x080,
+ 0x1bb, 0x2e2, 0x008, 0x00b, 0x00d, 0x0a1, 0x0c0, 0x01e,
+ 0x340, 0x01a, 0x01e, 0x001, 0x014, 0x1a8, 0x000, 0x011,
+ 0x320, 0x01a, 0x2c7, 0x01e, 0x2c7, 0x01a, 0x0a1, 0x151,
+ 0x01a, 0x003, 0x006, 0x2a9, 0x20b, 0x000, 0x3fa, 0x045,
+ 0x054, 0x287, 0x384, 0x0ff, 0x111, 0x020, 0x1b9, 0x000,
+ 0x044, 0x1b9, 0x000, 0x362, 0x01b, 0x016, 0x001, 0x007,
+ 0x362, 0x01b, 0x001, 0x009, 0x023, 0x3d7, 0x0fe, 0x019,
+ 0x044, 0x233, 0x000, 0x387, 0x0bb, 0x187, 0x385, 0x031,
+ 0x302, 0x01b, 0x001, 0x014, 0x387, 0x0b9, 0x187, 0x385,
+ 0x031, 0x302, 0x01b, 0x001, 0x00b, 0x387, 0x0b6, 0x187,
+ 0x385, 0x031, 0x302, 0x01b, 0x009, 0x003, 0x080, 0x045,
+ 0x327, 0x06e, 0x044, 0x340, 0x001, 0x0a8, 0x000, 0x342,
+ 0x327, 0x06f, 0x044, 0x340, 0x001, 0x0ac, 0x000, 0x356,
+ 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x00b, 0x009, 0x38d,
+ 0x0c0, 0x01c, 0x041, 0x1bb, 0x3a4, 0x007, 0x001, 0x047,
+ 0x131, 0x001, 0x02f, 0x131, 0x001, 0x017, 0x083, 0x3a7,
+ 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0, 0x044, 0x3f6,
+ 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x081, 0x044, 0x3f6,
+ 0x001, 0x000, 0x03e, 0x082, 0x3a7, 0x3ff, 0x044, 0x3f6,
+ 0x001, 0x084, 0x0a0, 0x044, 0x3f6, 0x001, 0x083, 0x044,
+ 0x3f6, 0x001, 0x081, 0x044, 0x3f6, 0x001, 0x000, 0x029,
+ 0x081, 0x3a7, 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0,
+ 0x044, 0x3f6, 0x001, 0x083, 0x044, 0x3f6, 0x001, 0x082,
+ 0x044, 0x3f6, 0x001, 0x000, 0x014, 0x081, 0x3a7, 0x3ff,
+ 0x044, 0x3f6, 0x001, 0x084, 0x044, 0x3f6, 0x001, 0x083,
+ 0x044, 0x3f6, 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x0c6,
+ 0x019, 0x0c0, 0x01b, 0x044, 0x222, 0x000, 0x0c5, 0x019,
+ 0x387, 0x0ff, 0x3a7, 0x3ff, 0x322, 0x04c, 0x303, 0x04d,
+ 0x2c7, 0x01b, 0x247, 0x01c, 0x044, 0x222, 0x000, 0x0c0,
+ 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x397, 0x3ff,
+ 0x044, 0x205, 0x000, 0x044, 0x233, 0x000, 0x3c4, 0x001,
+ 0x01b, 0x009, 0x3f5, 0x0c6, 0x019, 0x044, 0x233, 0x000,
+ 0x307, 0x01b, 0x043, 0x2a7, 0x003, 0x016, 0x0c0, 0x01a,
+ 0x397, 0x044, 0x04c, 0x322, 0x01b, 0x00b, 0x00b, 0x272,
+ 0x04b, 0x001, 0x007, 0x101, 0x141, 0x01a, 0x000, 0x3f4,
+ 0x307, 0x01a, 0x305, 0x033, 0x055, 0x045, 0x247, 0x019,
+ 0x2c7, 0x01b, 0x020, 0x222, 0x000, 0x090, 0x020, 0x362,
+ 0x002, 0x327, 0x06f, 0x287, 0x384, 0x0ff, 0x262, 0x0df,
+ 0x00b, 0x00a, 0x262, 0x0e7, 0x003, 0x006, 0x247, 0x018,
+ 0x000, 0x00a, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x001,
+ 0x009, 0x057, 0x0c7, 0x019, 0x0d1, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0d0, 0x01b, 0x044, 0x222, 0x000,
+ 0x0c0, 0x019, 0x0c0, 0x01b, 0x347, 0x034, 0x01c, 0x044,
+ 0x222, 0x000, 0x0ca, 0x019, 0x081, 0x189, 0x247, 0x012,
+ 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x133, 0x347, 0x013,
+ 0x01b, 0x347, 0x013, 0x01c, 0x044, 0x222, 0x000, 0x132,
+ 0x007, 0x3f6, 0x0c0, 0x019, 0x0c0, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0cb, 0x019, 0x0cc, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c7, 0x019, 0x0c2, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0c8, 0x019, 0x0c0, 0x01b, 0x044,
+ 0x222, 0x000, 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b0, 0x020, 0x296,
+ 0x001, 0x0c1, 0x019, 0x0c0, 0x01b, 0x0c1, 0x01c, 0x1cc,
+ 0x01c, 0x044, 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x054,
+ 0x01c, 0x044, 0x222, 0x000, 0x345, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x01d, 0x102, 0x050, 0x282, 0x00f,
+ 0x08f, 0x307, 0x01d, 0x104, 0x1a0, 0x280, 0x051, 0x1a7,
+ 0x101, 0x050, 0x285, 0x300, 0x01d, 0x051, 0x1a7, 0x2c7,
+ 0x01d, 0x101, 0x051, 0x2c5, 0x01d, 0x101, 0x051, 0x1a7,
+ 0x2c7, 0x01e, 0x101, 0x051, 0x2c5, 0x01e, 0x347, 0x034,
+ 0x01c, 0x227, 0x044, 0x222, 0x000, 0x287, 0x044, 0x11b,
+ 0x002, 0x044, 0x11b, 0x002, 0x347, 0x01d, 0x01b, 0x347,
+ 0x034, 0x01c, 0x044, 0x222, 0x000, 0x327, 0x01e, 0x001,
+ 0x057, 0x0ca, 0x019, 0x101, 0x052, 0x01b, 0x101, 0x051,
+ 0x041, 0x1a9, 0x2c5, 0x01b, 0x043, 0x1b5, 0x2c7, 0x01c,
+ 0x101, 0x051, 0x1a3, 0x325, 0x037, 0x2c5, 0x01c, 0x044,
+ 0x222, 0x000, 0x153, 0x01e, 0x007, 0x3e7, 0x0c0, 0x019,
+ 0x0c0, 0x01c, 0x044, 0x222, 0x000, 0x0ca, 0x01b, 0x0c5,
+ 0x019, 0x090, 0x187, 0x395, 0x03a, 0x247, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x020, 0x01b, 0x0c0,
+ 0x01c, 0x044, 0x222, 0x000, 0x347, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x045, 0x101, 0x051, 0x1a7, 0x2c7, 0x01b,
+ 0x101, 0x051, 0x2c5, 0x01b, 0x101, 0x051, 0x325, 0x037,
+ 0x2c7, 0x01c, 0x044, 0x222, 0x000, 0x045, 0x0c5, 0x01b,
+ 0x000, 0x3cf, 0x347, 0x000, 0x022, 0x347, 0x001, 0x023,
+ 0x0c0, 0x008, 0x0c0, 0x009, 0x307, 0x022, 0x384, 0x03f,
+ 0x229, 0x220, 0x3b0, 0x14b, 0x287, 0x199, 0x390, 0x002,
+ 0x041, 0x040, 0x045, 0x020, 0x21a, 0x002, 0x020, 0x223,
+ 0x002, 0x020, 0x22c, 0x002, 0x020, 0x351, 0x002, 0x020,
+ 0x281, 0x002, 0x020, 0x304, 0x002, 0x020, 0x309, 0x002,
+ 0x020, 0x30e, 0x002, 0x020, 0x313, 0x002, 0x020, 0x318,
+ 0x002, 0x020, 0x31d, 0x002, 0x020, 0x322, 0x002, 0x020,
+ 0x327, 0x002, 0x020, 0x32c, 0x002, 0x020, 0x331, 0x002,
+ 0x020, 0x336, 0x002, 0x020, 0x339, 0x002, 0x020, 0x33f,
+ 0x002, 0x020, 0x344, 0x002, 0x020, 0x348, 0x002, 0x020,
+ 0x354, 0x002, 0x020, 0x358, 0x002, 0x020, 0x361, 0x002,
+ 0x020, 0x36a, 0x002, 0x020, 0x36d, 0x002, 0x020, 0x370,
+ 0x002, 0x020, 0x2f1, 0x002, 0x020, 0x373, 0x002, 0x020,
+ 0x384, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x38e, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x3fd, 0x001, 0x020, 0x101,
+ 0x001, 0x020, 0x197, 0x001, 0x020, 0x3ba, 0x000, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x0c1, 0x008, 0x345, 0x033, 0x008,
+ 0x344, 0x035, 0x008, 0x04e, 0x047, 0x3c7, 0x3ff, 0x008,
+ 0x000, 0x3f8, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7,
+ 0x009, 0x000, 0x3ea, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001,
+ 0x2c7, 0x009, 0x000, 0x3e1, 0x080, 0x3b7, 0x003, 0x1a9,
+ 0x3b5, 0x105, 0x2c7, 0x023, 0x0c0, 0x028, 0x0c0, 0x029,
+ 0x052, 0x022, 0x044, 0x2d6, 0x002, 0x001, 0x005, 0x101,
+ 0x000, 0x3f8, 0x101, 0x052, 0x022, 0x346, 0x028, 0x022,
+ 0x3c4, 0x0ff, 0x022, 0x362, 0x022, 0x0ef, 0x009, 0x02e,
+ 0x101, 0x052, 0x022, 0x327, 0x028, 0x1b7, 0x2c6, 0x022,
+ 0x362, 0x022, 0x0be, 0x009, 0x021, 0x101, 0x052, 0x022,
+ 0x346, 0x029, 0x022, 0x3c4, 0x0ff, 0x022, 0x362, 0x022,
+ 0x0ad, 0x009, 0x013, 0x101, 0x052, 0x022, 0x327, 0x029,
+ 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0de, 0x009, 0x006,
+ 0x0c1, 0x009, 0x000, 0x391, 0x3c7, 0x3ff, 0x009, 0x000,
+ 0x38c, 0x081, 0x189, 0x247, 0x012, 0x0c0, 0x028, 0x0c0,
+ 0x029, 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x2c7, 0x023,
+ 0x247, 0x012, 0x080, 0x347, 0x013, 0x022, 0x044, 0x2d6,
+ 0x002, 0x001, 0x005, 0x101, 0x000, 0x3f7, 0x347, 0x013,
+ 0x022, 0x346, 0x028, 0x022, 0x3c4, 0x0ff, 0x022, 0x362,
+ 0x022, 0x0ef, 0x009, 0x3d2, 0x347, 0x013, 0x022, 0x327,
+ 0x028, 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0be, 0x009,
+ 0x3c5, 0x347, 0x013, 0x022, 0x346, 0x029, 0x022, 0x3c4,
+ 0x0ff, 0x022, 0x362, 0x022, 0x0ad, 0x009, 0x3b7, 0x347,
+ 0x013, 0x022, 0x327, 0x029, 0x1b7, 0x2c6, 0x022, 0x362,
+ 0x022, 0x0de, 0x009, 0x3aa, 0x000, 0x3a4, 0x340, 0x022,
+ 0x028, 0x3c1, 0x000, 0x029, 0x3c6, 0x0d8, 0x028, 0x349,
+ 0x028, 0x028, 0x34b, 0x029, 0x029, 0x00a, 0x008, 0x3c0,
+ 0x0e5, 0x028, 0x3c1, 0x000, 0x029, 0x327, 0x023, 0x222,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x051,
+ 0x1a7, 0x2c7, 0x023, 0x101, 0x051, 0x2c5, 0x023, 0x111,
+ 0x240, 0x023, 0x000, 0x332, 0x347, 0x023, 0x026, 0x000,
+ 0x304, 0x347, 0x023, 0x027, 0x000, 0x2ff, 0x347, 0x023,
+ 0x028, 0x000, 0x2fa, 0x347, 0x023, 0x029, 0x000, 0x2f5,
+ 0x347, 0x023, 0x02a, 0x000, 0x2f0, 0x347, 0x023, 0x02b,
+ 0x000, 0x2eb, 0x347, 0x028, 0x009, 0x000, 0x2e6, 0x347,
+ 0x029, 0x009, 0x000, 0x2e1, 0x347, 0x02a, 0x009, 0x000,
+ 0x2dc, 0x347, 0x02b, 0x009, 0x000, 0x2d7, 0x347, 0x023,
+ 0x02d, 0x055, 0x347, 0x023, 0x02c, 0x000, 0x2ce, 0x347,
+ 0x023, 0x021, 0x000, 0x2c9, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x327, 0x023, 0x04d, 0x000,
+ 0x2bc, 0x347, 0x023, 0x021, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x04c, 0x2c7, 0x009, 0x000,
+ 0x2ac, 0x08c, 0x347, 0x023, 0x06f, 0x055, 0x247, 0x02c,
+ 0x000, 0x2ab, 0x08d, 0x000, 0x3f7, 0x08e, 0x000, 0x3f4,
+ 0x08f, 0x000, 0x3f1, 0x307, 0x022, 0x183, 0x19b, 0x262,
+ 0x009, 0x00b, 0x007, 0x347, 0x037, 0x008, 0x000, 0x292,
+ 0x390, 0x044, 0x000, 0x3ca, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x009, 0x000, 0x27f, 0x000, 0x27d,
+ 0x364, 0x004, 0x002, 0x009, 0x090, 0x364, 0x004, 0x008,
+ 0x009, 0x165, 0x364, 0x004, 0x004, 0x009, 0x08e, 0x0c0,
+ 0x007, 0x3d5, 0x080, 0x00e, 0x347, 0x006, 0x062, 0x0c0,
+ 0x064, 0x307, 0x062, 0x197, 0x384, 0x00f, 0x229, 0x220,
+ 0x3b0, 0x3ca, 0x287, 0x199, 0x390, 0x002, 0x041, 0x040,
+ 0x045, 0x345, 0x033, 0x064, 0x347, 0x064, 0x007, 0x344,
+ 0x035, 0x007, 0x3d4, 0x37f, 0x00e, 0x3c5, 0x020, 0x00e,
+ 0x04e, 0x047, 0x020, 0x3ff, 0x002, 0x020, 0x00a, 0x003,
+ 0x020, 0x011, 0x003, 0x020, 0x01a, 0x003, 0x020, 0x01d,
+ 0x003, 0x020, 0x020, 0x003, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002,
+ 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa,
+ 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x345, 0x037, 0x064, 0x000, 0x3bf, 0x3b7,
+ 0x010, 0x1a7, 0x3a5, 0x013, 0x324, 0x038, 0x2c7, 0x064,
+ 0x000, 0x3b1, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001, 0x000,
+ 0x3f5, 0x088, 0x055, 0x247, 0x02c, 0x347, 0x062, 0x06e,
+ 0x000, 0x3ad, 0x089, 0x000, 0x3f7, 0x08a, 0x000, 0x3f4,
+ 0x08b, 0x000, 0x3f1, 0x055, 0x0c3, 0x02c, 0x3d5, 0x100,
+ 0x00e, 0x000, 0x39c, 0x347, 0x011, 0x066, 0x3c4, 0x3fc,
+ 0x066, 0x364, 0x065, 0x040, 0x009, 0x0a3, 0x044, 0x085,
+ 0x003, 0x001, 0x06f, 0x044, 0x06b, 0x003, 0x001, 0x005,
+ 0x055, 0x000, 0x384, 0x044, 0x074, 0x003, 0x3c4, 0x37f,
+ 0x065, 0x1b5, 0x3a4, 0x038, 0x3c4, 0x3c7, 0x065, 0x2c5,
+ 0x065, 0x1b2, 0x2e2, 0x007, 0x001, 0x047, 0x2e2, 0x006,
+ 0x001, 0x043, 0x2e2, 0x002, 0x001, 0x047, 0x2e2, 0x001,
+ 0x001, 0x031, 0x3d7, 0x07f, 0x064, 0x0a6, 0x044, 0x0bf,
+ 0x003, 0x000, 0x35c, 0x364, 0x010, 0x002, 0x009, 0x005,
+ 0x364, 0x011, 0x001, 0x045, 0x3d5, 0x040, 0x00e, 0x3c5,
+ 0x001, 0x010, 0x3c4, 0x3bf, 0x065, 0x327, 0x066, 0x00b,
+ 0x005, 0x3c5, 0x040, 0x065, 0x045, 0x327, 0x004, 0x307,
+ 0x066, 0x198, 0x384, 0x038, 0x1b8, 0x3a4, 0x038, 0x222,
+ 0x045, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7, 0x064,
+ 0x0a7, 0x000, 0x3cd, 0x347, 0x066, 0x064, 0x044, 0x0cd,
+ 0x003, 0x000, 0x324, 0x055, 0x0c4, 0x02c, 0x000, 0x31f,
+ 0x044, 0x074, 0x003, 0x3c5, 0x080, 0x065, 0x3c4, 0x3f8,
+ 0x065, 0x3a4, 0x1fc, 0x081, 0x245, 0x065, 0x1b1, 0x390,
+ 0x066, 0x04d, 0x3c5, 0x100, 0x065, 0x000, 0x308, 0x3c4,
+ 0x07f, 0x064, 0x1c1, 0x064, 0x307, 0x066, 0x198, 0x384,
+ 0x038, 0x285, 0x188, 0x245, 0x064, 0x3c4, 0x003, 0x010,
+ 0x345, 0x064, 0x010, 0x3c5, 0x002, 0x010, 0x045, 0x044,
+ 0x085, 0x003, 0x009, 0x009, 0x307, 0x065, 0x364, 0x065,
+ 0x080, 0x009, 0x00c, 0x044, 0x06b, 0x003, 0x009, 0x35a,
+ 0x044, 0x074, 0x003, 0x000, 0x3b0, 0x044, 0x074, 0x003,
+ 0x384, 0x007, 0x262, 0x007, 0x001, 0x2d1, 0x141, 0x065,
+ 0x101, 0x324, 0x035, 0x000, 0x3bb, 0x3d5, 0x010, 0x00e,
+ 0x055, 0x0d3, 0x02c, 0x000, 0x2c2, 0x04e, 0x062, 0x0b1,
+ 0x0e7, 0x079
+
+}; /* end fm10000_sbus_master_code_prd */
+
+/* Spico SerDes code (production version 2)
+ * source file: serdes.0x2055/serdes.0x2055_0045.rom
+ * KR support
+ */
+const uint32_t fm10000_serdes_spico_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_spico_code_size_prd2 = 12167;
+
+const uint16_t fm10000_serdes_spico_code_prd2[] = {
+ 0x035, 0x006, 0x00a, 0x000, 0x1a7, 0x006, 0x020, 0x055,
+ 0x000, 0x045, 0x3c7, 0x03f, 0x000, 0x0c0, 0x200, 0x307,
+ 0x200, 0x304, 0x02b, 0x009, 0x00b, 0x0a0, 0x397, 0x3bb,
+ 0x04d, 0x111, 0x272, 0x1ff, 0x009, 0x3fc, 0x3c8, 0x000,
+ 0x21e, 0x3ce, 0x3ff, 0x21f, 0x348, 0x21f, 0x21d, 0x34e,
+ 0x21d, 0x221, 0x34e, 0x221, 0x21c, 0x34e, 0x21c, 0x220,
+ 0x3c7, 0x3fd, 0x23d, 0x3c7, 0x020, 0x23e, 0x3c7, 0x077,
+ 0x22d, 0x0c0, 0x202, 0x044, 0x36d, 0x003, 0x044, 0x360,
+ 0x002, 0x3d7, 0x303, 0x030, 0x347, 0x032, 0x236, 0x3b7,
+ 0x213, 0x090, 0x044, 0x1a9, 0x000, 0x3b7, 0x204, 0x387,
+ 0x07c, 0x044, 0x1a9, 0x000, 0x0a4, 0x397, 0x27c, 0x044,
+ 0x1a9, 0x000, 0x3d5, 0x230, 0x232, 0x345, 0x21d, 0x232,
+ 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000, 0x3b7,
+ 0x080, 0x044, 0x192, 0x000, 0x385, 0x00c, 0x384, 0x3cf,
+ 0x044, 0x1bd, 0x000, 0x3c5, 0x040, 0x021, 0x3b7, 0x020,
+ 0x387, 0x021, 0x189, 0x044, 0x1a9, 0x000, 0x0c0, 0x008,
+ 0x0c0, 0x005, 0x0c0, 0x00b, 0x0c0, 0x000, 0x3c5, 0x020,
+ 0x027, 0x055, 0x307, 0x221, 0x304, 0x233, 0x009, 0x012,
+ 0x374, 0x027, 0x200, 0x001, 0x00d, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x009, 0x006, 0x3b8, 0x200, 0x2c4, 0x027,
+ 0x054, 0x364, 0x200, 0x004, 0x029, 0x3de, 0x003, 0x364,
+ 0x33b, 0x010, 0x029, 0x3de, 0x003, 0x054, 0x364, 0x200,
+ 0x040, 0x029, 0x034, 0x008, 0x364, 0x2cf, 0x040, 0x009,
+ 0x01c, 0x307, 0x026, 0x384, 0x003, 0x266, 0x003, 0x009,
+ 0x014, 0x362, 0x26b, 0x0e0, 0x009, 0x00f, 0x364, 0x234,
+ 0x002, 0x001, 0x00a, 0x3c5, 0x060, 0x2cf, 0x0c0, 0x26b,
+ 0x3c5, 0x040, 0x200, 0x054, 0x364, 0x200, 0x100, 0x029,
+ 0x22f, 0x003, 0x054, 0x044, 0x141, 0x001, 0x055, 0x364,
+ 0x23f, 0x003, 0x001, 0x012, 0x044, 0x34c, 0x001, 0x029,
+ 0x0af, 0x006, 0x044, 0x340, 0x001, 0x227, 0x326, 0x249,
+ 0x1ba, 0x029, 0x0af, 0x006, 0x3c4, 0x3df, 0x000, 0x364,
+ 0x33c, 0x020, 0x001, 0x017, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x055, 0x364, 0x200, 0x080, 0x001, 0x00d, 0x3c4,
+ 0x3fd, 0x26f, 0x3c4, 0x3fe, 0x200, 0x366, 0x200, 0x080,
+ 0x001, 0x00f, 0x140, 0x200, 0x009, 0x365, 0x364, 0x23f,
+ 0x003, 0x009, 0x360, 0x04f, 0x04e, 0x000, 0x35c, 0x3c5,
+ 0x030, 0x000, 0x347, 0x244, 0x054, 0x374, 0x000, 0x2c0,
+ 0x009, 0x34c, 0x387, 0x0c1, 0x187, 0x245, 0x054, 0x04f,
+ 0x055, 0x141, 0x26c, 0x347, 0x245, 0x200, 0x347, 0x245,
+ 0x26d, 0x0c0, 0x245, 0x364, 0x026, 0x002, 0x001, 0x336,
+ 0x364, 0x020, 0x005, 0x009, 0x007, 0x0c2, 0x26f, 0x044,
+ 0x1b8, 0x001, 0x055, 0x364, 0x020, 0x005, 0x009, 0x326,
+ 0x362, 0x26b, 0x0e0, 0x009, 0x321, 0x364, 0x234, 0x002,
+ 0x001, 0x31c, 0x3c5, 0x060, 0x2cf, 0x3c5, 0x040, 0x200,
+ 0x000, 0x314, 0x044, 0x290, 0x001, 0x3c5, 0x080, 0x200,
+ 0x347, 0x2cf, 0x26b, 0x3c4, 0x0e0, 0x26b, 0x3c4, 0x3bf,
+ 0x2cf, 0x045, 0x387, 0x080, 0x020, 0x2f8, 0x001, 0x387,
+ 0x02f, 0x111, 0x009, 0x3ff, 0x045, 0x054, 0x111, 0x009,
+ 0x3fe, 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x032, 0x054,
+ 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x033, 0x054, 0x045,
+ 0x055, 0x2c7, 0x030, 0x364, 0x035, 0x001, 0x009, 0x3fd,
+ 0x045, 0x044, 0x1a0, 0x000, 0x305, 0x033, 0x000, 0x012,
+ 0x044, 0x1a0, 0x000, 0x306, 0x033, 0x000, 0x00b, 0x208,
+ 0x044, 0x1a0, 0x000, 0x304, 0x033, 0x055, 0x2c7, 0x030,
+ 0x247, 0x031, 0x345, 0x21d, 0x030, 0x140, 0x035, 0x009,
+ 0x3fe, 0x346, 0x21d, 0x030, 0x054, 0x045, 0x040, 0x140,
+ 0x036, 0x00b, 0x020, 0x044, 0x206, 0x000, 0x344, 0x21f,
+ 0x036, 0x3c4, 0x3ef, 0x036, 0x09f, 0x044, 0x189, 0x000,
+ 0x000, 0x010, 0x040, 0x140, 0x036, 0x003, 0x00c, 0x044,
+ 0x206, 0x000, 0x345, 0x21d, 0x036, 0x3c4, 0x3ef, 0x036,
+ 0x054, 0x327, 0x22b, 0x3a4, 0x003, 0x133, 0x009, 0x00e,
+ 0x0a1, 0x1aa, 0x2a8, 0x2c4, 0x021, 0x08f, 0x044, 0x189,
+ 0x000, 0x2a8, 0x2c5, 0x021, 0x042, 0x045, 0x055, 0x088,
+ 0x3c5, 0x010, 0x036, 0x000, 0x37e, 0x3c4, 0x3fe, 0x026,
+ 0x327, 0x22c, 0x3a4, 0x007, 0x2e6, 0x001, 0x001, 0x01b,
+ 0x044, 0x1e2, 0x000, 0x364, 0x234, 0x001, 0x009, 0x013,
+ 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x3b6, 0x003,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x066, 0x004,
+ 0x054, 0x045, 0x364, 0x233, 0x002, 0x009, 0x00d, 0x347,
+ 0x271, 0x270, 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x044,
+ 0x02b, 0x008, 0x374, 0x232, 0x200, 0x001, 0x0a5, 0x3d7,
+ 0x200, 0x27b, 0x044, 0x29c, 0x001, 0x044, 0x2f6, 0x002,
+ 0x009, 0x005, 0x044, 0x028, 0x008, 0x3b7, 0x221, 0x364,
+ 0x22b, 0x003, 0x009, 0x007, 0x083, 0x18b, 0x044, 0x1a9,
+ 0x000, 0x398, 0x200, 0x244, 0x232, 0x3b7, 0x221, 0x091,
+ 0x189, 0x385, 0x128, 0x044, 0x1a9, 0x000, 0x083, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x220, 0x044, 0x192, 0x000, 0x197,
+ 0x264, 0x007, 0x009, 0x005, 0x091, 0x000, 0x003, 0x081,
+ 0x3b7, 0x221, 0x189, 0x044, 0x1b7, 0x000, 0x364, 0x246,
+ 0x010, 0x009, 0x002, 0x3b7, 0x221, 0x081, 0x18d, 0x044,
+ 0x1a9, 0x000, 0x081, 0x189, 0x044, 0x1a9, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x364, 0x22b, 0x003, 0x009,
+ 0x009, 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000,
+ 0x364, 0x02b, 0x020, 0x009, 0x020, 0x0a3, 0x364, 0x22b,
+ 0x003, 0x001, 0x006, 0x044, 0x025, 0x008, 0x0a0, 0x307,
+ 0x251, 0x197, 0x384, 0x00f, 0x220, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x327, 0x251, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x044, 0x1ce, 0x000, 0x364, 0x232,
+ 0x100, 0x001, 0x005, 0x044, 0x0a9, 0x001, 0x3c4, 0x3f0,
+ 0x22d, 0x3c5, 0x001, 0x026, 0x364, 0x026, 0x002, 0x001,
+ 0x1c5, 0x044, 0x283, 0x001, 0x3b7, 0x300, 0x387, 0x020,
+ 0x000, 0x2c1, 0x307, 0x036, 0x00b, 0x3ea, 0x044, 0x1ce,
+ 0x000, 0x000, 0x3e5, 0x387, 0x3ff, 0x327, 0x22c, 0x3a4,
+ 0x007, 0x131, 0x001, 0x01a, 0x3d5, 0x200, 0x232, 0x000,
+ 0x006, 0x080, 0x131, 0x001, 0x011, 0x3b7, 0x205, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x222, 0x044, 0x1bd, 0x000, 0x3b7,
+ 0x251, 0x044, 0x1bd, 0x000, 0x3b7, 0x214, 0x100, 0x009,
+ 0x00a, 0x044, 0x1bd, 0x000, 0x3c4, 0x37b, 0x021, 0x000,
+ 0x29e, 0x3c5, 0x084, 0x021, 0x000, 0x299, 0x364, 0x233,
+ 0x002, 0x009, 0x00d, 0x347, 0x272, 0x270, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x02e, 0x008, 0x307, 0x22d,
+ 0x384, 0x070, 0x386, 0x010, 0x001, 0x04a, 0x364, 0x02b,
+ 0x020, 0x009, 0x045, 0x3b7, 0x071, 0x387, 0x128, 0x044,
+ 0x1a9, 0x000, 0x387, 0x368, 0x044, 0x1b8, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x0a1, 0x307, 0x22b, 0x384,
+ 0x030, 0x001, 0x003, 0x0ac, 0x307, 0x256, 0x384, 0x01f,
+ 0x220, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x080, 0x327,
+ 0x22b, 0x3a4, 0x030, 0x3a2, 0x030, 0x009, 0x00e, 0x0a1,
+ 0x1ab, 0x2a8, 0x2c4, 0x024, 0x08f, 0x044, 0x189, 0x000,
+ 0x2a8, 0x2c5, 0x024, 0x327, 0x256, 0x1b7, 0x3a4, 0x07f,
+ 0x220, 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x044, 0x2f6,
+ 0x002, 0x009, 0x007, 0x0c0, 0x27b, 0x044, 0x29c, 0x001,
+ 0x044, 0x0e6, 0x001, 0x3c4, 0x30f, 0x22d, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x031, 0x008, 0x345, 0x21d,
+ 0x01c, 0x307, 0x21d, 0x304, 0x233, 0x001, 0x007, 0x3a7,
+ 0x071, 0x044, 0x1a9, 0x000, 0x3c5, 0x002, 0x026, 0x044,
+ 0x141, 0x001, 0x020, 0x283, 0x001, 0x055, 0x3c4, 0x3ef,
+ 0x027, 0x3c4, 0x3fd, 0x026, 0x364, 0x26f, 0x002, 0x001,
+ 0x008, 0x3c4, 0x3fd, 0x26f, 0x044, 0x290, 0x001, 0x344,
+ 0x21f, 0x01c, 0x3c4, 0x3bf, 0x200, 0x3c4, 0x39f, 0x2cf,
+ 0x0c0, 0x2d8, 0x0c0, 0x26b, 0x364, 0x234, 0x002, 0x009,
+ 0x012, 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x066,
+ 0x004, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x3b6,
+ 0x003, 0x054, 0x0a0, 0x081, 0x189, 0x020, 0x1a9, 0x000,
+ 0x044, 0x3fd, 0x000, 0x387, 0x3ff, 0x044, 0x02b, 0x001,
+ 0x3a7, 0x022, 0x020, 0x1bd, 0x000, 0x0a0, 0x081, 0x189,
+ 0x020, 0x1a9, 0x000, 0x080, 0x044, 0x02b, 0x001, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x364, 0x024,
+ 0x002, 0x001, 0x006, 0x399, 0x305, 0x000, 0x004, 0x397,
+ 0x200, 0x044, 0x322, 0x001, 0x3a7, 0x022, 0x044, 0x1bd,
+ 0x000, 0x364, 0x232, 0x002, 0x001, 0x005, 0x044, 0x382,
+ 0x001, 0x000, 0x0c9, 0x3a7, 0x061, 0x044, 0x1bd, 0x000,
+ 0x0a5, 0x044, 0x1bd, 0x000, 0x3a7, 0x072, 0x044, 0x1bd,
+ 0x000, 0x3a7, 0x0f1, 0x044, 0x1bd, 0x000, 0x385, 0x004,
+ 0x0b1, 0x020, 0x1bd, 0x000, 0x364, 0x22c, 0x002, 0x009,
+ 0x2ac, 0x364, 0x234, 0x001, 0x001, 0x058, 0x020, 0x232,
+ 0x000, 0x3c4, 0x3fe, 0x027, 0x364, 0x234, 0x002, 0x001,
+ 0x04d, 0x000, 0x2cd, 0x3b7, 0x211, 0x044, 0x199, 0x000,
+ 0x384, 0x00f, 0x140, 0x239, 0x00b, 0x004, 0x385, 0x010,
+ 0x140, 0x237, 0x00b, 0x004, 0x385, 0x020, 0x044, 0x1bd,
+ 0x000, 0x307, 0x239, 0x00b, 0x004, 0x208, 0x101, 0x384,
+ 0x01f, 0x183, 0x327, 0x237, 0x00b, 0x004, 0x2a8, 0x121,
+ 0x3a4, 0x00f, 0x285, 0x184, 0x3c4, 0x01f, 0x238, 0x362,
+ 0x238, 0x017, 0x00b, 0x005, 0x0a1, 0x1ad, 0x285, 0x305,
+ 0x238, 0x32a, 0x238, 0x3a4, 0x008, 0x285, 0x3b7, 0x210,
+ 0x044, 0x1bd, 0x000, 0x347, 0x237, 0x23a, 0x347, 0x238,
+ 0x23c, 0x347, 0x239, 0x23b, 0x045, 0x345, 0x21c, 0x021,
+ 0x045, 0x308, 0x21c, 0x244, 0x021, 0x0c0, 0x240, 0x307,
+ 0x022, 0x384, 0x001, 0x245, 0x240, 0x142, 0x240, 0x364,
+ 0x240, 0x100, 0x009, 0x3eb, 0x0a1, 0x044, 0x0d9, 0x001,
+ 0x307, 0x240, 0x327, 0x022, 0x286, 0x384, 0x001, 0x001,
+ 0x3ee, 0x0a2, 0x307, 0x22b, 0x384, 0x003, 0x001, 0x008,
+ 0x0aa, 0x262, 0x003, 0x001, 0x003, 0x0a5, 0x289, 0x240,
+ 0x240, 0x387, 0x084, 0x020, 0x2f8, 0x001, 0x094, 0x020,
+ 0x2f2, 0x001, 0x090, 0x020, 0x2f2, 0x001, 0x0a0, 0x082,
+ 0x18b, 0x304, 0x020, 0x009, 0x3b9, 0x081, 0x189, 0x020,
+ 0x1b7, 0x000, 0x0a1, 0x1a9, 0x364, 0x234, 0x002, 0x001,
+ 0x018, 0x364, 0x22c, 0x020, 0x001, 0x016, 0x364, 0x22c,
+ 0x010, 0x001, 0x009, 0x364, 0x232, 0x080, 0x001, 0x00c,
+ 0x000, 0x007, 0x364, 0x232, 0x040, 0x001, 0x005, 0x2c5,
+ 0x024, 0x045, 0x309, 0x024, 0x003, 0x3fb, 0x2a8, 0x2c4,
+ 0x024, 0x045, 0x3b7, 0x213, 0x364, 0x234, 0x004, 0x009,
+ 0x017, 0x307, 0x233, 0x182, 0x00a, 0x007, 0x3c5, 0x040,
+ 0x021, 0x000, 0x005, 0x3c4, 0x3bf, 0x021, 0x082, 0x044,
+ 0x1b7, 0x000, 0x088, 0x020, 0x1a9, 0x000, 0x082, 0x044,
+ 0x1a9, 0x000, 0x3c4, 0x3bf, 0x021, 0x088, 0x020, 0x1b7,
+ 0x000, 0x055, 0x364, 0x026, 0x002, 0x001, 0x031, 0x364,
+ 0x024, 0x002, 0x009, 0x007, 0x364, 0x025, 0x004, 0x009,
+ 0x027, 0x309, 0x01c, 0x003, 0x004, 0x00a, 0x021, 0x364,
+ 0x2cf, 0x011, 0x009, 0x01c, 0x364, 0x33b, 0x007, 0x001,
+ 0x007, 0x364, 0x33b, 0x002, 0x001, 0x012, 0x3c5, 0x010,
+ 0x027, 0x364, 0x234, 0x040, 0x001, 0x005, 0x3c4, 0x37f,
+ 0x024, 0x3c5, 0x008, 0x025, 0x000, 0x010, 0x364, 0x234,
+ 0x040, 0x001, 0x005, 0x3c5, 0x080, 0x024, 0x3c4, 0x3ef,
+ 0x027, 0x3c5, 0x010, 0x026, 0x054, 0x045, 0x364, 0x026,
+ 0x001, 0x001, 0x008, 0x3c5, 0x004, 0x232, 0x044, 0x20d,
+ 0x000, 0x09f, 0x18a, 0x208, 0x244, 0x243, 0x244, 0x250,
+ 0x327, 0x243, 0x326, 0x250, 0x001, 0x308, 0x347, 0x243,
+ 0x250, 0x087, 0x187, 0x208, 0x244, 0x020, 0x364, 0x026,
+ 0x002, 0x001, 0x2fb, 0x3c5, 0x008, 0x232, 0x020, 0x3b5,
+ 0x000, 0x364, 0x233, 0x008, 0x001, 0x004, 0x307, 0x220,
+ 0x3c5, 0x001, 0x00c, 0x247, 0x01b, 0x083, 0x18d, 0x244,
+ 0x01c, 0x044, 0x1c9, 0x001, 0x0a3, 0x1a8, 0x2c5, 0x01f,
+ 0x045, 0x3c4, 0x0ff, 0x01f, 0x085, 0x044, 0x189, 0x000,
+ 0x3d5, 0x100, 0x01f, 0x3b7, 0x300, 0x397, 0x200, 0x044,
+ 0x1a9, 0x000, 0x3c7, 0x07f, 0x252, 0x055, 0x151, 0x252,
+ 0x001, 0x006, 0x140, 0x01e, 0x00b, 0x3fa, 0x397, 0x200,
+ 0x044, 0x1b7, 0x000, 0x3c4, 0x2ff, 0x01f, 0x045, 0x327,
+ 0x22c, 0x307, 0x22d, 0x2e4, 0x004, 0x009, 0x00d, 0x286,
+ 0x384, 0x007, 0x001, 0x00f, 0x2e4, 0x007, 0x001, 0x032,
+ 0x000, 0x039, 0x384, 0x007, 0x3c4, 0x3f8, 0x22c, 0x245,
+ 0x22c, 0x327, 0x22c, 0x307, 0x22d, 0x2e4, 0x040, 0x009,
+ 0x016, 0x286, 0x264, 0x070, 0x001, 0x018, 0x3a4, 0x070,
+ 0x001, 0x029, 0x1aa, 0x002, 0x02e, 0x044, 0x3b5, 0x000,
+ 0x044, 0x3fd, 0x000, 0x000, 0x009, 0x384, 0x070, 0x3c4,
+ 0x38f, 0x22c, 0x245, 0x22c, 0x347, 0x22c, 0x22d, 0x045,
+ 0x227, 0x044, 0x301, 0x000, 0x044, 0x232, 0x000, 0x000,
+ 0x3d2, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x000,
+ 0x3ca, 0x044, 0x003, 0x001, 0x044, 0x326, 0x000, 0x000,
+ 0x3e5, 0x044, 0x0f2, 0x001, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x000, 0x3da, 0x3a7, 0x080, 0x384, 0x001,
+ 0x001, 0x011, 0x082, 0x044, 0x1a9, 0x000, 0x081, 0x044,
+ 0x1b7, 0x000, 0x3c5, 0x004, 0x024, 0x3c5, 0x008, 0x025,
+ 0x045, 0x366, 0x22e, 0x020, 0x001, 0x010, 0x082, 0x044,
+ 0x1b7, 0x000, 0x081, 0x044, 0x1a9, 0x000, 0x3c4, 0x3fb,
+ 0x024, 0x3c4, 0x3f7, 0x025, 0x045, 0x308, 0x248, 0x19e,
+ 0x020, 0x254, 0x001, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x020, 0x1b6, 0x001,
+ 0x3c5, 0x001, 0x00c, 0x0c0, 0x01b, 0x044, 0x1c9, 0x001,
+ 0x3c4, 0x3fe, 0x00c, 0x045, 0x0a1, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x247, 0x24f, 0x397, 0x3e0, 0x044, 0x1a9,
+ 0x000, 0x0a0, 0x325, 0x27b, 0x387, 0x140, 0x044, 0x1a9,
+ 0x000, 0x044, 0x187, 0x000, 0x084, 0x044, 0x1a9, 0x000,
+ 0x044, 0x187, 0x000, 0x387, 0x040, 0x044, 0x1b7, 0x000,
+ 0x387, 0x106, 0x044, 0x189, 0x000, 0x387, 0x100, 0x044,
+ 0x1b7, 0x000, 0x387, 0x099, 0x044, 0x189, 0x000, 0x0a1,
+ 0x325, 0x27b, 0x307, 0x24f, 0x044, 0x1bd, 0x000, 0x0a0,
+ 0x325, 0x27b, 0x084, 0x044, 0x1b7, 0x000, 0x387, 0x099,
+ 0x044, 0x2f6, 0x002, 0x009, 0x004, 0x399, 0x3ff, 0x020,
+ 0x189, 0x000, 0x041, 0x327, 0x236, 0x3a4, 0x00c, 0x134,
+ 0x043, 0x045, 0x055, 0x3d7, 0x071, 0x030, 0x000, 0x006,
+ 0x055, 0x3d7, 0x221, 0x030, 0x347, 0x034, 0x258, 0x0c0,
+ 0x034, 0x140, 0x035, 0x009, 0x3fe, 0x347, 0x032, 0x031,
+ 0x345, 0x21d, 0x030, 0x246, 0x031, 0x374, 0x030, 0x200,
+ 0x001, 0x007, 0x246, 0x031, 0x044, 0x18c, 0x000, 0x131,
+ 0x009, 0x3f3, 0x346, 0x21d, 0x030, 0x347, 0x258, 0x034,
+ 0x054, 0x045, 0x364, 0x233, 0x040, 0x001, 0x006, 0x3b7,
+ 0x200, 0x2a8, 0x284, 0x045, 0x227, 0x055, 0x040, 0x397,
+ 0x203, 0x300, 0x202, 0x04d, 0x141, 0x202, 0x362, 0x202,
+ 0x018, 0x009, 0x004, 0x0c0, 0x202, 0x042, 0x054, 0x045,
+ 0x055, 0x3c7, 0x0f2, 0x030, 0x364, 0x035, 0x001, 0x009,
+ 0x3fd, 0x307, 0x032, 0x045, 0x0a1, 0x1aa, 0x324, 0x233,
+ 0x045, 0x327, 0x222, 0x3a4, 0x07f, 0x001, 0x02c, 0x055,
+ 0x307, 0x034, 0x0c0, 0x034, 0x040, 0x140, 0x035, 0x009,
+ 0x3fe, 0x3c7, 0x071, 0x030, 0x387, 0x0d7, 0x347, 0x032,
+ 0x031, 0x345, 0x21d, 0x030, 0x141, 0x23d, 0x009, 0x007,
+ 0x153, 0x23d, 0x346, 0x21c, 0x031, 0x246, 0x031, 0x131,
+ 0x009, 0x3f4, 0x346, 0x21d, 0x030, 0x042, 0x247, 0x034,
+ 0x054, 0x045, 0x083, 0x18d, 0x345, 0x21d, 0x233, 0x364,
+ 0x2cf, 0x004, 0x001, 0x005, 0x245, 0x305, 0x045, 0x3a7,
+ 0x071, 0x020, 0x1a9, 0x000, 0x083, 0x18d, 0x364, 0x2cf,
+ 0x004, 0x001, 0x006, 0x208, 0x244, 0x305, 0x045, 0x328,
+ 0x21d, 0x2c4, 0x233, 0x3a7, 0x071, 0x020, 0x1b7, 0x000,
+ 0x120, 0x001, 0x027, 0x041, 0x2f4, 0x200, 0x001, 0x00a,
+ 0x3a7, 0x073, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x009,
+ 0x3a7, 0x073, 0x387, 0x3fd, 0x044, 0x1b8, 0x000, 0x043,
+ 0x2e4, 0x180, 0x001, 0x016, 0x2e4, 0x080, 0x009, 0x017,
+ 0x3c5, 0x002, 0x232, 0x044, 0x382, 0x001, 0x000, 0x012,
+ 0x3c4, 0x3fd, 0x232, 0x044, 0x394, 0x001, 0x000, 0x00a,
+ 0x044, 0x353, 0x001, 0x000, 0x005, 0x081, 0x043, 0x040,
+ 0x020, 0x181, 0x006, 0x3c4, 0x3fe, 0x03a, 0x247, 0x25e,
+ 0x0c5, 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263,
+ 0x3c4, 0x3bf, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03a,
+ 0x3c5, 0x040, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042,
+ 0x100, 0x00b, 0x008, 0x247, 0x03c, 0x042, 0x247, 0x03b,
+ 0x042, 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03a,
+ 0x041, 0x307, 0x25d, 0x040, 0x152, 0x03a, 0x345, 0x263,
+ 0x03a, 0x045, 0x3c4, 0x3fe, 0x03d, 0x247, 0x25f, 0x0c5,
+ 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263, 0x3c4,
+ 0x37f, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03d, 0x3c5,
+ 0x080, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042, 0x100,
+ 0x00b, 0x008, 0x247, 0x03f, 0x042, 0x247, 0x03e, 0x042,
+ 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03d, 0x041,
+ 0x307, 0x25d, 0x040, 0x152, 0x03d, 0x345, 0x263, 0x03d,
+ 0x045, 0x3c4, 0x3fe, 0x040, 0x0c1, 0x041, 0x3c4, 0x3cf,
+ 0x00c, 0x151, 0x041, 0x0c0, 0x263, 0x247, 0x262, 0x0c0,
+ 0x040, 0x384, 0x003, 0x227, 0x1a0, 0x285, 0x384, 0x3fd,
+ 0x187, 0x245, 0x040, 0x245, 0x041, 0x245, 0x263, 0x3c5,
+ 0x030, 0x00c, 0x3c7, 0x3ff, 0x039, 0x142, 0x040, 0x152,
+ 0x040, 0x144, 0x040, 0x364, 0x262, 0x100, 0x001, 0x004,
+ 0x148, 0x040, 0x042, 0x043, 0x247, 0x261, 0x2c7, 0x260,
+ 0x044, 0x0de, 0x002, 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x04a,
+ 0x307, 0x262, 0x384, 0x030, 0x187, 0x245, 0x04a, 0x0a1,
+ 0x307, 0x262, 0x384, 0x030, 0x193, 0x262, 0x000, 0x001,
+ 0x008, 0x262, 0x001, 0x001, 0x007, 0x000, 0x006, 0x0a0,
+ 0x000, 0x003, 0x0a2, 0x1ad, 0x2c5, 0x263, 0x0a3, 0x307,
+ 0x262, 0x384, 0x00c, 0x191, 0x262, 0x001, 0x001, 0x008,
+ 0x262, 0x002, 0x001, 0x007, 0x000, 0x006, 0x0a1, 0x000,
+ 0x003, 0x0a2, 0x1a5, 0x2c5, 0x263, 0x374, 0x262, 0x200,
+ 0x001, 0x00e, 0x307, 0x261, 0x327, 0x260, 0x041, 0x040,
+ 0x347, 0x263, 0x041, 0x141, 0x040, 0x045, 0x347, 0x263,
+ 0x041, 0x141, 0x040, 0x020, 0x089, 0x000, 0x042, 0x043,
+ 0x247, 0x25c, 0x042, 0x264, 0x3ff, 0x00b, 0x008, 0x247,
+ 0x043, 0x042, 0x247, 0x042, 0x042, 0x247, 0x039, 0x042,
+ 0x247, 0x038, 0x142, 0x040, 0x152, 0x040, 0x307, 0x25c,
+ 0x041, 0x040, 0x045, 0x055, 0x140, 0x259, 0x001, 0x009,
+ 0x322, 0x259, 0x001, 0x007, 0x0a1, 0x054, 0x045, 0x2c7,
+ 0x259, 0x264, 0x008, 0x009, 0x06e, 0x0c0, 0x012, 0x043,
+ 0x2c7, 0x25a, 0x043, 0x2c7, 0x25b, 0x227, 0x274, 0x200,
+ 0x009, 0x022, 0x3a4, 0x006, 0x2c7, 0x00d, 0x227, 0x3c4,
+ 0x3fd, 0x00c, 0x3c4, 0x3fc, 0x018, 0x384, 0x001, 0x245,
+ 0x018, 0x142, 0x00c, 0x042, 0x247, 0x014, 0x042, 0x247,
+ 0x013, 0x120, 0x00b, 0x008, 0x042, 0x247, 0x011, 0x042,
+ 0x247, 0x010, 0x3c7, 0x050, 0x26e, 0x142, 0x012, 0x141,
+ 0x00d, 0x080, 0x2e4, 0x100, 0x001, 0x003, 0x108, 0x151,
+ 0x26e, 0x001, 0x007, 0x364, 0x012, 0x040, 0x009, 0x3f9,
+ 0x152, 0x012, 0x151, 0x00d, 0x2e4, 0x080, 0x001, 0x003,
+ 0x104, 0x245, 0x012, 0x307, 0x25b, 0x040, 0x307, 0x25a,
+ 0x040, 0x2a9, 0x003, 0x017, 0x151, 0x26e, 0x003, 0x007,
+ 0x364, 0x012, 0x080, 0x009, 0x3f9, 0x364, 0x026, 0x002,
+ 0x001, 0x007, 0x364, 0x025, 0x004, 0x009, 0x007, 0x141,
+ 0x012, 0x0a0, 0x054, 0x045, 0x364, 0x2cf, 0x007, 0x001,
+ 0x3f8, 0x3c4, 0x3fc, 0x2cf, 0x3d5, 0x200, 0x2cf, 0x020,
+ 0x33a, 0x002, 0x247, 0x017, 0x287, 0x00b, 0x006, 0x3a7,
+ 0x3ff, 0x000, 0x003, 0x1bb, 0x384, 0x0ff, 0x264, 0x080,
+ 0x001, 0x004, 0x385, 0x300, 0x380, 0x011, 0x00b, 0x018,
+ 0x041, 0x0a0, 0x131, 0x380, 0x014, 0x003, 0x3fd, 0x041,
+ 0x0bf, 0x1a6, 0x2a8, 0x2c4, 0x018, 0x186, 0x245, 0x018,
+ 0x083, 0x18d, 0x245, 0x017, 0x042, 0x043, 0x103, 0x121,
+ 0x18a, 0x1a7, 0x285, 0x247, 0x01a, 0x387, 0x3ff, 0x247,
+ 0x019, 0x18b, 0x245, 0x018, 0x307, 0x017, 0x19d, 0x262,
+ 0x003, 0x001, 0x00e, 0x262, 0x001, 0x001, 0x013, 0x262,
+ 0x002, 0x009, 0x037, 0x3a7, 0x0aa, 0x000, 0x00d, 0x0c1,
+ 0x019, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x000, 0x039,
+ 0x3a7, 0x055, 0x041, 0x3a7, 0x071, 0x044, 0x192, 0x000,
+ 0x00b, 0x010, 0x040, 0x3a7, 0x0f2, 0x044, 0x192, 0x000,
+ 0x198, 0x043, 0x3a4, 0x001, 0x009, 0x004, 0x386, 0x001,
+ 0x043, 0x264, 0x001, 0x001, 0x004, 0x3a6, 0x0ff, 0x287,
+ 0x187, 0x285, 0x247, 0x019, 0x18b, 0x103, 0x244, 0x018,
+ 0x364, 0x24d, 0x030, 0x009, 0x00c, 0x08f, 0x18b, 0x208,
+ 0x244, 0x018, 0x397, 0x1fe, 0x182, 0x244, 0x019, 0x327,
+ 0x017, 0x1ba, 0x3a4, 0x007, 0x001, 0x019, 0x2e2, 0x003,
+ 0x001, 0x015, 0x041, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x043, 0x384, 0x008, 0x001, 0x00a, 0x2e2, 0x003, 0x003,
+ 0x004, 0x3a6, 0x002, 0x3a6, 0x001, 0x387, 0x002, 0x189,
+ 0x395, 0x23f, 0x280, 0x050, 0x245, 0x01a, 0x045, 0x0ff,
+ 0x00c, 0x030, 0x03c, 0x00f, 0x0f0, 0x004, 0x020, 0x307,
+ 0x26f, 0x384, 0x003, 0x229, 0x220, 0x3b0, 0x04b, 0x287,
+ 0x199, 0x390, 0x008, 0x041, 0x040, 0x045, 0x055, 0x329,
+ 0x01f, 0x00b, 0x08b, 0x364, 0x234, 0x080, 0x001, 0x00e,
+ 0x364, 0x024, 0x002, 0x009, 0x009, 0x364, 0x025, 0x004,
+ 0x001, 0x004, 0x000, 0x012, 0x327, 0x01d, 0x009, 0x00a,
+ 0x081, 0x18d, 0x245, 0x020, 0x327, 0x01d, 0x246, 0x020,
+ 0x3a2, 0x020, 0x003, 0x02b, 0x309, 0x01c, 0x003, 0x022,
+ 0x3c4, 0x3ef, 0x027, 0x3c5, 0x010, 0x026, 0x0a0, 0x387,
+ 0x3df, 0x044, 0x1b8, 0x000, 0x081, 0x189, 0x044, 0x1a9,
+ 0x000, 0x140, 0x01c, 0x00b, 0x00d, 0x0a1, 0x044, 0x192,
+ 0x000, 0x247, 0x27f, 0x395, 0x3ff, 0x044, 0x1bd, 0x000,
+ 0x344, 0x21f, 0x01c, 0x000, 0x02a, 0x307, 0x01c, 0x003,
+ 0x026, 0x345, 0x21d, 0x01c, 0x209, 0x003, 0x020, 0x307,
+ 0x020, 0x181, 0x003, 0x01b, 0x364, 0x025, 0x004, 0x009,
+ 0x005, 0x3c5, 0x008, 0x025, 0x0a0, 0x081, 0x189, 0x044,
+ 0x1b7, 0x000, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1,
+ 0x307, 0x27f, 0x044, 0x1bd, 0x000, 0x055, 0x364, 0x234,
+ 0x002, 0x001, 0x00f, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x044, 0x1b6, 0x001,
+ 0x054, 0x020, 0x0a3, 0x006, 0x346, 0x21d, 0x232, 0x02b,
+ 0x181, 0x006, 0x364, 0x233, 0x001, 0x021, 0x08d, 0x006,
+ 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4, 0x000, 0x327, 0x21d,
+ 0x324, 0x020, 0x045, 0x307, 0x28d, 0x384, 0x07f, 0x22a,
+ 0x286, 0x040, 0x0b0, 0x044, 0x192, 0x000, 0x384, 0x380,
+ 0x043, 0x285, 0x0b0, 0x020, 0x1bd, 0x000, 0x0a2, 0x141,
+ 0x041, 0x141, 0x2f1, 0x397, 0x200, 0x151, 0x041, 0x044,
+ 0x0fb, 0x002, 0x055, 0x364, 0x200, 0x080, 0x009, 0x00e,
+ 0x364, 0x234, 0x002, 0x001, 0x009, 0x054, 0x364, 0x012,
+ 0x040, 0x001, 0x3f1, 0x045, 0x364, 0x2cf, 0x007, 0x009,
+ 0x3fc, 0x044, 0x33f, 0x002, 0x3c4, 0x080, 0x2cf, 0x3c4,
+ 0x2bf, 0x200, 0x080, 0x046, 0x020, 0x0d3, 0x000, 0x0c0,
+ 0x013, 0x0c0, 0x014, 0x0c0, 0x259, 0x045, 0x088, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x364, 0x259, 0x002, 0x009, 0x007,
+ 0x042, 0x042, 0x020, 0x0d3, 0x000, 0x387, 0x085, 0x187,
+ 0x385, 0x080, 0x247, 0x041, 0x3c5, 0x020, 0x00c, 0x045,
+ 0x387, 0x07f, 0x247, 0x2a1, 0x247, 0x2a2, 0x247, 0x2a3,
+ 0x247, 0x2a4, 0x247, 0x2a7, 0x247, 0x2a8, 0x247, 0x2a5,
+ 0x247, 0x2a6, 0x044, 0x150, 0x003, 0x0c3, 0x2e0, 0x364,
+ 0x02b, 0x010, 0x009, 0x011, 0x044, 0x38c, 0x002, 0x044,
+ 0x39f, 0x002, 0x347, 0x290, 0x2c6, 0x347, 0x28f, 0x2c5,
+ 0x347, 0x28e, 0x2c4, 0x045, 0x0c0, 0x2ce, 0x0c8, 0x2d5,
+ 0x0cf, 0x2ca, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x0c5, 0x2cc,
+ 0x0cf, 0x2d1, 0x0c0, 0x2d2, 0x0dd, 0x2cd, 0x045, 0x0c0,
+ 0x2d8, 0x044, 0x3b3, 0x002, 0x3c7, 0x038, 0x290, 0x0cc,
+ 0x28f, 0x0c0, 0x28e, 0x0cd, 0x291, 0x044, 0x034, 0x003,
+ 0x020, 0x06e, 0x003, 0x364, 0x02b, 0x010, 0x009, 0x013,
+ 0x0ca, 0x293, 0x0a0, 0x397, 0x294, 0x04d, 0x101, 0x272,
+ 0x2a0, 0x009, 0x3fc, 0x044, 0x06e, 0x003, 0x044, 0x096,
+ 0x003, 0x090, 0x3a7, 0x022, 0x044, 0x1a9, 0x000, 0x347,
+ 0x2aa, 0x2a1, 0x347, 0x2ab, 0x2a2, 0x347, 0x2ac, 0x2a3,
+ 0x347, 0x2ad, 0x2a4, 0x020, 0x3f0, 0x002, 0x044, 0x034,
+ 0x003, 0x020, 0x096, 0x003, 0x3a7, 0x020, 0x084, 0x020,
+ 0x1b7, 0x000, 0x3a7, 0x020, 0x084, 0x020, 0x1a9, 0x000,
+ 0x307, 0x2a1, 0x187, 0x327, 0x2a3, 0x00b, 0x004, 0x141,
+ 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x024, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a2, 0x187, 0x327, 0x2a4, 0x00b,
+ 0x004, 0x141, 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7,
+ 0x026, 0x044, 0x1bd, 0x000, 0x307, 0x2a7, 0x187, 0x327,
+ 0x2a5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x025, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a8, 0x187, 0x327, 0x2a6, 0x3a4,
+ 0x0ff, 0x285, 0x208, 0x3a7, 0x027, 0x020, 0x1bd, 0x000,
+ 0x0c0, 0x28f, 0x000, 0x00d, 0x140, 0x28f, 0x003, 0x3fa,
+ 0x362, 0x28f, 0x00f, 0x00b, 0x004, 0x0cf, 0x28f, 0x140,
+ 0x28e, 0x003, 0x00b, 0x362, 0x28e, 0x00f, 0x00b, 0x008,
+ 0x0cf, 0x28e, 0x000, 0x004, 0x0c0, 0x28e, 0x307, 0x28f,
+ 0x183, 0x305, 0x28e, 0x22a, 0x3a4, 0x3f7, 0x286, 0x327,
+ 0x290, 0x1a7, 0x285, 0x3a7, 0x02b, 0x044, 0x1bd, 0x000,
+ 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x0c2, 0x2fa, 0x045,
+ 0x045, 0x347, 0x2ca, 0x293, 0x000, 0x011, 0x307, 0x2ca,
+ 0x302, 0x293, 0x003, 0x3f7, 0x327, 0x2c9, 0x322, 0x293,
+ 0x003, 0x005, 0x347, 0x2c9, 0x293, 0x307, 0x291, 0x044,
+ 0x106, 0x003, 0x0a1, 0x364, 0x236, 0x100, 0x009, 0x003,
+ 0x0a5, 0x1a3, 0x285, 0x183, 0x305, 0x292, 0x183, 0x305,
+ 0x293, 0x3a7, 0x02c, 0x020, 0x1bd, 0x000, 0x307, 0x296,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x295, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x294, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x028, 0x044,
+ 0x1bd, 0x000, 0x307, 0x299, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x298, 0x044, 0x106, 0x003, 0x040, 0x307, 0x297,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x029, 0x044, 0x1bd, 0x000, 0x307, 0x29f,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x29e, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x29d, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x031, 0x044,
+ 0x1bd, 0x000, 0x307, 0x29c, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x29b, 0x044, 0x106, 0x003, 0x040, 0x307, 0x29a,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x02a, 0x020, 0x1bd, 0x000, 0x100, 0x003,
+ 0x005, 0x22a, 0x286, 0x045, 0x228, 0x121, 0x28a, 0x286,
+ 0x385, 0x010, 0x045, 0x083, 0x182, 0x0a1, 0x182, 0x385,
+ 0x001, 0x181, 0x285, 0x181, 0x385, 0x002, 0x181, 0x285,
+ 0x3a7, 0x030, 0x020, 0x1bd, 0x000, 0x044, 0x15b, 0x003,
+ 0x382, 0x081, 0x240, 0x2a5, 0x240, 0x2a6, 0x044, 0x134,
+ 0x003, 0x020, 0x3f0, 0x002, 0x364, 0x2a5, 0x300, 0x001,
+ 0x007, 0x003, 0x010, 0x3c7, 0x0ff, 0x2a5, 0x364, 0x2a6,
+ 0x300, 0x001, 0x007, 0x003, 0x00a, 0x3c7, 0x0ff, 0x2a6,
+ 0x045, 0x0c0, 0x2a5, 0x000, 0x3f3, 0x0c0, 0x2a6, 0x045,
+ 0x397, 0x2a1, 0x04c, 0x109, 0x04d, 0x118, 0x272, 0x2a9,
+ 0x009, 0x3fa, 0x045, 0x347, 0x2ae, 0x2a5, 0x347, 0x2af,
+ 0x2a6, 0x045, 0x342, 0x2fe, 0x00f, 0x003, 0x03f, 0x045,
+ 0x302, 0x2fe, 0x003, 0x019, 0x042, 0x042, 0x045, 0x327,
+ 0x00e, 0x309, 0x00f, 0x002, 0x005, 0x2a8, 0x208, 0x121,
+ 0x20e, 0x040, 0x041, 0x322, 0x2fd, 0x001, 0x3eb, 0x303,
+ 0x2fe, 0x00b, 0x3eb, 0x042, 0x247, 0x2fd, 0x042, 0x247,
+ 0x2fe, 0x000, 0x02e, 0x0c0, 0x2ff, 0x044, 0x30e, 0x002,
+ 0x364, 0x017, 0x004, 0x009, 0x3dc, 0x340, 0x046, 0x047,
+ 0x001, 0x021, 0x342, 0x2fd, 0x00e, 0x001, 0x3c5, 0x343,
+ 0x2fe, 0x00f, 0x00b, 0x017, 0x347, 0x00e, 0x2fd, 0x347,
+ 0x00f, 0x2fe, 0x009, 0x00d, 0x140, 0x2fd, 0x003, 0x009,
+ 0x342, 0x300, 0x2fd, 0x00b, 0x004, 0x0c0, 0x2fd, 0x0c1,
+ 0x2ff, 0x045, 0x30e, 0x21d, 0x102, 0x000, 0x004, 0x30e,
+ 0x21d, 0x327, 0x2c7, 0x041, 0x327, 0x2c8, 0x041, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x045, 0x366, 0x200, 0x080, 0x009,
+ 0x019, 0x364, 0x026, 0x002, 0x001, 0x014, 0x054, 0x140,
+ 0x2fa, 0x00b, 0x3f3, 0x3c7, 0x3ff, 0x2fa, 0x360, 0x2fa,
+ 0x00a, 0x003, 0x007, 0x054, 0x140, 0x01c, 0x00b, 0x3f8,
+ 0x045, 0x0c0, 0x301, 0x3ce, 0x3ff, 0x2fe, 0x040, 0x044,
+ 0x3f0, 0x002, 0x044, 0x18b, 0x003, 0x042, 0x04c, 0x140,
+ 0x301, 0x001, 0x010, 0x346, 0x301, 0x00f, 0x003, 0x029,
+ 0x320, 0x303, 0x2e4, 0x100, 0x009, 0x02a, 0x04d, 0x000,
+ 0x3e7, 0x0c1, 0x303, 0x347, 0x00f, 0x301, 0x00b, 0x004,
+ 0x152, 0x303, 0x009, 0x004, 0x0c1, 0x301, 0x040, 0x3a7,
+ 0x060, 0x044, 0x192, 0x000, 0x264, 0x008, 0x001, 0x007,
+ 0x348, 0x303, 0x303, 0x141, 0x303, 0x000, 0x3d0, 0x140,
+ 0x2ff, 0x009, 0x005, 0x322, 0x303, 0x04d, 0x045, 0x364,
+ 0x2cf, 0x080, 0x009, 0x019, 0x044, 0x346, 0x002, 0x0a0,
+ 0x397, 0x294, 0x04d, 0x101, 0x272, 0x2a0, 0x009, 0x3fc,
+ 0x044, 0x096, 0x003, 0x0c0, 0x293, 0x044, 0x06e, 0x003,
+ 0x044, 0x2b1, 0x003, 0x140, 0x2d3, 0x009, 0x002, 0x3c4,
+ 0x2fd, 0x00d, 0x044, 0x33f, 0x002, 0x3c4, 0x2ff, 0x200,
+ 0x020, 0x089, 0x000, 0x0cd, 0x2ce, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x307, 0x2e3, 0x140, 0x304, 0x009,
+ 0x004, 0x208, 0x101, 0x240, 0x2a6, 0x240, 0x2a5, 0x044,
+ 0x134, 0x003, 0x044, 0x1ba, 0x003, 0x0c0, 0x309, 0x387,
+ 0x071, 0x140, 0x304, 0x001, 0x003, 0x108, 0x364, 0x309,
+ 0x001, 0x001, 0x004, 0x380, 0x040, 0x187, 0x106, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x397, 0x2a5, 0x300, 0x309, 0x044,
+ 0x1e9, 0x003, 0x141, 0x309, 0x362, 0x309, 0x002, 0x009,
+ 0x3e0, 0x044, 0x3f0, 0x002, 0x347, 0x2a5, 0x2e3, 0x342,
+ 0x2ae, 0x2e3, 0x00b, 0x007, 0x348, 0x2e3, 0x2e3, 0x141,
+ 0x2e3, 0x140, 0x307, 0x009, 0x005, 0x347, 0x2e3, 0x307,
+ 0x045, 0x364, 0x2cf, 0x080, 0x009, 0x0ac, 0x0c1, 0x2ce,
+ 0x055, 0x3c5, 0x004, 0x2cf, 0x3a7, 0x071, 0x044, 0x192,
+ 0x000, 0x247, 0x305, 0x054, 0x080, 0x044, 0x1bd, 0x000,
+ 0x3a7, 0x02c, 0x387, 0x30f, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x022, 0x08a, 0x044, 0x1bd, 0x000, 0x081, 0x189, 0x0a0,
+ 0x044, 0x1a9, 0x000, 0x30e, 0x21d, 0x102, 0x0b0, 0x041,
+ 0x0a0, 0x041, 0x0a2, 0x044, 0x0fb, 0x002, 0x080, 0x0a0,
+ 0x262, 0x001, 0x00b, 0x004, 0x3a7, 0x0ff, 0x040, 0x397,
+ 0x2a1, 0x04d, 0x101, 0x04d, 0x101, 0x04d, 0x101, 0x04d,
+ 0x113, 0x043, 0x280, 0x3a7, 0x080, 0x04d, 0x040, 0x392,
+ 0x2a1, 0x262, 0x005, 0x00b, 0x007, 0x040, 0x044, 0x3e4,
+ 0x002, 0x042, 0x227, 0x3a4, 0x001, 0x121, 0x1a5, 0x191,
+ 0x285, 0x187, 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x042,
+ 0x044, 0x1e9, 0x003, 0x041, 0x101, 0x392, 0x2a1, 0x264,
+ 0x008, 0x001, 0x3c6, 0x390, 0x2a1, 0x111, 0x043, 0x04d,
+ 0x272, 0x2a1, 0x003, 0x3fb, 0x044, 0x150, 0x003, 0x044,
+ 0x3ea, 0x002, 0x044, 0x06e, 0x003, 0x364, 0x292, 0x008,
+ 0x009, 0x008, 0x080, 0x3a7, 0x022, 0x044, 0x1bd, 0x000,
+ 0x307, 0x020, 0x192, 0x002, 0x013, 0x19a, 0x002, 0x010,
+ 0x081, 0x189, 0x0a0, 0x044, 0x1b7, 0x000, 0x055, 0x3a7,
+ 0x071, 0x307, 0x305, 0x044, 0x1bd, 0x000, 0x044, 0x3f0,
+ 0x002, 0x3c4, 0x3fb, 0x2cf, 0x3c5, 0x080, 0x2cf, 0x054,
+ 0x045, 0x397, 0x36a, 0x247, 0x369, 0x0a0, 0x101, 0x04d,
+ 0x272, 0x38e, 0x009, 0x3fc, 0x045, 0x0c3, 0x34e, 0x1cb,
+ 0x34e, 0x3c5, 0x03f, 0x34e, 0x0c3, 0x362, 0x0c4, 0x363,
+ 0x0c0, 0x33c, 0x0c0, 0x356, 0x0c0, 0x357, 0x3d7, 0x2f9,
+ 0x352, 0x1c2, 0x352, 0x3d7, 0x254, 0x353, 0x1c0, 0x353,
+ 0x08d, 0x18b, 0x385, 0x05e, 0x247, 0x354, 0x3c7, 0x0b2,
+ 0x355, 0x345, 0x21d, 0x355, 0x0c4, 0x341, 0x0c2, 0x33f,
+ 0x0d2, 0x340, 0x0d3, 0x342, 0x0ce, 0x344, 0x0de, 0x343,
+ 0x0c0, 0x348, 0x045, 0x055, 0x3c4, 0x008, 0x33b, 0x144,
+ 0x33b, 0x0c6, 0x33d, 0x3c4, 0x3fb, 0x200, 0x307, 0x027,
+ 0x384, 0x324, 0x101, 0x247, 0x027, 0x045, 0x055, 0x3c4,
+ 0x3ec, 0x052, 0x044, 0x05b, 0x001, 0x3c4, 0x3fb, 0x00c,
+ 0x0c0, 0x359, 0x3c4, 0x1ff, 0x35b, 0x045, 0x055, 0x0c0,
+ 0x33d, 0x3c4, 0x3fe, 0x03d, 0x0df, 0x053, 0x3c4, 0x373,
+ 0x00c, 0x0c0, 0x349, 0x0c0, 0x35a, 0x3c4, 0x330, 0x027,
+ 0x0c0, 0x33b, 0x344, 0x21f, 0x33c, 0x045, 0x044, 0x050,
+ 0x005, 0x307, 0x33c, 0x00b, 0x018, 0x362, 0x33d, 0x003,
+ 0x007, 0x013, 0x009, 0x005, 0x044, 0x073, 0x004, 0x055,
+ 0x364, 0x33c, 0x040, 0x001, 0x008, 0x3c4, 0x3fb, 0x200,
+ 0x044, 0x103, 0x004, 0x044, 0x1bb, 0x005, 0x020, 0x0ad,
+ 0x000, 0x364, 0x33c, 0x002, 0x001, 0x055, 0x3c5, 0x008,
+ 0x00c, 0x364, 0x33c, 0x080, 0x009, 0x023, 0x0c0, 0x392,
+ 0x347, 0x354, 0x03e, 0x347, 0x355, 0x03f, 0x347, 0x352,
+ 0x038, 0x347, 0x353, 0x039, 0x307, 0x352, 0x040, 0x307,
+ 0x353, 0x040, 0x307, 0x354, 0x040, 0x307, 0x355, 0x305,
+ 0x21d, 0x040, 0x397, 0x2a0, 0x044, 0x01a, 0x002, 0x0c0,
+ 0x290, 0x044, 0x034, 0x003, 0x044, 0x361, 0x003, 0x0c7,
+ 0x2d1, 0x0c3, 0x2cc, 0x0c0, 0x2c8, 0x0c1, 0x2c7, 0x1cd,
+ 0x2c7, 0x0c1, 0x33d, 0x142, 0x027, 0x0c0, 0x33e, 0x0c0,
+ 0x365, 0x020, 0x04c, 0x004, 0x085, 0x364, 0x33b, 0x020,
+ 0x001, 0x003, 0x102, 0x18a, 0x385, 0x11f, 0x247, 0x053,
+ 0x045, 0x0c2, 0x33b, 0x0c5, 0x33d, 0x3c5, 0x010, 0x027,
+ 0x0df, 0x053, 0x045, 0x044, 0x3b6, 0x003, 0x044, 0x3c6,
+ 0x003, 0x044, 0x38c, 0x002, 0x044, 0x39f, 0x002, 0x3c4,
+ 0x3fb, 0x200, 0x045, 0x055, 0x364, 0x33c, 0x020, 0x009,
+ 0x007, 0x0c4, 0x33d, 0x044, 0x103, 0x004, 0x0c5, 0x33d,
+ 0x3c4, 0x3fe, 0x03d, 0x3c5, 0x00a, 0x33b, 0x307, 0x027,
+ 0x384, 0x334, 0x305, 0x392, 0x247, 0x027, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x377, 0x00c, 0x020, 0x38c, 0x002, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x044, 0x006,
+ 0x005, 0x364, 0x33b, 0x010, 0x3c4, 0x3bf, 0x33c, 0x364,
+ 0x33c, 0x020, 0x009, 0x007, 0x364, 0x33c, 0x002, 0x001,
+ 0x015, 0x3c5, 0x004, 0x00c, 0x364, 0x33c, 0x020, 0x001,
+ 0x005, 0x3c5, 0x080, 0x058, 0x0c0, 0x350, 0x044, 0x0c5,
+ 0x004, 0x3c5, 0x001, 0x052, 0x045, 0x0c0, 0x04c, 0x347,
+ 0x357, 0x04d, 0x347, 0x356, 0x04e, 0x347, 0x357, 0x364,
+ 0x347, 0x356, 0x365, 0x387, 0x067, 0x187, 0x385, 0x080,
+ 0x3c4, 0x04c, 0x052, 0x245, 0x052, 0x309, 0x026, 0x003,
+ 0x00c, 0x0c0, 0x029, 0x3c7, 0x022, 0x029, 0x345, 0x21d,
+ 0x029, 0x000, 0x00f, 0x347, 0x362, 0x02c, 0x399, 0x380,
+ 0x208, 0x244, 0x029, 0x307, 0x363, 0x187, 0x245, 0x029,
+ 0x081, 0x18c, 0x245, 0x053, 0x347, 0x021, 0x35c, 0x3c4,
+ 0x3cf, 0x021, 0x045, 0x055, 0x364, 0x052, 0x001, 0x001,
+ 0x02a, 0x142, 0x052, 0x086, 0x187, 0x385, 0x072, 0x364,
+ 0x33c, 0x020, 0x001, 0x008, 0x044, 0x18d, 0x000, 0x055,
+ 0x000, 0x005, 0x044, 0x189, 0x000, 0x347, 0x35c, 0x021,
+ 0x3c7, 0x080, 0x052, 0x3c4, 0x3fb, 0x00c, 0x364, 0x33c,
+ 0x200, 0x009, 0x008, 0x3c5, 0x020, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x045, 0x055, 0x140, 0x33d, 0x001, 0x3fc, 0x055,
+ 0x307, 0x368, 0x304, 0x34e, 0x0c0, 0x351, 0x227, 0x1a2,
+ 0x002, 0x071, 0x00b, 0x08b, 0x044, 0x03b, 0x005, 0x044,
+ 0x006, 0x005, 0x141, 0x374, 0x140, 0x346, 0x001, 0x00b,
+ 0x342, 0x342, 0x346, 0x00b, 0x019, 0x0c4, 0x350, 0x000,
+ 0x019, 0x340, 0x347, 0x345, 0x001, 0x00a, 0x342, 0x347,
+ 0x345, 0x364, 0x33c, 0x004, 0x001, 0x3ec, 0x0cc, 0x350,
+ 0x141, 0x384, 0x000, 0x009, 0x0c8, 0x350, 0x141, 0x383,
+ 0x340, 0x342, 0x346, 0x140, 0x345, 0x001, 0x00c, 0x342,
+ 0x344, 0x345, 0x00b, 0x00e, 0x3c5, 0x001, 0x350, 0x000,
+ 0x00e, 0x3c5, 0x003, 0x350, 0x141, 0x37c, 0x000, 0x00a,
+ 0x3c5, 0x002, 0x350, 0x141, 0x37b, 0x340, 0x344, 0x345,
+ 0x140, 0x347, 0x001, 0x00c, 0x342, 0x343, 0x347, 0x00b,
+ 0x00f, 0x3c5, 0x010, 0x350, 0x000, 0x00f, 0x3c5, 0x030,
+ 0x350, 0x141, 0x38c, 0x020, 0x3fa, 0x004, 0x3c5, 0x020,
+ 0x350, 0x141, 0x38b, 0x340, 0x343, 0x347, 0x020, 0x3fa,
+ 0x004, 0x044, 0x03b, 0x005, 0x141, 0x375, 0x0c0, 0x346,
+ 0x0c0, 0x347, 0x0c0, 0x345, 0x044, 0x01e, 0x005, 0x3c7,
+ 0x03f, 0x350, 0x141, 0x37c, 0x141, 0x384, 0x141, 0x38c,
+ 0x0c0, 0x348, 0x020, 0x3fa, 0x004, 0x205, 0x001, 0x182,
+ 0x044, 0x03b, 0x005, 0x227, 0x3a4, 0x00c, 0x1b1, 0x001,
+ 0x013, 0x364, 0x350, 0x00c, 0x009, 0x05c, 0x364, 0x04d,
+ 0x00c, 0x009, 0x057, 0x132, 0x003, 0x00b, 0x001, 0x027,
+ 0x000, 0x050, 0x3c4, 0x3f3, 0x350, 0x000, 0x04b, 0x140,
+ 0x351, 0x009, 0x004, 0x141, 0x37f, 0x140, 0x346, 0x001,
+ 0x00b, 0x151, 0x348, 0x151, 0x346, 0x3c5, 0x004, 0x350,
+ 0x000, 0x038, 0x3c5, 0x00c, 0x350, 0x140, 0x351, 0x001,
+ 0x031, 0x141, 0x384, 0x000, 0x02d, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x380, 0x342, 0x23e, 0x348, 0x001, 0x016,
+ 0x342, 0x342, 0x346, 0x001, 0x00e, 0x141, 0x348, 0x141,
+ 0x346, 0x340, 0x342, 0x346, 0x3c5, 0x004, 0x350, 0x000,
+ 0x00e, 0x340, 0x342, 0x346, 0x3c5, 0x008, 0x350, 0x140,
+ 0x351, 0x001, 0x004, 0x141, 0x383, 0x340, 0x23e, 0x348,
+ 0x227, 0x3a4, 0x003, 0x001, 0x013, 0x364, 0x350, 0x003,
+ 0x009, 0x071, 0x364, 0x04d, 0x003, 0x009, 0x06c, 0x132,
+ 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065, 0x3c4, 0x3fc,
+ 0x350, 0x000, 0x060, 0x140, 0x351, 0x009, 0x004, 0x141,
+ 0x377, 0x140, 0x345, 0x009, 0x00d, 0x3c5, 0x003, 0x350,
+ 0x140, 0x351, 0x001, 0x04f, 0x141, 0x37c, 0x000, 0x04b,
+ 0x342, 0x342, 0x346, 0x009, 0x007, 0x340, 0x342, 0x346,
+ 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151, 0x348, 0x141,
+ 0x346, 0x152, 0x345, 0x3c5, 0x001, 0x350, 0x000, 0x033,
+ 0x140, 0x351, 0x009, 0x004, 0x141, 0x378, 0x342, 0x23e,
+ 0x348, 0x001, 0x01c, 0x342, 0x344, 0x345, 0x001, 0x014,
+ 0x140, 0x346, 0x001, 0x010, 0x141, 0x348, 0x151, 0x346,
+ 0x142, 0x345, 0x340, 0x344, 0x345, 0x3c5, 0x001, 0x350,
+ 0x000, 0x00e, 0x340, 0x344, 0x345, 0x3c5, 0x002, 0x350,
+ 0x140, 0x351, 0x001, 0x004, 0x141, 0x37b, 0x340, 0x23e,
+ 0x348, 0x227, 0x1b3, 0x3a4, 0x003, 0x001, 0x013, 0x364,
+ 0x350, 0x030, 0x009, 0x071, 0x364, 0x04d, 0x030, 0x009,
+ 0x06c, 0x132, 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065,
+ 0x3c4, 0x3cf, 0x350, 0x000, 0x060, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x387, 0x140, 0x347, 0x009, 0x00d, 0x3c5,
+ 0x030, 0x350, 0x140, 0x351, 0x001, 0x04f, 0x141, 0x38c,
+ 0x000, 0x04b, 0x342, 0x342, 0x346, 0x009, 0x007, 0x340,
+ 0x342, 0x346, 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151,
+ 0x348, 0x141, 0x346, 0x152, 0x347, 0x3c5, 0x010, 0x350,
+ 0x000, 0x033, 0x140, 0x351, 0x009, 0x004, 0x141, 0x388,
+ 0x342, 0x23e, 0x348, 0x001, 0x01c, 0x342, 0x343, 0x347,
+ 0x001, 0x014, 0x140, 0x346, 0x001, 0x010, 0x141, 0x348,
+ 0x151, 0x346, 0x142, 0x347, 0x340, 0x343, 0x347, 0x3c5,
+ 0x010, 0x350, 0x000, 0x00e, 0x340, 0x343, 0x347, 0x3c5,
+ 0x020, 0x350, 0x140, 0x351, 0x001, 0x004, 0x141, 0x38b,
+ 0x340, 0x23e, 0x348, 0x328, 0x04d, 0x324, 0x350, 0x2e4,
+ 0x02a, 0x001, 0x026, 0x140, 0x351, 0x009, 0x022, 0x141,
+ 0x351, 0x2e4, 0x020, 0x001, 0x005, 0x3c4, 0x3cf, 0x350,
+ 0x2e4, 0x008, 0x001, 0x005, 0x3c4, 0x3f3, 0x350, 0x2e4,
+ 0x002, 0x001, 0x28a, 0x3c4, 0x3fc, 0x350, 0x000, 0x285,
+ 0x0c0, 0x350, 0x3c4, 0x3bf, 0x027, 0x000, 0x0a8, 0x328,
+ 0x04d, 0x324, 0x350, 0x287, 0x3a4, 0x003, 0x131, 0x009,
+ 0x029, 0x140, 0x345, 0x001, 0x01c, 0x342, 0x344, 0x345,
+ 0x044, 0x030, 0x005, 0x003, 0x006, 0x140, 0x345, 0x009,
+ 0x00b, 0x364, 0x368, 0x002, 0x001, 0x006, 0x141, 0x350,
+ 0x141, 0x37b, 0x340, 0x344, 0x345, 0x000, 0x00b, 0x364,
+ 0x368, 0x001, 0x001, 0x006, 0x142, 0x350, 0x141, 0x37c,
+ 0x227, 0x3a4, 0x00c, 0x134, 0x009, 0x039, 0x140, 0x346,
+ 0x001, 0x01c, 0x342, 0x342, 0x346, 0x044, 0x030, 0x005,
+ 0x003, 0x006, 0x140, 0x346, 0x009, 0x00b, 0x364, 0x368,
+ 0x008, 0x001, 0x006, 0x144, 0x350, 0x141, 0x383, 0x340,
+ 0x342, 0x346, 0x000, 0x01b, 0x364, 0x368, 0x008, 0x009,
+ 0x3e3, 0x364, 0x368, 0x004, 0x001, 0x011, 0x327, 0x345,
+ 0x320, 0x347, 0x001, 0x007, 0x364, 0x33c, 0x004, 0x001,
+ 0x006, 0x148, 0x350, 0x141, 0x384, 0x227, 0x3a4, 0x030,
+ 0x3a2, 0x010, 0x009, 0x02b, 0x140, 0x347, 0x001, 0x01d,
+ 0x342, 0x343, 0x347, 0x044, 0x030, 0x005, 0x003, 0x006,
+ 0x140, 0x347, 0x009, 0x00c, 0x364, 0x368, 0x020, 0x001,
+ 0x007, 0x3c0, 0x010, 0x350, 0x141, 0x38b, 0x340, 0x343,
+ 0x347, 0x000, 0x00c, 0x364, 0x368, 0x010, 0x001, 0x007,
+ 0x3c0, 0x020, 0x350, 0x141, 0x38c, 0x044, 0x3fa, 0x004,
+ 0x000, 0x026, 0x3c5, 0x040, 0x027, 0x307, 0x04d, 0x384,
+ 0x200, 0x305, 0x350, 0x247, 0x04d, 0x045, 0x08f, 0x3b7,
+ 0x211, 0x044, 0x1b8, 0x000, 0x347, 0x33f, 0x346, 0x347,
+ 0x340, 0x347, 0x347, 0x341, 0x345, 0x347, 0x347, 0x348,
+ 0x340, 0x345, 0x348, 0x340, 0x346, 0x348, 0x307, 0x347,
+ 0x183, 0x305, 0x345, 0x184, 0x305, 0x346, 0x32a, 0x346,
+ 0x3a4, 0x008, 0x285, 0x3b7, 0x210, 0x020, 0x1bd, 0x000,
+ 0x364, 0x33c, 0x008, 0x001, 0x007, 0x327, 0x23e, 0x322,
+ 0x348, 0x131, 0x045, 0x040, 0x227, 0x306, 0x373, 0x001,
+ 0x00f, 0x307, 0x369, 0x101, 0x272, 0x372, 0x009, 0x003,
+ 0x118, 0x04d, 0x247, 0x369, 0x2c7, 0x373, 0x042, 0x045,
+ 0x140, 0x35a, 0x00b, 0x0b0, 0x344, 0x21f, 0x35a, 0x0a3,
+ 0x1ad, 0x2a8, 0x2c4, 0x027, 0x055, 0x044, 0x340, 0x001,
+ 0x054, 0x264, 0x002, 0x001, 0x005, 0x0c0, 0x33d, 0x045,
+ 0x083, 0x18b, 0x304, 0x050, 0x305, 0x221, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3f4, 0x308, 0x221, 0x244, 0x027, 0x387,
+ 0x030, 0x304, 0x050, 0x187, 0x044, 0x0d3, 0x005, 0x001,
+ 0x3e6, 0x08c, 0x304, 0x050, 0x189, 0x044, 0x0d3, 0x005,
+ 0x001, 0x3dd, 0x083, 0x304, 0x050, 0x18b, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3d4, 0x083, 0x18b, 0x304, 0x365, 0x044,
+ 0x0d3, 0x005, 0x001, 0x3cb, 0x387, 0x030, 0x304, 0x365,
+ 0x187, 0x044, 0x0d3, 0x005, 0x001, 0x3c1, 0x08c, 0x304,
+ 0x365, 0x189, 0x044, 0x0d3, 0x005, 0x001, 0x3b8, 0x083,
+ 0x304, 0x365, 0x18b, 0x044, 0x0d3, 0x005, 0x001, 0x3af,
+ 0x307, 0x21d, 0x304, 0x364, 0x192, 0x364, 0x052, 0x020,
+ 0x001, 0x00d, 0x327, 0x21d, 0x324, 0x04f, 0x001, 0x005,
+ 0x345, 0x21d, 0x33c, 0x1b1, 0x285, 0x044, 0x0d3, 0x005,
+ 0x001, 0x395, 0x045, 0x364, 0x33b, 0x001, 0x001, 0x02c,
+ 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x027, 0x245, 0x027, 0x345,
+ 0x21d, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x01d, 0x044,
+ 0x103, 0x005, 0x364, 0x35b, 0x002, 0x001, 0x3f5, 0x344,
+ 0x21f, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x00d, 0x044,
+ 0x103, 0x005, 0x307, 0x35b, 0x264, 0x002, 0x009, 0x3f4,
+ 0x264, 0x001, 0x045, 0x055, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x044, 0x340, 0x001, 0x247, 0x35b, 0x374, 0x027,
+ 0x200, 0x001, 0x00a, 0x264, 0x080, 0x009, 0x006, 0x3b8,
+ 0x200, 0x2c4, 0x027, 0x054, 0x045, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x001, 0x088, 0x3b8, 0x300, 0x2c4, 0x027,
+ 0x264, 0x040, 0x001, 0x00f, 0x140, 0x359, 0x001, 0x00b,
+ 0x0c0, 0x33d, 0x0c0, 0x359, 0x3d5, 0x200, 0x027, 0x000,
+ 0x060, 0x384, 0x030, 0x327, 0x359, 0x2e2, 0x008, 0x003,
+ 0x3f1, 0x2a9, 0x3b0, 0x15a, 0x041, 0x1b9, 0x3b0, 0x005,
+ 0x041, 0x045, 0x000, 0x012, 0x000, 0x015, 0x000, 0x015,
+ 0x000, 0x016, 0x000, 0x017, 0x000, 0x01a, 0x000, 0x01c,
+ 0x000, 0x01f, 0x000, 0x022, 0x187, 0x247, 0x367, 0x000,
+ 0x033, 0x000, 0x02f, 0x191, 0x000, 0x02c, 0x193, 0x000,
+ 0x029, 0x187, 0x247, 0x366, 0x000, 0x026, 0x245, 0x366,
+ 0x000, 0x022, 0x191, 0x245, 0x366, 0x000, 0x01d, 0x193,
+ 0x245, 0x366, 0x000, 0x018, 0x229, 0x3a4, 0x040, 0x2c5,
+ 0x33c, 0x329, 0x04d, 0x18b, 0x2cc, 0x04d, 0x347, 0x367,
+ 0x368, 0x347, 0x366, 0x04e, 0x0c0, 0x359, 0x000, 0x006,
+ 0x245, 0x367, 0x141, 0x359, 0x3d5, 0x300, 0x027, 0x140,
+ 0x359, 0x009, 0x011, 0x140, 0x04d, 0x00b, 0x00a, 0x364,
+ 0x33c, 0x040, 0x001, 0x005, 0x020, 0x103, 0x004, 0x044,
+ 0x137, 0x004, 0x045, 0x362, 0x33e, 0x007, 0x009, 0x003,
+ 0x045, 0x055, 0x362, 0x33d, 0x000, 0x006, 0x077, 0x362,
+ 0x33d, 0x005, 0x00f, 0x072, 0x140, 0x34d, 0x009, 0x007,
+ 0x044, 0x23e, 0x005, 0x000, 0x069, 0x308, 0x356, 0x304,
+ 0x365, 0x001, 0x007, 0x044, 0x36a, 0x005, 0x000, 0x05e,
+ 0x140, 0x349, 0x001, 0x017, 0x364, 0x04f, 0x03f, 0x009,
+ 0x055, 0x0c0, 0x349, 0x362, 0x34f, 0x000, 0x006, 0x04e,
+ 0x307, 0x34f, 0x044, 0x000, 0x006, 0x0c0, 0x34f, 0x000,
+ 0x045, 0x140, 0x33e, 0x009, 0x007, 0x044, 0x23e, 0x005,
+ 0x000, 0x03c, 0x362, 0x33e, 0x001, 0x009, 0x007, 0x044,
+ 0x269, 0x005, 0x000, 0x032, 0x362, 0x33e, 0x002, 0x009,
+ 0x007, 0x044, 0x26f, 0x005, 0x000, 0x028, 0x362, 0x33e,
+ 0x003, 0x009, 0x007, 0x044, 0x27f, 0x005, 0x000, 0x01e,
+ 0x362, 0x33e, 0x004, 0x009, 0x007, 0x044, 0x29a, 0x005,
+ 0x000, 0x014, 0x362, 0x33e, 0x005, 0x009, 0x007, 0x044,
+ 0x2b7, 0x005, 0x000, 0x00a, 0x362, 0x33e, 0x006, 0x009,
+ 0x005, 0x044, 0x34f, 0x005, 0x054, 0x045, 0x081, 0x189,
+ 0x304, 0x053, 0x009, 0x00c, 0x0c1, 0x33d, 0x083, 0x189,
+ 0x395, 0x3f7, 0x244, 0x027, 0x000, 0x01c, 0x0c2, 0x33d,
+ 0x0c1, 0x34d, 0x3c5, 0x008, 0x027, 0x0c0, 0x34b, 0x0c0,
+ 0x34c, 0x140, 0x35d, 0x009, 0x00b, 0x0c0, 0x34f, 0x088,
+ 0x189, 0x245, 0x365, 0x044, 0x3c2, 0x005, 0x0c1, 0x33e,
+ 0x045, 0x347, 0x35e, 0x34a, 0x0c2, 0x33e, 0x045, 0x307,
+ 0x34b, 0x302, 0x360, 0x006, 0x009, 0x141, 0x34b, 0x044,
+ 0x3f3, 0x005, 0x000, 0x004, 0x0c3, 0x33e, 0x045, 0x307,
+ 0x34c, 0x302, 0x361, 0x006, 0x014, 0x141, 0x34c, 0x088,
+ 0x3c7, 0x020, 0x34f, 0x3c7, 0x028, 0x372, 0x141, 0x38a,
+ 0x141, 0x382, 0x044, 0x000, 0x006, 0x000, 0x004, 0x0c4,
+ 0x33e, 0x045, 0x0c5, 0x33e, 0x0c0, 0x2d3, 0x364, 0x02b,
+ 0x080, 0x009, 0x015, 0x3c5, 0x010, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x362, 0x201, 0x002, 0x001, 0x00a, 0x0c1, 0x244,
+ 0x3c5, 0x004, 0x245, 0x3c4, 0x3fb, 0x200, 0x045, 0x364,
+ 0x2cf, 0x010, 0x009, 0x094, 0x364, 0x34a, 0x001, 0x001,
+ 0x009, 0x0c6, 0x33e, 0x044, 0x33f, 0x002, 0x000, 0x088,
+ 0x140, 0x259, 0x009, 0x009, 0x044, 0x346, 0x002, 0x0c0,
+ 0x330, 0x0c0, 0x331, 0x044, 0x012, 0x006, 0x140, 0x330,
+ 0x009, 0x00c, 0x140, 0x331, 0x009, 0x008, 0x347, 0x32e,
+ 0x330, 0x347, 0x32f, 0x331, 0x362, 0x331, 0x000, 0x00f,
+ 0x00c, 0x362, 0x32f, 0x000, 0x00f, 0x007, 0x044, 0x3f3,
+ 0x005, 0x000, 0x057, 0x362, 0x331, 0x000, 0x007, 0x00c,
+ 0x362, 0x32f, 0x000, 0x007, 0x007, 0x044, 0x3e6, 0x005,
+ 0x000, 0x048, 0x044, 0x33f, 0x002, 0x0c6, 0x33e, 0x362,
+ 0x331, 0x000, 0x00f, 0x021, 0x308, 0x330, 0x348, 0x331,
+ 0x336, 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x330, 0x347,
+ 0x336, 0x331, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302,
+ 0x32e, 0x343, 0x32f, 0x336, 0x002, 0x024, 0x044, 0x3e6,
+ 0x005, 0x000, 0x01f, 0x308, 0x32e, 0x348, 0x32f, 0x336,
+ 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x32e, 0x347, 0x336,
+ 0x32f, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302, 0x32e,
+ 0x343, 0x32f, 0x336, 0x002, 0x005, 0x044, 0x3f3, 0x005,
+ 0x347, 0x32e, 0x330, 0x347, 0x32f, 0x331, 0x045, 0x3c5,
+ 0x008, 0x33b, 0x0c3, 0x33d, 0x3c5, 0x004, 0x027, 0x0c7,
+ 0x33e, 0x364, 0x33c, 0x020, 0x001, 0x00a, 0x345, 0x21d,
+ 0x364, 0x347, 0x21d, 0x35a, 0x000, 0x005, 0x345, 0x21d,
+ 0x04d, 0x045, 0x364, 0x04f, 0x03f, 0x001, 0x054, 0x307,
+ 0x04f, 0x384, 0x003, 0x113, 0x009, 0x007, 0x3c5, 0x001,
+ 0x34a, 0x141, 0x37e, 0x307, 0x04f, 0x384, 0x003, 0x112,
+ 0x009, 0x007, 0x3c5, 0x001, 0x34a, 0x141, 0x37d, 0x307,
+ 0x04f, 0x384, 0x00c, 0x11c, 0x009, 0x007, 0x3c5, 0x002,
+ 0x34a, 0x141, 0x386, 0x307, 0x04f, 0x384, 0x00c, 0x118,
+ 0x009, 0x007, 0x3c5, 0x002, 0x34a, 0x141, 0x385, 0x307,
+ 0x04f, 0x384, 0x030, 0x382, 0x030, 0x009, 0x007, 0x3c5,
+ 0x004, 0x34a, 0x141, 0x38e, 0x307, 0x04f, 0x384, 0x030,
+ 0x382, 0x020, 0x009, 0x007, 0x3c5, 0x004, 0x34a, 0x141,
+ 0x38d, 0x347, 0x356, 0x365, 0x0c1, 0x349, 0x044, 0x3c2,
+ 0x005, 0x045, 0x088, 0x189, 0x304, 0x365, 0x001, 0x004,
+ 0x141, 0x376, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x347,
+ 0x21d, 0x35a, 0x000, 0x013, 0x347, 0x365, 0x04e, 0x307,
+ 0x04e, 0x306, 0x356, 0x001, 0x007, 0x3c5, 0x080, 0x027,
+ 0x000, 0x005, 0x3c4, 0x37f, 0x027, 0x045, 0x081, 0x0c4,
+ 0x34f, 0x0c5, 0x372, 0x141, 0x379, 0x141, 0x381, 0x044,
+ 0x000, 0x006, 0x045, 0x088, 0x0c2, 0x34f, 0x0ca, 0x372,
+ 0x141, 0x37a, 0x141, 0x382, 0x044, 0x000, 0x006, 0x045,
+ 0x364, 0x33c, 0x010, 0x001, 0x009, 0x22a, 0x3a4, 0x015,
+ 0x209, 0x384, 0x02a, 0x285, 0x245, 0x365, 0x044, 0x3c2,
+ 0x005, 0x045, 0x0c2, 0x2ce, 0x0c0, 0x304, 0x044, 0x25b,
+ 0x003, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x086, 0x189, 0x395,
+ 0x104, 0x0a2, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003,
+ 0x3c5, 0x100, 0x00d, 0x044, 0x30e, 0x002, 0x347, 0x00e,
+ 0x32e, 0x347, 0x00f, 0x32f, 0x045, 0x307, 0x000, 0x183,
+ 0x247, 0x21b, 0x003, 0x00b, 0x347, 0x21e, 0x008, 0x307,
+ 0x006, 0x327, 0x007, 0x000, 0x009, 0x347, 0x21e, 0x00b,
+ 0x307, 0x009, 0x327, 0x00a, 0x020, 0x1bb, 0x006, 0x347,
+ 0x059, 0x24b, 0x347, 0x248, 0x24a, 0x044, 0x340, 0x001,
+ 0x247, 0x248, 0x054, 0x364, 0x001, 0x080, 0x009, 0x059,
+ 0x347, 0x24a, 0x248, 0x364, 0x001, 0x001, 0x009, 0x171,
+ 0x364, 0x001, 0x002, 0x009, 0x171, 0x364, 0x001, 0x004,
+ 0x009, 0x17c, 0x364, 0x001, 0x008, 0x009, 0x1a4, 0x364,
+ 0x001, 0x010, 0x029, 0x042, 0x008, 0x364, 0x001, 0x020,
+ 0x029, 0x01f, 0x008, 0x364, 0x001, 0x040, 0x009, 0x1b1,
+ 0x364, 0x001, 0x100, 0x009, 0x1ac, 0x327, 0x21c, 0x324,
+ 0x020, 0x029, 0x0ab, 0x006, 0x364, 0x23f, 0x003, 0x001,
+ 0x00c, 0x044, 0x340, 0x001, 0x227, 0x1ba, 0x326, 0x248,
+ 0x054, 0x009, 0x026, 0x307, 0x24b, 0x306, 0x059, 0x384,
+ 0x080, 0x009, 0x01b, 0x055, 0x142, 0x000, 0x047, 0x3c5,
+ 0x020, 0x000, 0x3c5, 0x001, 0x233, 0x000, 0x012, 0x364,
+ 0x233, 0x001, 0x001, 0x00a, 0x3c5, 0x020, 0x000, 0x347,
+ 0x24a, 0x248, 0x000, 0x3e9, 0x044, 0x340, 0x001, 0x247,
+ 0x248, 0x347, 0x059, 0x24b, 0x044, 0x34c, 0x001, 0x009,
+ 0x012, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x364, 0x24b,
+ 0x080, 0x009, 0x008, 0x306, 0x249, 0x19a, 0x001, 0x3cd,
+ 0x054, 0x364, 0x23f, 0x002, 0x001, 0x052, 0x307, 0x248,
+ 0x19e, 0x00a, 0x01c, 0x3c5, 0x084, 0x021, 0x3c4, 0x3fe,
+ 0x026, 0x307, 0x248, 0x306, 0x249, 0x209, 0x00b, 0x022,
+ 0x3b7, 0x210, 0x044, 0x192, 0x000, 0x247, 0x257, 0x080,
+ 0x044, 0x1bd, 0x000, 0x000, 0x015, 0x364, 0x026, 0x001,
+ 0x009, 0x010, 0x3c4, 0x37b, 0x021, 0x364, 0x234, 0x001,
+ 0x001, 0x008, 0x3c5, 0x001, 0x026, 0x044, 0x342, 0x006,
+ 0x307, 0x248, 0x19d, 0x00a, 0x007, 0x3c4, 0x3fd, 0x026,
+ 0x000, 0x00d, 0x364, 0x234, 0x002, 0x001, 0x008, 0x3c5,
+ 0x002, 0x026, 0x044, 0x141, 0x001, 0x327, 0x248, 0x326,
+ 0x249, 0x00b, 0x005, 0x044, 0x27d, 0x001, 0x364, 0x23f,
+ 0x001, 0x001, 0x013, 0x044, 0x34c, 0x001, 0x001, 0x007,
+ 0x2a8, 0x2c4, 0x233, 0x000, 0x00b, 0x327, 0x248, 0x326,
+ 0x249, 0x1ba, 0x3a4, 0x003, 0x001, 0x01f, 0x327, 0x248,
+ 0x1ba, 0x287, 0x190, 0x002, 0x006, 0x3a4, 0x3fb, 0x000,
+ 0x004, 0x3a5, 0x004, 0x3a4, 0x007, 0x307, 0x234, 0x384,
+ 0x3f8, 0x225, 0x344, 0x21f, 0x232, 0x347, 0x248, 0x249,
+ 0x020, 0x022, 0x008, 0x347, 0x248, 0x249, 0x364, 0x33c,
+ 0x020, 0x001, 0x005, 0x044, 0x12d, 0x005, 0x364, 0x233,
+ 0x001, 0x001, 0x314, 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4,
+ 0x000, 0x328, 0x21b, 0x044, 0x32d, 0x001, 0x043, 0x307,
+ 0x21b, 0x009, 0x00e, 0x2c7, 0x005, 0x144, 0x000, 0x364,
+ 0x233, 0x001, 0x001, 0x2fb, 0x020, 0x0a3, 0x006, 0x00b,
+ 0x009, 0x0c0, 0x21b, 0x2c7, 0x00b, 0x148, 0x000, 0x047,
+ 0x0c0, 0x21b, 0x2c7, 0x008, 0x141, 0x000, 0x047, 0x3c4,
+ 0x3df, 0x000, 0x364, 0x000, 0x100, 0x001, 0x2a2, 0x364,
+ 0x233, 0x001, 0x001, 0x005, 0x3c5, 0x020, 0x000, 0x307,
+ 0x003, 0x327, 0x004, 0x041, 0x044, 0x32c, 0x001, 0x043,
+ 0x044, 0x32d, 0x001, 0x040, 0x209, 0x002, 0x083, 0x003,
+ 0x089, 0x188, 0x199, 0x041, 0x229, 0x280, 0x043, 0x390,
+ 0x057, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x3c4,
+ 0x3f3, 0x012, 0x000, 0x2b3, 0x0af, 0x1a7, 0x2a8, 0x2c4,
+ 0x01f, 0x3c5, 0x001, 0x200, 0x344, 0x21f, 0x232, 0x151,
+ 0x2fa, 0x020, 0x247, 0x002, 0x364, 0x03a, 0x040, 0x009,
+ 0x007, 0x3c4, 0x3f7, 0x03a, 0x000, 0x299, 0x3c4, 0x3f0,
+ 0x03a, 0x364, 0x2cf, 0x001, 0x001, 0x01b, 0x364, 0x2cf,
+ 0x002, 0x009, 0x016, 0x142, 0x03a, 0x141, 0x2e6, 0x3c1,
+ 0x000, 0x2e7, 0x32a, 0x03a, 0x2f4, 0x200, 0x001, 0x3fc,
+ 0x143, 0x03a, 0x374, 0x25e, 0x200, 0x009, 0x002, 0x000,
+ 0x276, 0x364, 0x03d, 0x040, 0x009, 0x007, 0x3c4, 0x3f7,
+ 0x03d, 0x000, 0x26c, 0x3c4, 0x3f0, 0x03d, 0x364, 0x33b,
+ 0x001, 0x001, 0x007, 0x044, 0x3a3, 0x003, 0x000, 0x25f,
+ 0x374, 0x25f, 0x200, 0x009, 0x002, 0x000, 0x258, 0x158,
+ 0x025, 0x044, 0x141, 0x001, 0x020, 0x0a3, 0x006, 0x042,
+ 0x3a7, 0x020, 0x1a7, 0x3a5, 0x055, 0x020, 0x1de, 0x007,
+ 0x042, 0x394, 0x3ff, 0x04d, 0x040, 0x020, 0x181, 0x006,
+ 0x042, 0x394, 0x3ff, 0x04c, 0x020, 0x1de, 0x007, 0x364,
+ 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x042, 0x186, 0x003,
+ 0x06b, 0x002, 0x06d, 0x2e4, 0x001, 0x001, 0x059, 0x287,
+ 0x384, 0x00e, 0x190, 0x3a4, 0x370, 0x285, 0x041, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x043, 0x1b3, 0x3c4, 0x3fd, 0x00c,
+ 0x3c4, 0x3fc, 0x018, 0x287, 0x384, 0x070, 0x382, 0x030,
+ 0x001, 0x007, 0x3a4, 0x007, 0x133, 0x009, 0x007, 0x3c4,
+ 0x3ef, 0x021, 0x141, 0x018, 0x142, 0x00c, 0x0a1, 0x1aa,
+ 0x364, 0x026, 0x004, 0x009, 0x004, 0x2c5, 0x02a, 0x2a8,
+ 0x082, 0x044, 0x189, 0x000, 0x2c4, 0x02a, 0x3c4, 0x3f7,
+ 0x00d, 0x3c5, 0x001, 0x00d, 0x3c4, 0x3fe, 0x00d, 0x140,
+ 0x013, 0x009, 0x00b, 0x140, 0x014, 0x009, 0x007, 0x3c5,
+ 0x008, 0x00d, 0x000, 0x014, 0x0c0, 0x012, 0x0c2, 0x012,
+ 0x0c0, 0x012, 0x0c5, 0x012, 0x000, 0x00a, 0x0c0, 0x00d,
+ 0x044, 0x33f, 0x002, 0x3c4, 0x3fd, 0x00c, 0x0a3, 0x020,
+ 0x1de, 0x007, 0x2c7, 0x013, 0x000, 0x3fa, 0x2c7, 0x014,
+ 0x000, 0x3f6, 0x020, 0x181, 0x006, 0x2e4, 0x100, 0x009,
+ 0x042, 0x041, 0x3a4, 0x007, 0x387, 0x006, 0x189, 0x395,
+ 0x321, 0x280, 0x050, 0x0c3, 0x265, 0x262, 0x005, 0x00b,
+ 0x004, 0x0c7, 0x265, 0x043, 0x1b3, 0x2e2, 0x003, 0x009,
+ 0x00b, 0x041, 0x040, 0x084, 0x0b1, 0x044, 0x1b7, 0x000,
+ 0x042, 0x043, 0x2a9, 0x349, 0x265, 0x265, 0x111, 0x00b,
+ 0x3fb, 0x2aa, 0x34a, 0x265, 0x265, 0x348, 0x265, 0x265,
+ 0x041, 0x3a7, 0x030, 0x044, 0x192, 0x000, 0x043, 0x304,
+ 0x265, 0x285, 0x3a7, 0x030, 0x044, 0x1bd, 0x000, 0x000,
+ 0x007, 0x084, 0x0b1, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x000, 0x006, 0x009, 0x002, 0x004, 0x2c7, 0x22c,
+ 0x2f4, 0x100, 0x001, 0x013, 0x044, 0x342, 0x006, 0x364,
+ 0x23f, 0x002, 0x009, 0x005, 0x020, 0x181, 0x006, 0x345,
+ 0x21d, 0x232, 0x020, 0x2ea, 0x002, 0x044, 0x1ef, 0x001,
+ 0x000, 0x3ef, 0x044, 0x1ef, 0x001, 0x091, 0x044, 0x189,
+ 0x000, 0x3b7, 0x210, 0x307, 0x257, 0x020, 0x1bd, 0x000,
+ 0x1ad, 0x1b9, 0x3c4, 0x3cf, 0x232, 0x2c5, 0x232, 0x020,
+ 0x181, 0x006, 0x287, 0x2e4, 0x100, 0x001, 0x016, 0x041,
+ 0x384, 0x001, 0x182, 0x040, 0x3b7, 0x211, 0x044, 0x1a9,
+ 0x000, 0x042, 0x385, 0x3f3, 0x3b7, 0x211, 0x044, 0x1b8,
+ 0x000, 0x043, 0x287, 0x2f4, 0x200, 0x001, 0x016, 0x384,
+ 0x010, 0x190, 0x227, 0x190, 0x285, 0x040, 0x3a7, 0x060,
+ 0x044, 0x1a9, 0x000, 0x042, 0x385, 0x3f3, 0x3a7, 0x060,
+ 0x044, 0x1b8, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x100,
+ 0x009, 0x059, 0x287, 0x384, 0x0ff, 0x264, 0x080, 0x001,
+ 0x004, 0x385, 0x300, 0x1bd, 0x3a4, 0x003, 0x009, 0x004,
+ 0x247, 0x23a, 0x131, 0x009, 0x006, 0x247, 0x23c, 0x003,
+ 0x036, 0x131, 0x009, 0x004, 0x247, 0x23b, 0x131, 0x009,
+ 0x00a, 0x247, 0x23a, 0x003, 0x02a, 0x247, 0x23c, 0x247,
+ 0x23b, 0x327, 0x23a, 0x00b, 0x004, 0x2a8, 0x121, 0x140,
+ 0x23b, 0x00b, 0x006, 0x322, 0x23b, 0x000, 0x004, 0x320,
+ 0x23b, 0x320, 0x23c, 0x322, 0x23e, 0x007, 0x010, 0x347,
+ 0x23a, 0x237, 0x347, 0x23b, 0x239, 0x347, 0x23c, 0x238,
+ 0x044, 0x05b, 0x001, 0x000, 0x038, 0x347, 0x237, 0x23a,
+ 0x347, 0x239, 0x23b, 0x347, 0x238, 0x23c, 0x020, 0x1dc,
+ 0x007, 0x140, 0x33b, 0x009, 0x015, 0x289, 0x002, 0x00c,
+ 0x003, 0x006, 0x327, 0x237, 0x000, 0x008, 0x327, 0x238,
+ 0x000, 0x004, 0x327, 0x239, 0x042, 0x020, 0x1de, 0x007,
+ 0x289, 0x002, 0x00c, 0x003, 0x006, 0x327, 0x345, 0x000,
+ 0x3f5, 0x327, 0x346, 0x000, 0x3f1, 0x327, 0x347, 0x000,
+ 0x3ed, 0x2c7, 0x01a, 0x020, 0x181, 0x006, 0x327, 0x00c,
+ 0x3c5, 0x002, 0x00c, 0x3c5, 0x001, 0x00d, 0x08f, 0x044,
+ 0x189, 0x000, 0x3c4, 0x3fe, 0x00d, 0x2c7, 0x00c, 0x020,
+ 0x181, 0x006, 0x130, 0x001, 0x013, 0x083, 0x044, 0x189,
+ 0x000, 0x3c6, 0x001, 0x02b, 0x083, 0x044, 0x189, 0x000,
+ 0x3c6, 0x001, 0x02b, 0x131, 0x009, 0x3f1, 0x020, 0x181,
+ 0x006, 0x327, 0x00c, 0x3c5, 0x002, 0x00c, 0x387, 0x07c,
+ 0x187, 0x385, 0x08f, 0x244, 0x02a, 0x088, 0x187, 0x107,
+ 0x245, 0x02a, 0x082, 0x044, 0x189, 0x000, 0x081, 0x18a,
+ 0x208, 0x244, 0x02a, 0x2c7, 0x00c, 0x020, 0x181, 0x006,
+ 0x364, 0x026, 0x002, 0x001, 0x007, 0x364, 0x012, 0x040,
+ 0x001, 0x3f8, 0x020, 0x181, 0x006, 0x042, 0x327, 0x00d,
+ 0x000, 0x16e, 0x042, 0x327, 0x012, 0x000, 0x169, 0x042,
+ 0x197, 0x264, 0x001, 0x009, 0x011, 0x264, 0x002, 0x009,
+ 0x015, 0x264, 0x004, 0x009, 0x017, 0x264, 0x010, 0x009,
+ 0x01b, 0x0a0, 0x000, 0x154, 0x325, 0x21d, 0x2c7, 0x054,
+ 0x384, 0x3fe, 0x000, 0x3e7, 0x0c0, 0x054, 0x384, 0x3fd,
+ 0x000, 0x3e1, 0x324, 0x21f, 0x2c7, 0x054, 0x384, 0x3fb,
+ 0x000, 0x3d9, 0x044, 0x048, 0x008, 0x3c6, 0x100, 0x02b,
+ 0x0c0, 0x000, 0x364, 0x000, 0x020, 0x009, 0x3f5, 0x000,
+ 0x3db, 0x042, 0x040, 0x197, 0x001, 0x00d, 0x111, 0x001,
+ 0x028, 0x111, 0x001, 0x029, 0x111, 0x001, 0x02a, 0x000,
+ 0x041, 0x2c7, 0x254, 0x3b7, 0x241, 0x387, 0x100, 0x044,
+ 0x1a9, 0x000, 0x121, 0x044, 0x192, 0x000, 0x384, 0x030,
+ 0x040, 0x132, 0x044, 0x192, 0x000, 0x3b8, 0x300, 0x284,
+ 0x043, 0x1a3, 0x285, 0x395, 0x080, 0x000, 0x01e, 0x2c7,
+ 0x254, 0x000, 0x01f, 0x2c7, 0x255, 0x000, 0x01b, 0x287,
+ 0x040, 0x3b7, 0x241, 0x044, 0x1bd, 0x000, 0x131, 0x044,
+ 0x192, 0x000, 0x3a7, 0x061, 0x1a6, 0x2a8, 0x284, 0x043,
+ 0x3a4, 0x200, 0x285, 0x3b7, 0x240, 0x044, 0x1bd, 0x000,
+ 0x020, 0x181, 0x006, 0x362, 0x391, 0x001, 0x021, 0x181,
+ 0x006, 0x0c1, 0x391, 0x042, 0x0c0, 0x38f, 0x0c0, 0x390,
+ 0x080, 0x051, 0x2c0, 0x38f, 0x3c1, 0x000, 0x390, 0x3c6,
+ 0x0d8, 0x38f, 0x349, 0x38f, 0x38f, 0x34b, 0x390, 0x390,
+ 0x00a, 0x008, 0x3c0, 0x0e5, 0x38f, 0x3c1, 0x000, 0x390,
+ 0x3b7, 0x00b, 0x1a9, 0x3b5, 0x382, 0x222, 0x001, 0x005,
+ 0x101, 0x000, 0x3e0, 0x101, 0x051, 0x326, 0x38f, 0x3a4,
+ 0x0ff, 0x2e2, 0x0ef, 0x009, 0x029, 0x101, 0x051, 0x040,
+ 0x307, 0x38f, 0x197, 0x226, 0x2e2, 0x0be, 0x009, 0x01e,
+ 0x042, 0x101, 0x051, 0x326, 0x390, 0x3a4, 0x0ff, 0x2e2,
+ 0x0ad, 0x009, 0x013, 0x101, 0x050, 0x327, 0x390, 0x1b7,
+ 0x286, 0x262, 0x0de, 0x009, 0x009, 0x080, 0x040, 0x0c0,
+ 0x391, 0x020, 0x181, 0x006, 0x387, 0x0ff, 0x000, 0x3f8,
+ 0x043, 0x0a0, 0x1a7, 0x3a5, 0x045, 0x000, 0x071, 0x345,
+ 0x221, 0x233, 0x28a, 0x00a, 0x00c, 0x3a4, 0x300, 0x3c4,
+ 0x0ff, 0x027, 0x2c5, 0x027, 0x020, 0x181, 0x006, 0x327,
+ 0x059, 0x1a7, 0x042, 0x000, 0x05b, 0x2e4, 0x020, 0x001,
+ 0x012, 0x081, 0x18c, 0x245, 0x233, 0x2e4, 0x010, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x044, 0x11a, 0x001, 0x000,
+ 0x042, 0x041, 0x091, 0x189, 0x2e4, 0x002, 0x001, 0x015,
+ 0x3c5, 0x040, 0x233, 0x245, 0x024, 0x364, 0x026, 0x002,
+ 0x001, 0x01d, 0x3a7, 0x022, 0x397, 0x200, 0x044, 0x1b7,
+ 0x000, 0x000, 0x014, 0x3c4, 0x3bf, 0x233, 0x364, 0x026,
+ 0x002, 0x001, 0x00c, 0x208, 0x244, 0x024, 0x3a7, 0x022,
+ 0x397, 0x200, 0x044, 0x1a9, 0x000, 0x043, 0x081, 0x189,
+ 0x2e4, 0x001, 0x001, 0x009, 0x3a7, 0x020, 0x044, 0x1a9,
+ 0x000, 0x000, 0x008, 0x3a7, 0x020, 0x208, 0x044, 0x1b8,
+ 0x000, 0x020, 0x181, 0x006, 0x043, 0x0a0, 0x041, 0x020,
+ 0x181, 0x006, 0x2e2, 0x00f, 0x001, 0x060, 0x307, 0x2cf,
+ 0x0c0, 0x2d3, 0x120, 0x001, 0x05f, 0x00b, 0x018, 0x0c1,
+ 0x2d3, 0x3c4, 0x00f, 0x2cf, 0x3c5, 0x100, 0x200, 0x000,
+ 0x048, 0x0c0, 0x2d8, 0x3a4, 0x3f7, 0x001, 0x042, 0x0c1,
+ 0x2d8, 0x2c7, 0x2da, 0x000, 0x03c, 0x2e4, 0x008, 0x009,
+ 0x3f2, 0x2e2, 0x004, 0x009, 0x021, 0x364, 0x026, 0x002,
+ 0x001, 0x3cc, 0x364, 0x027, 0x010, 0x001, 0x3c7, 0x140,
+ 0x2f9, 0x009, 0x3c3, 0x140, 0x2d8, 0x001, 0x3bf, 0x327,
+ 0x2da, 0x364, 0x026, 0x002, 0x001, 0x3b8, 0x364, 0x2cf,
+ 0x073, 0x029, 0x1dc, 0x007, 0x3c4, 0x08f, 0x2cf, 0x2e4,
+ 0x003, 0x001, 0x010, 0x287, 0x1b5, 0x2c7, 0x2d4, 0x384,
+ 0x007, 0x183, 0x245, 0x2cf, 0x3c5, 0x040, 0x200, 0x0c0,
+ 0x2f9, 0x020, 0x181, 0x006, 0x30a, 0x2cf, 0x384, 0x020,
+ 0x000, 0x3f2, 0x347, 0x2cf, 0x2f9, 0x3c4, 0x3df, 0x2cf,
+ 0x000, 0x3f1, 0x042, 0x2c7, 0x269, 0x287, 0x384, 0x0ff,
+ 0x1a1, 0x002, 0x00b, 0x00b, 0x014, 0x247, 0x28d, 0x044,
+ 0x2fb, 0x002, 0x000, 0x005, 0x044, 0x125, 0x003, 0x140,
+ 0x269, 0x00b, 0x01b, 0x0a1, 0x044, 0x30f, 0x002, 0x140,
+ 0x269, 0x00b, 0x013, 0x364, 0x012, 0x040, 0x001, 0x3fd,
+ 0x347, 0x00e, 0x266, 0x347, 0x00f, 0x267, 0x397, 0x200,
+ 0x0a1, 0x044, 0x0fb, 0x002, 0x347, 0x28d, 0x268, 0x307,
+ 0x269, 0x197, 0x384, 0x003, 0x390, 0x266, 0x04c, 0x041,
+ 0x020, 0x181, 0x006, 0x042, 0x247, 0x265, 0x041, 0x397,
+ 0x007, 0x189, 0x395, 0x2ca, 0x1bb, 0x3a4, 0x007, 0x280,
+ 0x050, 0x043, 0x041, 0x1b7, 0x3a4, 0x00f, 0x280, 0x043,
+ 0x364, 0x265, 0x100, 0x009, 0x01b, 0x120, 0x00b, 0x008,
+ 0x3a4, 0x0ff, 0x2a8, 0x121, 0x000, 0x004, 0x3a4, 0x0ff,
+ 0x04d, 0x041, 0x044, 0x3f0, 0x002, 0x044, 0x3de, 0x002,
+ 0x044, 0x06e, 0x003, 0x020, 0x181, 0x006, 0x04c, 0x041,
+ 0x000, 0x3fb, 0x2c4, 0x2a1, 0x28e, 0x293, 0x2b2, 0x2d4,
+ 0x2e4, 0x347, 0x22e, 0x22f, 0x2c7, 0x22e, 0x2e4, 0x040,
+ 0x001, 0x01a, 0x3a7, 0x080, 0x044, 0x192, 0x000, 0x384,
+ 0x3c3, 0x327, 0x22e, 0x1b5, 0x3a4, 0x03c, 0x285, 0x3a7,
+ 0x080, 0x044, 0x1bd, 0x000, 0x347, 0x22f, 0x22e, 0x020,
+ 0x181, 0x006, 0x3c5, 0x004, 0x233, 0x364, 0x22e, 0x001,
+ 0x009, 0x005, 0x3c4, 0x3fb, 0x233, 0x364, 0x22e, 0x020,
+ 0x009, 0x005, 0x080, 0x000, 0x003, 0x081, 0x044, 0x254,
+ 0x001, 0x364, 0x22e, 0x001, 0x001, 0x007, 0x0c8, 0x291,
+ 0x044, 0x06e, 0x003, 0x307, 0x22e, 0x20a, 0x384, 0x001,
+ 0x044, 0x045, 0x008, 0x020, 0x181, 0x006, 0x2c7, 0x223,
+ 0x0c0, 0x224, 0x000, 0x01e, 0x309, 0x223, 0x002, 0x01d,
+ 0x003, 0x020, 0x001, 0x030, 0x362, 0x224, 0x001, 0x003,
+ 0x011, 0x112, 0x001, 0x01f, 0x112, 0x001, 0x057, 0x116,
+ 0x001, 0x06a, 0x114, 0x001, 0x055, 0x112, 0x001, 0x060,
+ 0x020, 0x181, 0x006, 0x20e, 0x384, 0x0ff, 0x000, 0x055,
+ 0x041, 0x229, 0x1b1, 0x042, 0x044, 0x1bd, 0x000, 0x000,
+ 0x3f1, 0x344, 0x21f, 0x02a, 0x083, 0x18b, 0x245, 0x02a,
+ 0x000, 0x005, 0x344, 0x21f, 0x029, 0x3b4, 0x3ff, 0x307,
+ 0x224, 0x262, 0x007, 0x00e, 0x3dd, 0x20a, 0x183, 0x247,
+ 0x02d, 0x30a, 0x224, 0x002, 0x006, 0x2c7, 0x02c, 0x000,
+ 0x009, 0x287, 0x189, 0x245, 0x02c, 0x1b5, 0x2c5, 0x02d,
+ 0x307, 0x223, 0x001, 0x00a, 0x3c5, 0x080, 0x02d, 0x3c4,
+ 0x37f, 0x02d, 0x000, 0x030, 0x3c5, 0x040, 0x02d, 0x3c4,
+ 0x3bf, 0x02d, 0x000, 0x028, 0x397, 0x013, 0x000, 0x004,
+ 0x397, 0x2c7, 0x300, 0x224, 0x140, 0x224, 0x009, 0x005,
+ 0x04d, 0x101, 0x0a0, 0x04d, 0x000, 0x016, 0x2c7, 0x2cb,
+ 0x000, 0x3a0, 0x140, 0x224, 0x009, 0x006, 0x2c7, 0x019,
+ 0x000, 0x00a, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x1ab,
+ 0x2c5, 0x018, 0x141, 0x224, 0x000, 0x38c, 0x042, 0x309,
+ 0x223, 0x002, 0x03e, 0x003, 0x043, 0x116, 0x001, 0x049,
+ 0x112, 0x001, 0x00e, 0x114, 0x029, 0x1dd, 0x007, 0x397,
+ 0x200, 0x300, 0x224, 0x04c, 0x041, 0x000, 0x3e5, 0x307,
+ 0x224, 0x262, 0x007, 0x02e, 0x1dd, 0x007, 0x364, 0x026,
+ 0x004, 0x029, 0x1dd, 0x007, 0x20a, 0x187, 0x0a3, 0x1a7,
+ 0x2a8, 0x2c4, 0x02a, 0x245, 0x02a, 0x327, 0x02e, 0x307,
+ 0x224, 0x20a, 0x00a, 0x009, 0x1b9, 0x307, 0x02f, 0x384,
+ 0x00f, 0x185, 0x225, 0x3b4, 0x3ff, 0x000, 0x022, 0x20a,
+ 0x384, 0x0ff, 0x04c, 0x041, 0x000, 0x344, 0x304, 0x21f,
+ 0x22e, 0x044, 0x192, 0x000, 0x040, 0x000, 0x33b, 0x30a,
+ 0x224, 0x002, 0x011, 0x307, 0x00d, 0x3c4, 0x3f7, 0x00d,
+ 0x347, 0x00f, 0x26a, 0x327, 0x00e, 0x247, 0x00d, 0x041,
+ 0x000, 0x39a, 0x327, 0x26a, 0x000, 0x3fb, 0x04e, 0x020,
+ 0x23b, 0x009, 0x020, 0x28e, 0x00a, 0x020, 0x03a, 0x00a,
+ 0x020, 0x0fe, 0x00a, 0x020, 0x3f3, 0x008, 0x020, 0x013,
+ 0x009, 0x020, 0x075, 0x00a, 0x0c1, 0x244, 0x3c5, 0x040,
+ 0x245, 0x3c4, 0x3bf, 0x200, 0x044, 0x172, 0x000, 0x020,
+ 0x0d3, 0x000, 0x020, 0x2c3, 0x008, 0x020, 0x375, 0x00b,
+ 0x0c1, 0x201, 0x045, 0x020, 0x04a, 0x009, 0x020, 0x04e,
+ 0x009, 0x020, 0x256, 0x002, 0x020, 0x17f, 0x009, 0x020,
+ 0x23f, 0x006, 0x020, 0x285, 0x00a, 0x020, 0x209, 0x00b,
+ 0x020, 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x394,
+ 0x00a, 0x020, 0x048, 0x00b, 0x020, 0x1dc, 0x007, 0x020,
+ 0x27f, 0x00a, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007,
+ 0x020, 0x295, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x277,
+ 0x00b, 0x020, 0x2ba, 0x00b, 0x020, 0x2d5, 0x006, 0x020,
+ 0x326, 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007,
+ 0x020, 0x35a, 0x006, 0x020, 0x0a4, 0x00b, 0x020, 0x38e,
+ 0x006, 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020,
+ 0x31e, 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007,
+ 0x020, 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060,
+ 0x007, 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020,
+ 0x2d1, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252,
+ 0x007, 0x020, 0x293, 0x007, 0x020, 0x2d8, 0x00b, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007,
+ 0x020, 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x30e, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007,
+ 0x020, 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5,
+ 0x008, 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287,
+ 0x264, 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5,
+ 0x020, 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004,
+ 0x3a4, 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009,
+ 0x093, 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe,
+ 0x052, 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de,
+ 0x007, 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c,
+ 0x100, 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c,
+ 0x3d5, 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066,
+ 0x004, 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b,
+ 0x262, 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4,
+ 0x3ef, 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009,
+ 0x007, 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200,
+ 0x004, 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001,
+ 0x028, 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344,
+ 0x21f, 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183,
+ 0x19b, 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001,
+ 0x004, 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5,
+ 0x080, 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044,
+ 0x001, 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5,
+ 0x004, 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044,
+ 0x097, 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d,
+ 0x001, 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390,
+ 0x1d7, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000,
+ 0x021, 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000,
+ 0x025, 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000,
+ 0x041, 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000,
+ 0x031, 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007,
+ 0x044, 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072,
+ 0x2c7, 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a,
+ 0x2c7, 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062,
+ 0x2c7, 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a,
+ 0x2c7, 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052,
+ 0x2c7, 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a,
+ 0x307, 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007,
+ 0x006, 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020,
+ 0x1de, 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3,
+ 0x052, 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c,
+ 0x2c5, 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7,
+ 0x3ff, 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf,
+ 0x354, 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222,
+ 0x355, 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a,
+ 0x006, 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357,
+ 0x020, 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4,
+ 0x00f, 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020,
+ 0x131, 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001,
+ 0x025, 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132,
+ 0x182, 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de,
+ 0x007, 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373,
+ 0x000, 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3,
+ 0x322, 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7,
+ 0x327, 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df,
+ 0x307, 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108,
+ 0x04c, 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b,
+ 0x364, 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f,
+ 0x181, 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5,
+ 0x080, 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7,
+ 0x392, 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5,
+ 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364,
+ 0x33c, 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000,
+ 0x018, 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140,
+ 0x04f, 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3,
+ 0x2c5, 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081,
+ 0x189, 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027,
+ 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053,
+ 0x3c4, 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004,
+ 0x020, 0x08d, 0x006, 0x2c7, 0x231, 0x2e4, 0x100, 0x001,
+ 0x045, 0x2e4, 0x001, 0x001, 0x020, 0x3b7, 0x213, 0x084,
+ 0x044, 0x1a9, 0x000, 0x3b7, 0x250, 0x084, 0x044, 0x1a9,
+ 0x000, 0x399, 0x305, 0x3a7, 0x022, 0x044, 0x1a9, 0x000,
+ 0x044, 0x322, 0x001, 0x3c5, 0x002, 0x024, 0x3c5, 0x008,
+ 0x292, 0x000, 0x020, 0x3c4, 0x3fd, 0x024, 0x364, 0x234,
+ 0x002, 0x001, 0x009, 0x3a7, 0x022, 0x399, 0x305, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x213, 0x084, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x250, 0x084, 0x044, 0x1b7, 0x000, 0x3c4, 0x007,
+ 0x292, 0x044, 0x06e, 0x003, 0x307, 0x231, 0x274, 0x200,
+ 0x001, 0x01b, 0x274, 0x010, 0x001, 0x00b, 0x387, 0x020,
+ 0x245, 0x021, 0x185, 0x245, 0x024, 0x000, 0x00b, 0x387,
+ 0x3df, 0x244, 0x021, 0x185, 0x385, 0x0ff, 0x244, 0x024,
+ 0x044, 0x03a, 0x00a, 0x045, 0x327, 0x230, 0x364, 0x233,
+ 0x004, 0x001, 0x003, 0x045, 0x0c2, 0x22a, 0x347, 0x275,
+ 0x270, 0x0c0, 0x27b, 0x264, 0x010, 0x009, 0x00a, 0x0c1,
+ 0x22a, 0x347, 0x274, 0x270, 0x3d7, 0x200, 0x27b, 0x327,
+ 0x230, 0x264, 0x004, 0x001, 0x012, 0x347, 0x277, 0x270,
+ 0x264, 0x010, 0x009, 0x007, 0x347, 0x276, 0x270, 0x384,
+ 0x00f, 0x112, 0x1b7, 0x000, 0x004, 0x3a4, 0x0ff, 0x2c7,
+ 0x225, 0x384, 0x00f, 0x262, 0x003, 0x001, 0x00b, 0x101,
+ 0x3c2, 0x100, 0x270, 0x34a, 0x225, 0x225, 0x000, 0x3f3,
+ 0x044, 0x237, 0x00a, 0x044, 0x35f, 0x009, 0x364, 0x22a,
+ 0x002, 0x001, 0x006, 0x347, 0x270, 0x272, 0x045, 0x347,
+ 0x270, 0x271, 0x045, 0x044, 0x2f6, 0x002, 0x009, 0x015,
+ 0x307, 0x241, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x3d7, 0x200, 0x27b,
+ 0x044, 0x35f, 0x009, 0x347, 0x241, 0x226, 0x0c1, 0x22a,
+ 0x020, 0x032, 0x009, 0x044, 0x2f6, 0x002, 0x009, 0x014,
+ 0x307, 0x243, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x0c0, 0x27b, 0x044,
+ 0x35f, 0x009, 0x347, 0x243, 0x226, 0x0c2, 0x22a, 0x020,
+ 0x032, 0x009, 0x327, 0x226, 0x2c7, 0x225, 0x3c4, 0x0ff,
+ 0x225, 0x1b7, 0x3a4, 0x007, 0x001, 0x00d, 0x34a, 0x225,
+ 0x225, 0x002, 0x008, 0x131, 0x009, 0x3fa, 0x044, 0x237,
+ 0x00a, 0x045, 0x0c1, 0x26f, 0x000, 0x050, 0x327, 0x01d,
+ 0x2e4, 0x380, 0x009, 0x00a, 0x13a, 0x00b, 0x00a, 0x3c5,
+ 0x008, 0x233, 0x000, 0x005, 0x3c4, 0x3f7, 0x233, 0x327,
+ 0x01d, 0x021, 0x2ed, 0x009, 0x364, 0x020, 0x004, 0x009,
+ 0x01a, 0x309, 0x232, 0x00b, 0x009, 0x347, 0x270, 0x271,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x271, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x04c, 0x347, 0x270, 0x271, 0x000,
+ 0x018, 0x309, 0x232, 0x001, 0x009, 0x347, 0x270, 0x272,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x272, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x089, 0x347, 0x270, 0x272, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x364, 0x26f, 0x002, 0x009,
+ 0x005, 0x3c4, 0x3fe, 0x200, 0x364, 0x020, 0x005, 0x029,
+ 0x2e4, 0x002, 0x364, 0x020, 0x010, 0x009, 0x0e9, 0x364,
+ 0x020, 0x100, 0x009, 0x12b, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3f7, 0x233, 0x364, 0x020, 0x001, 0x009, 0x00c, 0x364,
+ 0x020, 0x004, 0x009, 0x05d, 0x000, 0x3d8, 0x3c5, 0x080,
+ 0x020, 0x364, 0x233, 0x006, 0x001, 0x031, 0x081, 0x189,
+ 0x304, 0x232, 0x009, 0x017, 0x347, 0x271, 0x274, 0x080,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x085, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245, 0x232, 0x000,
+ 0x0c8, 0x347, 0x271, 0x276, 0x081, 0x189, 0x208, 0x244,
+ 0x232, 0x307, 0x22b, 0x197, 0x384, 0x007, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x000, 0x005, 0x044, 0x3f3, 0x008,
+ 0x3b7, 0x200, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x203, 0x387, 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020,
+ 0x385, 0x042, 0x384, 0x3ce, 0x247, 0x020, 0x044, 0x044,
+ 0x001, 0x000, 0x383, 0x0a1, 0x1aa, 0x2c5, 0x020, 0x364,
+ 0x233, 0x006, 0x001, 0x039, 0x081, 0x189, 0x304, 0x232,
+ 0x009, 0x017, 0x347, 0x272, 0x275, 0x080, 0x364, 0x233,
+ 0x004, 0x009, 0x003, 0x095, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x081, 0x189, 0x245, 0x232, 0x000, 0x0ce, 0x347,
+ 0x272, 0x277, 0x081, 0x189, 0x208, 0x244, 0x232, 0x307,
+ 0x22b, 0x197, 0x364, 0x233, 0x002, 0x001, 0x007, 0x193,
+ 0x384, 0x007, 0x385, 0x010, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x000, 0x008, 0x347, 0x272, 0x270, 0x044, 0x013,
+ 0x009, 0x081, 0x18a, 0x208, 0x244, 0x232, 0x0a3, 0x387,
+ 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020, 0x0a4, 0x1a7,
+ 0x3a5, 0x008, 0x285, 0x3b7, 0x304, 0x2a8, 0x284, 0x247,
+ 0x020, 0x0a0, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x044,
+ 0x2f6, 0x002, 0x009, 0x00f, 0x364, 0x2cf, 0x080, 0x009,
+ 0x00a, 0x0c3, 0x26f, 0x044, 0x1b6, 0x001, 0x020, 0x2e4,
+ 0x002, 0x044, 0x051, 0x001, 0x000, 0x308, 0x3c4, 0x3f7,
+ 0x233, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x083, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x3d7,
+ 0x200, 0x27b, 0x0c0, 0x271, 0x3b7, 0x300, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x221, 0x091, 0x189, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x201, 0x387, 0x0a6, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x021, 0x020, 0x3c4, 0x3ed, 0x020, 0x3b7,
+ 0x200, 0x387, 0x020, 0x044, 0x020, 0x00a, 0x009, 0x32a,
+ 0x347, 0x271, 0x270, 0x000, 0x055, 0x3a7, 0x071, 0x307,
+ 0x21d, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x233, 0x044,
+ 0x2f6, 0x002, 0x009, 0x007, 0x0c1, 0x2d3, 0x3c5, 0x100,
+ 0x200, 0x081, 0x18a, 0x245, 0x232, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x093, 0x0c2, 0x22a,
+ 0x044, 0x39c, 0x008, 0x0c0, 0x27b, 0x0c0, 0x272, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1, 0x397,
+ 0x205, 0x247, 0x27f, 0x044, 0x1bd, 0x000, 0x3d5, 0x204,
+ 0x020, 0x3c4, 0x2ff, 0x020, 0x0a0, 0x387, 0x021, 0x184,
+ 0x044, 0x020, 0x00a, 0x009, 0x336, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3fe, 0x200, 0x3c4, 0x3fe,
+ 0x26f, 0x344, 0x21f, 0x232, 0x0c0, 0x020, 0x0a3, 0x1ab,
+ 0x2c5, 0x020, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000,
+ 0x044, 0x3b5, 0x000, 0x044, 0x3f0, 0x000, 0x0a9, 0x1ab,
+ 0x09f, 0x18a, 0x208, 0x304, 0x055, 0x225, 0x044, 0x394,
+ 0x00a, 0x307, 0x055, 0x19b, 0x384, 0x003, 0x227, 0x1a3,
+ 0x225, 0x044, 0x0a4, 0x00b, 0x0a7, 0x020, 0x022, 0x008,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x006, 0x387, 0x07f,
+ 0x000, 0x013, 0x399, 0x240, 0x385, 0x07f, 0x306, 0x270,
+ 0x001, 0x06c, 0x399, 0x240, 0x327, 0x270, 0x3a4, 0x380,
+ 0x226, 0x001, 0x015, 0x327, 0x220, 0x324, 0x232, 0x009,
+ 0x019, 0x364, 0x232, 0x001, 0x001, 0x02b, 0x327, 0x01d,
+ 0x307, 0x01b, 0x190, 0x282, 0x003, 0x041, 0x307, 0x01b,
+ 0x191, 0x380, 0x010, 0x282, 0x003, 0x034, 0x000, 0x019,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x00a, 0x327, 0x270,
+ 0x384, 0x07f, 0x3a4, 0x07f, 0x000, 0x008, 0x327, 0x270,
+ 0x3a4, 0x07f, 0x001, 0x005, 0x286, 0x001, 0x02f, 0x307,
+ 0x270, 0x384, 0x07f, 0x386, 0x07f, 0x009, 0x00a, 0x387,
+ 0x080, 0x382, 0x07f, 0x240, 0x270, 0x000, 0x01c, 0x141,
+ 0x270, 0x364, 0x270, 0x07f, 0x009, 0x015, 0x000, 0x013,
+ 0x3c0, 0x040, 0x270, 0x000, 0x00e, 0x307, 0x270, 0x196,
+ 0x384, 0x00f, 0x266, 0x009, 0x001, 0x008, 0x3c0, 0x080,
+ 0x270, 0x044, 0x35f, 0x009, 0x045, 0x309, 0x232, 0x00b,
+ 0x013, 0x347, 0x271, 0x270, 0x374, 0x27b, 0x200, 0x009,
+ 0x005, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x020,
+ 0x0b7, 0x009, 0x327, 0x232, 0x1a2, 0x023, 0x0b7, 0x009,
+ 0x307, 0x270, 0x196, 0x119, 0x021, 0x0b7, 0x009, 0x307,
+ 0x232, 0x181, 0x023, 0x0b7, 0x009, 0x364, 0x270, 0x080,
+ 0x029, 0x0b7, 0x009, 0x083, 0x18c, 0x245, 0x232, 0x3c4,
+ 0x3f7, 0x233, 0x3c4, 0x380, 0x270, 0x3c0, 0x080, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x09c, 0x009, 0x000, 0x000, 0x000, 0x002, 0x000,
+ 0x00a, 0x000, 0x02a, 0x000, 0x0aa, 0x002, 0x0aa, 0x00a,
+ 0x0aa, 0x02a, 0x0aa, 0x06a, 0x0aa, 0x07a, 0x0aa, 0x07e,
+ 0x0aa, 0x07f, 0x0aa, 0x07f, 0x0ba, 0x07f, 0x0be, 0x07f,
+ 0x0fe, 0x07f, 0x0ff, 0x007, 0x000, 0x017, 0x001, 0x027,
+ 0x007, 0x037, 0x00f, 0x047, 0x03f, 0x080, 0x0ff, 0x364,
+ 0x270, 0x07f, 0x307, 0x270, 0x384, 0x07f, 0x009, 0x00d,
+ 0x089, 0x186, 0x306, 0x270, 0x001, 0x007, 0x3c5, 0x001,
+ 0x232, 0x000, 0x005, 0x3c4, 0x3fe, 0x232, 0x347, 0x270,
+ 0x27c, 0x3c4, 0x007, 0x27c, 0x374, 0x27b, 0x200, 0x001,
+ 0x00a, 0x364, 0x233, 0x008, 0x001, 0x005, 0x044, 0x0fe,
+ 0x00a, 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384,
+ 0x3f8, 0x305, 0x27c, 0x044, 0x1bd, 0x000, 0x307, 0x270,
+ 0x192, 0x384, 0x00f, 0x3b9, 0x200, 0x324, 0x233, 0x001,
+ 0x014, 0x229, 0x387, 0x009, 0x189, 0x395, 0x333, 0x280,
+ 0x051, 0x2c7, 0x27d, 0x1c7, 0x27d, 0x101, 0x051, 0x2c5,
+ 0x27d, 0x000, 0x011, 0x0c0, 0x27d, 0x384, 0x00f, 0x001,
+ 0x00b, 0x349, 0x27d, 0x27d, 0x3c5, 0x001, 0x27d, 0x111,
+ 0x000, 0x3f7, 0x0a2, 0x325, 0x27b, 0x307, 0x27d, 0x044,
+ 0x1bd, 0x000, 0x374, 0x27b, 0x200, 0x001, 0x02c, 0x347,
+ 0x270, 0x27a, 0x3c4, 0x07f, 0x27a, 0x0c0, 0x279, 0x387,
+ 0x009, 0x189, 0x395, 0x353, 0x051, 0x041, 0x101, 0x051,
+ 0x2c7, 0x279, 0x101, 0x043, 0x322, 0x27a, 0x00b, 0x004,
+ 0x000, 0x3f4, 0x1c7, 0x279, 0x0a3, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x384, 0x0ff, 0x305, 0x279, 0x044, 0x1bd,
+ 0x000, 0x327, 0x220, 0x324, 0x232, 0x009, 0x022, 0x0c0,
+ 0x27e, 0x307, 0x270, 0x197, 0x00a, 0x004, 0x000, 0x005,
+ 0x3c5, 0x018, 0x27e, 0x0a4, 0x222, 0x2c5, 0x27e, 0x0a4,
+ 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x3e0, 0x3c4,
+ 0x01f, 0x27e, 0x305, 0x27e, 0x044, 0x1bd, 0x000, 0x045,
+ 0x044, 0x1a9, 0x000, 0x083, 0x18c, 0x208, 0x244, 0x232,
+ 0x3c5, 0x001, 0x00c, 0x0a3, 0x325, 0x27b, 0x088, 0x044,
+ 0x1a9, 0x000, 0x387, 0x3f8, 0x044, 0x1b8, 0x000, 0x020,
+ 0x2f6, 0x002, 0x3c7, 0x020, 0x280, 0x3b7, 0x215, 0x044,
+ 0x192, 0x000, 0x247, 0x247, 0x151, 0x280, 0x001, 0x016,
+ 0x0a1, 0x044, 0x182, 0x000, 0x3b7, 0x215, 0x044, 0x192,
+ 0x000, 0x306, 0x247, 0x001, 0x3f1, 0x0a2, 0x364, 0x22b,
+ 0x003, 0x001, 0x003, 0x0a5, 0x020, 0x182, 0x000, 0x1f6,
+ 0x185, 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122,
+ 0x0ff, 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194,
+ 0x0ce, 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020,
+ 0x009, 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9,
+ 0x286, 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x05f,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051,
+ 0x2c7, 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f,
+ 0x040, 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b,
+ 0x005, 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4,
+ 0x00f, 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288,
+ 0x286, 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b,
+ 0x3fc, 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285,
+ 0x307, 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd,
+ 0x000, 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2,
+ 0x12c, 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002,
+ 0x005, 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b,
+ 0x1f0, 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081,
+ 0x138, 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f,
+ 0x080, 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d,
+ 0x1e0, 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b,
+ 0x347, 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044,
+ 0x1bc, 0x00a, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364,
+ 0x233, 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff,
+ 0x242, 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289,
+ 0x397, 0x00a, 0x189, 0x395, 0x0ca, 0x044, 0x2ea, 0x001,
+ 0x009, 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9,
+ 0x121, 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f,
+ 0x282, 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000,
+ 0x3ed, 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233,
+ 0x002, 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff,
+ 0x281, 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366,
+ 0x282, 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282,
+ 0x044, 0x1bc, 0x00a, 0x001, 0x00d, 0x141, 0x28b, 0x362,
+ 0x28b, 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5,
+ 0x362, 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397,
+ 0x00a, 0x189, 0x395, 0x0d8, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005,
+ 0x105, 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f,
+ 0x121, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042,
+ 0x003, 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387,
+ 0x00a, 0x189, 0x395, 0x0d3, 0x300, 0x28a, 0x050, 0x247,
+ 0x28a, 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a,
+ 0x044, 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282,
+ 0x281, 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045,
+ 0x000, 0x301, 0x103, 0x307, 0x006, 0x10e, 0x21e, 0x33e,
+ 0x13c, 0x038, 0x078, 0x1f8, 0x0f0, 0x000, 0x001, 0x003,
+ 0x002, 0x006, 0x187, 0x227, 0x387, 0x00a, 0x189, 0x395,
+ 0x1c8, 0x300, 0x227, 0x111, 0x050, 0x247, 0x24c, 0x1d7,
+ 0x24c, 0x384, 0x0ff, 0x285, 0x044, 0x226, 0x00a, 0x364,
+ 0x22a, 0x001, 0x001, 0x00e, 0x3d5, 0x200, 0x232, 0x040,
+ 0x1aa, 0x285, 0x3b7, 0x220, 0x044, 0x1bd, 0x000, 0x042,
+ 0x364, 0x22a, 0x002, 0x001, 0x022, 0x040, 0x3a7, 0x073,
+ 0x044, 0x192, 0x000, 0x384, 0x31f, 0x044, 0x226, 0x00a,
+ 0x1a4, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000, 0x3a7,
+ 0x070, 0x044, 0x192, 0x000, 0x19a, 0x18a, 0x043, 0x285,
+ 0x3a7, 0x070, 0x044, 0x1bd, 0x000, 0x045, 0x040, 0x387,
+ 0x00a, 0x189, 0x395, 0x1d5, 0x300, 0x278, 0x051, 0x042,
+ 0x045, 0x00a, 0x058, 0x026, 0x015, 0x044, 0x032, 0x0c0,
+ 0x278, 0x0a0, 0x397, 0x00a, 0x189, 0x395, 0x231, 0x280,
+ 0x050, 0x247, 0x228, 0x3c4, 0x00f, 0x228, 0x347, 0x228,
+ 0x229, 0x040, 0x307, 0x278, 0x001, 0x008, 0x340, 0x229,
+ 0x228, 0x111, 0x000, 0x3fa, 0x042, 0x193, 0x0c0, 0x227,
+ 0x347, 0x225, 0x226, 0x141, 0x227, 0x366, 0x227, 0x00e,
+ 0x001, 0x00b, 0x342, 0x228, 0x226, 0x007, 0x3f6, 0x140,
+ 0x226, 0x001, 0x011, 0x121, 0x2e2, 0x006, 0x009, 0x3cc,
+ 0x141, 0x278, 0x366, 0x278, 0x005, 0x009, 0x3c4, 0x0a1,
+ 0x000, 0x006, 0x044, 0x1da, 0x00a, 0x0a0, 0x045, 0x044,
+ 0x333, 0x008, 0x020, 0x181, 0x006, 0x345, 0x21d, 0x232,
+ 0x364, 0x23f, 0x001, 0x029, 0x2e4, 0x002, 0x2c7, 0x234,
+ 0x399, 0x200, 0x245, 0x233, 0x364, 0x234, 0x020, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x083, 0x18c, 0x208, 0x244,
+ 0x036, 0x2e4, 0x008, 0x001, 0x007, 0x345, 0x220, 0x232,
+ 0x000, 0x007, 0x081, 0x18b, 0x208, 0x244, 0x232, 0x307,
+ 0x235, 0x226, 0x3a4, 0x001, 0x001, 0x010, 0x327, 0x234,
+ 0x2e4, 0x001, 0x009, 0x041, 0x041, 0x044, 0x20d, 0x000,
+ 0x044, 0x2f3, 0x000, 0x043, 0x0a3, 0x1ad, 0x3a5, 0x039,
+ 0x326, 0x236, 0x001, 0x02f, 0x3a7, 0x0bf, 0x1a7, 0x3a5,
+ 0x039, 0x326, 0x236, 0x001, 0x026, 0x327, 0x234, 0x307,
+ 0x235, 0x286, 0x384, 0x002, 0x001, 0x01d, 0x2e4, 0x002,
+ 0x009, 0x051, 0x041, 0x0a1, 0x1ad, 0x2c5, 0x024, 0x307,
+ 0x024, 0x19a, 0x002, 0x005, 0x2a8, 0x2c4, 0x024, 0x044,
+ 0x3b5, 0x000, 0x044, 0x0f2, 0x001, 0x044, 0x3f0, 0x000,
+ 0x043, 0x000, 0x067, 0x364, 0x026, 0x001, 0x009, 0x3c6,
+ 0x041, 0x0a0, 0x044, 0x301, 0x000, 0x364, 0x26f, 0x002,
+ 0x021, 0x310, 0x00a, 0x044, 0x290, 0x001, 0x0c1, 0x26f,
+ 0x043, 0x307, 0x020, 0x193, 0x384, 0x007, 0x110, 0x001,
+ 0x00c, 0x364, 0x232, 0x010, 0x009, 0x007, 0x044, 0x044,
+ 0x001, 0x000, 0x3a3, 0x3c4, 0x3fe, 0x26f, 0x3c5, 0x001,
+ 0x200, 0x3c5, 0x010, 0x020, 0x3c4, 0x31f, 0x020, 0x000,
+ 0x395, 0x364, 0x026, 0x002, 0x009, 0x3c5, 0x041, 0x044,
+ 0x003, 0x001, 0x043, 0x307, 0x020, 0x197, 0x384, 0x007,
+ 0x110, 0x001, 0x00f, 0x364, 0x232, 0x020, 0x009, 0x00a,
+ 0x044, 0x051, 0x001, 0x3c5, 0x001, 0x027, 0x000, 0x3ab,
+ 0x3c4, 0x3fc, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x100,
+ 0x020, 0x087, 0x188, 0x208, 0x244, 0x020, 0x000, 0x39b,
+ 0x044, 0x11a, 0x001, 0x307, 0x234, 0x306, 0x235, 0x384,
+ 0x003, 0x021, 0x2e4, 0x002, 0x347, 0x234, 0x235, 0x366,
+ 0x26f, 0x002, 0x021, 0x2e4, 0x002, 0x364, 0x234, 0x003,
+ 0x021, 0x2e4, 0x002, 0x327, 0x21c, 0x324, 0x020, 0x021,
+ 0x385, 0x00a, 0x020, 0x247, 0x002, 0x364, 0x23f, 0x001,
+ 0x021, 0x38e, 0x00a, 0x020, 0x247, 0x002, 0x345, 0x21d,
+ 0x232, 0x020, 0x247, 0x002, 0x2c7, 0x241, 0x307, 0x242,
+ 0x286, 0x001, 0x092, 0x0c1, 0x22a, 0x327, 0x242, 0x326,
+ 0x241, 0x3a4, 0x0ff, 0x001, 0x005, 0x3c4, 0x38f, 0x020,
+ 0x347, 0x241, 0x242, 0x347, 0x241, 0x226, 0x307, 0x21d,
+ 0x304, 0x241, 0x001, 0x00a, 0x0c3, 0x22a, 0x347, 0x241,
+ 0x243, 0x044, 0x39f, 0x002, 0x327, 0x241, 0x3a4, 0x0ff,
+ 0x2c7, 0x225, 0x3c4, 0x300, 0x230, 0x2c5, 0x230, 0x044,
+ 0x099, 0x00b, 0x081, 0x18b, 0x304, 0x226, 0x009, 0x00d,
+ 0x364, 0x232, 0x100, 0x009, 0x008, 0x081, 0x18a, 0x208,
+ 0x044, 0x1b8, 0x000, 0x044, 0x186, 0x001, 0x327, 0x226,
+ 0x003, 0x032, 0x044, 0x2f6, 0x002, 0x001, 0x019, 0x044,
+ 0x237, 0x00a, 0x347, 0x241, 0x226, 0x081, 0x306, 0x22a,
+ 0x384, 0x003, 0x001, 0x005, 0x347, 0x243, 0x226, 0x044,
+ 0x032, 0x009, 0x002, 0x02d, 0x000, 0x031, 0x044, 0x237,
+ 0x00a, 0x009, 0x01c, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x001, 0x023, 0x364, 0x233, 0x001, 0x000,
+ 0x01e, 0x045, 0x044, 0x186, 0x001, 0x3c5, 0x003, 0x22a,
+ 0x347, 0x241, 0x243, 0x000, 0x3c7, 0x3a4, 0x3ff, 0x001,
+ 0x00e, 0x327, 0x21c, 0x324, 0x020, 0x009, 0x006, 0x043,
+ 0x3a7, 0x0ff, 0x041, 0x0c0, 0x270, 0x044, 0x082, 0x00b,
+ 0x3c4, 0x3fc, 0x22a, 0x327, 0x21c, 0x324, 0x020, 0x009,
+ 0x010, 0x307, 0x270, 0x003, 0x005, 0x020, 0x181, 0x006,
+ 0x043, 0x3a7, 0x0ff, 0x041, 0x020, 0x181, 0x006, 0x045,
+ 0x2c7, 0x243, 0x307, 0x250, 0x286, 0x001, 0x3de, 0x0c2,
+ 0x22a, 0x041, 0x044, 0x39f, 0x002, 0x043, 0x2c7, 0x226,
+ 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x0ff, 0x230, 0x1a7,
+ 0x2c5, 0x230, 0x307, 0x21d, 0x304, 0x243, 0x001, 0x013,
+ 0x3c4, 0x0ff, 0x243, 0x347, 0x243, 0x241, 0x347, 0x243,
+ 0x242, 0x3c5, 0x003, 0x22a, 0x044, 0x186, 0x001, 0x000,
+ 0x005, 0x044, 0x191, 0x001, 0x327, 0x243, 0x003, 0x394,
+ 0x000, 0x362, 0x364, 0x232, 0x004, 0x001, 0x008, 0x044,
+ 0x232, 0x000, 0x3c4, 0x3fb, 0x232, 0x364, 0x232, 0x008,
+ 0x001, 0x008, 0x044, 0x326, 0x000, 0x3c4, 0x3f7, 0x232,
+ 0x045, 0x3b7, 0x221, 0x083, 0x189, 0x044, 0x1a9, 0x000,
+ 0x083, 0x020, 0x1b7, 0x000, 0x2c7, 0x24d, 0x364, 0x233,
+ 0x002, 0x001, 0x019, 0x3b9, 0x381, 0x121, 0x081, 0x18a,
+ 0x225, 0x304, 0x24d, 0x001, 0x005, 0x044, 0x1ff, 0x00b,
+ 0x140, 0x24d, 0x00b, 0x008, 0x1a3, 0x044, 0x1ff, 0x00b,
+ 0x000, 0x012, 0x307, 0x220, 0x284, 0x001, 0x00d, 0x3c4,
+ 0x33f, 0x232, 0x1b6, 0x3a4, 0x0c0, 0x2c5, 0x232, 0x020,
+ 0x18a, 0x00b, 0x327, 0x24d, 0x364, 0x020, 0x070, 0x001,
+ 0x00a, 0x399, 0x383, 0x101, 0x326, 0x22b, 0x234, 0x001,
+ 0x02e, 0x3c5, 0x080, 0x233, 0x3d5, 0x200, 0x232, 0x364,
+ 0x026, 0x001, 0x001, 0x00a, 0x3c5, 0x004, 0x232, 0x044,
+ 0x20d, 0x000, 0x307, 0x24d, 0x307, 0x24d, 0x384, 0x007,
+ 0x001, 0x09a, 0x113, 0x001, 0x0a8, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3f8, 0x103, 0x044, 0x1bd, 0x000,
+ 0x3c4, 0x3f7, 0x021, 0x000, 0x0af, 0x0a7, 0x1a7, 0x324,
+ 0x020, 0x001, 0x014, 0x327, 0x24d, 0x326, 0x22b, 0x397,
+ 0x370, 0x364, 0x233, 0x002, 0x001, 0x006, 0x087, 0x18b,
+ 0x385, 0x070, 0x234, 0x001, 0x02b, 0x3c5, 0x100, 0x233,
+ 0x364, 0x026, 0x002, 0x001, 0x008, 0x3c5, 0x008, 0x232,
+ 0x044, 0x3b5, 0x000, 0x307, 0x24d, 0x384, 0x070, 0x001,
+ 0x091, 0x382, 0x030, 0x001, 0x0a6, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x018, 0x044, 0x1bd,
+ 0x000, 0x3c4, 0x3f7, 0x024, 0x000, 0x0a4, 0x347, 0x24d,
+ 0x22b, 0x364, 0x233, 0x006, 0x001, 0x033, 0x307, 0x22b,
+ 0x364, 0x233, 0x002, 0x001, 0x023, 0x364, 0x233, 0x080,
+ 0x001, 0x00a, 0x307, 0x24d, 0x197, 0x384, 0x007, 0x044,
+ 0x3a4, 0x008, 0x364, 0x233, 0x100, 0x001, 0x00c, 0x307,
+ 0x24d, 0x19b, 0x384, 0x007, 0x385, 0x010, 0x044, 0x3a4,
+ 0x008, 0x3c4, 0x27f, 0x233, 0x000, 0x00b, 0x3c5, 0x003,
+ 0x22a, 0x197, 0x044, 0x39c, 0x008, 0x003, 0x298, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x021, 0x181,
+ 0x006, 0x045, 0x3b7, 0x301, 0x087, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7,
+ 0x021, 0x000, 0x36c, 0x3b7, 0x301, 0x044, 0x192, 0x000,
+ 0x384, 0x3f8, 0x105, 0x044, 0x1bd, 0x000, 0x3b7, 0x211,
+ 0x082, 0x044, 0x1a9, 0x000, 0x3c5, 0x008, 0x021, 0x344,
+ 0x21f, 0x021, 0x3b7, 0x250, 0x088, 0x044, 0x1a9, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x347,
+ 0x3b7, 0x301, 0x387, 0x3c7, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x060, 0x083, 0x044, 0x1b7, 0x000, 0x3b7, 0x070, 0x083,
+ 0x18a, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x024, 0x000,
+ 0x36f, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x028, 0x044, 0x1bd, 0x000, 0x3c5, 0x048, 0x024,
+ 0x3a7, 0x060, 0x083, 0x044, 0x1a9, 0x000, 0x3b7, 0x070,
+ 0x083, 0x18a, 0x044, 0x1a9, 0x000, 0x000, 0x351, 0x307,
+ 0x22b, 0x284, 0x2a8, 0x2c4, 0x24d, 0x245, 0x24d, 0x2a8,
+ 0x045, 0x2f4, 0x200, 0x001, 0x008, 0x364, 0x2cf, 0x007,
+ 0x029, 0x1dc, 0x007, 0x287, 0x385, 0x300, 0x208, 0x001,
+ 0x04b, 0x364, 0x33b, 0x010, 0x029, 0x1dc, 0x007, 0x3c5,
+ 0x002, 0x00c, 0x287, 0x384, 0x0ff, 0x305, 0x21d, 0x1b7,
+ 0x2e4, 0x001, 0x001, 0x01d, 0x040, 0x384, 0x007, 0x117,
+ 0x001, 0x00c, 0x3c5, 0x040, 0x02d, 0x081, 0x044, 0x189,
+ 0x000, 0x3c4, 0x3bf, 0x02d, 0x042, 0x247, 0x029, 0x364,
+ 0x018, 0x001, 0x009, 0x005, 0x3c5, 0x010, 0x021, 0x2e4,
+ 0x002, 0x001, 0x017, 0x264, 0x080, 0x001, 0x005, 0x0a4,
+ 0x1a7, 0x285, 0x040, 0x3c5, 0x080, 0x02d, 0x081, 0x044,
+ 0x189, 0x000, 0x3c4, 0x37f, 0x02d, 0x042, 0x247, 0x02a,
+ 0x000, 0x014, 0x38e, 0x3ff, 0x1b8, 0x00a, 0x00a, 0x244,
+ 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4, 0x3fd, 0x00c, 0x120,
+ 0x001, 0x004, 0x244, 0x02a, 0x020, 0x181, 0x006, 0x289,
+ 0x00a, 0x004, 0x2c7, 0x251, 0x22e, 0x364, 0x026, 0x001,
+ 0x021, 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x002, 0x009,
+ 0x01b, 0x3c4, 0x2ff, 0x232, 0x120, 0x001, 0x012, 0x3d5,
+ 0x300, 0x232, 0x044, 0x099, 0x00b, 0x32a, 0x026, 0x00a,
+ 0x008, 0x044, 0x20d, 0x000, 0x044, 0x232, 0x000, 0x020,
+ 0x181, 0x006, 0x042, 0x307, 0x240, 0x20a, 0x040, 0x020,
+ 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x256, 0x22e,
+ 0x364, 0x026, 0x002, 0x021, 0x181, 0x006, 0x041, 0x3a4,
+ 0x01f, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x043, 0x1b7,
+ 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x020, 0x181, 0x006,
+ 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044, 0x340, 0x001,
+ 0x247, 0x249, 0x054, 0x347, 0x249, 0x248, 0x2c7, 0x23f,
+ 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f, 0x001, 0x001,
+ 0x006, 0x245, 0x057, 0x000, 0x005, 0x208, 0x244, 0x057,
+ 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f, 0x002, 0x001,
+ 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001, 0x000, 0x005,
+ 0x208, 0x244, 0x057, 0x020, 0x181, 0x006, 0x2c7, 0x246,
+ 0x3b7, 0x300, 0x044, 0x192, 0x000, 0x384, 0x33f, 0x327,
+ 0x246, 0x3a4, 0x0c0, 0x285, 0x3b7, 0x300, 0x044, 0x1bd,
+ 0x000, 0x3b7, 0x200, 0x044, 0x192, 0x000, 0x0a3, 0x1a9,
+ 0x2a8, 0x284, 0x040, 0x307, 0x246, 0x384, 0x030, 0x264,
+ 0x010, 0x001, 0x012, 0x040, 0x041, 0x3b7, 0x201, 0x044,
+ 0x192, 0x000, 0x384, 0x3e0, 0x385, 0x006, 0x044, 0x1bd,
+ 0x000, 0x043, 0x042, 0x185, 0x2a8, 0x284, 0x227, 0x042,
+ 0x285, 0x3b7, 0x200, 0x044, 0x1bd, 0x000, 0x307, 0x246,
+ 0x384, 0x007, 0x044, 0x375, 0x00b, 0x3a7, 0x073, 0x307,
+ 0x246, 0x184, 0x00a, 0x014, 0x044, 0x192, 0x000, 0x384,
+ 0x3e2, 0x327, 0x246, 0x1b5, 0x3a4, 0x01c, 0x285, 0x3a7,
+ 0x073, 0x044, 0x1bd, 0x000, 0x000, 0x006, 0x081, 0x044,
+ 0x1a9, 0x000, 0x020, 0x181, 0x006, 0x3c5, 0x001, 0x00c,
+ 0x187, 0x0a7, 0x1a7, 0x2a8, 0x324, 0x00c, 0x225, 0x2c7,
+ 0x00c, 0x045, 0x04e, 0x087, 0x011, 0x054, 0x084
+
+}; /* end fm10000_serdes_spico_code_prd2 */
+
+
+
+/* Spico Swap code (production version 2)
+ * source file: serdes.0x2055_0045.swap
+ * Note: if swap code exists, if must be loaded on top of
+ * the sbus master image.
+ */
+
+const uint32_t fm10000_serdes_swap_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_swap_code_size_prd2 = 7842;
+
+const uint16_t fm10000_serdes_swap_code_prd2[] = {
+ 0x01e, 0x09d, 0x002, 0x000, 0x000, 0x008, 0x00f, 0x07b,
+ 0x020, 0x01e, 0x00f, 0x069, 0x01d, 0x029, 0x00b, 0x01d,
+ 0x02c, 0x382, 0x04e, 0x020, 0x23b, 0x009, 0x020, 0x28e,
+ 0x00a, 0x020, 0x03a, 0x00a, 0x020, 0x0fe, 0x00a, 0x020,
+ 0x3f3, 0x008, 0x020, 0x013, 0x009, 0x020, 0x075, 0x00a,
+ 0x0c1, 0x244, 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200,
+ 0x044, 0x172, 0x000, 0x020, 0x0d3, 0x000, 0x020, 0x2c3,
+ 0x008, 0x020, 0x375, 0x00b, 0x0c1, 0x201, 0x045, 0x020,
+ 0x04a, 0x009, 0x020, 0x04e, 0x009, 0x020, 0x256, 0x002,
+ 0x020, 0x17f, 0x009, 0x020, 0x23f, 0x006, 0x020, 0x285,
+ 0x00a, 0x020, 0x209, 0x00b, 0x020, 0x257, 0x006, 0x020,
+ 0x117, 0x008, 0x020, 0x394, 0x00a, 0x020, 0x048, 0x00b,
+ 0x020, 0x1dc, 0x007, 0x020, 0x27f, 0x00a, 0x020, 0x2d2,
+ 0x006, 0x020, 0x1e2, 0x007, 0x020, 0x295, 0x00b, 0x020,
+ 0x3a8, 0x001, 0x020, 0x277, 0x00b, 0x020, 0x2ba, 0x00b,
+ 0x020, 0x2d5, 0x006, 0x020, 0x326, 0x006, 0x020, 0x350,
+ 0x006, 0x020, 0x1dc, 0x007, 0x020, 0x35a, 0x006, 0x020,
+ 0x0a4, 0x00b, 0x020, 0x38e, 0x006, 0x020, 0x011, 0x007,
+ 0x020, 0x016, 0x007, 0x020, 0x31e, 0x007, 0x020, 0x324,
+ 0x007, 0x020, 0x3b6, 0x007, 0x020, 0x02a, 0x007, 0x020,
+ 0x041, 0x007, 0x020, 0x060, 0x007, 0x020, 0x06d, 0x007,
+ 0x020, 0x072, 0x007, 0x020, 0x2d1, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x252, 0x007, 0x020, 0x293, 0x007,
+ 0x020, 0x2d8, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x16f, 0x007, 0x020, 0x185, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x30e, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x077, 0x007, 0x020, 0x0b1, 0x007, 0x020,
+ 0x103, 0x007, 0x020, 0x1c5, 0x008, 0x020, 0x1dc, 0x007,
+ 0x020, 0x168, 0x007, 0x287, 0x264, 0x004, 0x001, 0x049,
+ 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020, 0x33b, 0x18e, 0x00a,
+ 0x013, 0x044, 0x04c, 0x004, 0x3a4, 0x001, 0x001, 0x014,
+ 0x364, 0x052, 0x001, 0x009, 0x093, 0x044, 0x0bc, 0x004,
+ 0x000, 0x08e, 0x3c4, 0x2ff, 0x053, 0x3c4, 0x030, 0x027,
+ 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052, 0x000, 0x081, 0x043,
+ 0x327, 0x33b, 0x020, 0x1de, 0x007, 0x2e4, 0x080, 0x009,
+ 0x00d, 0x3d5, 0x220, 0x33c, 0x100, 0x009, 0x06e, 0x044,
+ 0x3b6, 0x003, 0x000, 0x06c, 0x3d5, 0x120, 0x33c, 0x100,
+ 0x009, 0x022, 0x044, 0x066, 0x004, 0x000, 0x061, 0x384,
+ 0x003, 0x3c4, 0x3df, 0x33b, 0x262, 0x003, 0x001, 0x3d9,
+ 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef, 0x027, 0x2e4, 0x040,
+ 0x009, 0x3d5, 0x100, 0x009, 0x007, 0x044, 0x063, 0x004,
+ 0x000, 0x046, 0x364, 0x200, 0x004, 0x029, 0x1dc, 0x007,
+ 0x0c0, 0x2d8, 0x111, 0x001, 0x028, 0x3c5, 0x002, 0x33c,
+ 0x3c5, 0x001, 0x33b, 0x344, 0x21f, 0x33c, 0x287, 0x19b,
+ 0x247, 0x361, 0x287, 0x183, 0x19b, 0x247, 0x360, 0x0c0,
+ 0x35e, 0x2e4, 0x020, 0x001, 0x004, 0x0c5, 0x35e, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x289, 0x384,
+ 0x010, 0x245, 0x33c, 0x044, 0x001, 0x004, 0x364, 0x33c,
+ 0x002, 0x001, 0x005, 0x3c5, 0x004, 0x200, 0x364, 0x33c,
+ 0x020, 0x009, 0x005, 0x044, 0x097, 0x004, 0x020, 0x181,
+ 0x006, 0x287, 0x19b, 0x11d, 0x001, 0x002, 0x287, 0x1a3,
+ 0x1b3, 0x19b, 0x180, 0x390, 0x1d7, 0x040, 0x199, 0x390,
+ 0x008, 0x040, 0x045, 0x000, 0x021, 0x000, 0x073, 0x000,
+ 0x05f, 0x000, 0x023, 0x000, 0x025, 0x000, 0x033, 0x000,
+ 0x035, 0x000, 0x037, 0x000, 0x041, 0x000, 0x01f, 0x000,
+ 0x021, 0x000, 0x023, 0x000, 0x031, 0x000, 0x082, 0x000,
+ 0x002, 0x020, 0x1dc, 0x007, 0x044, 0x378, 0x003, 0x3c4,
+ 0x3fd, 0x233, 0x000, 0x072, 0x2c7, 0x363, 0x000, 0x06e,
+ 0x2c7, 0x362, 0x000, 0x06a, 0x2c7, 0x341, 0x000, 0x066,
+ 0x2c7, 0x33f, 0x000, 0x062, 0x2c7, 0x340, 0x000, 0x05e,
+ 0x2c7, 0x344, 0x000, 0x05a, 0x2c7, 0x342, 0x000, 0x056,
+ 0x2c7, 0x343, 0x000, 0x052, 0x2c7, 0x341, 0x2c7, 0x33f,
+ 0x2c7, 0x340, 0x000, 0x04a, 0x307, 0x340, 0x300, 0x341,
+ 0x300, 0x33f, 0x282, 0x007, 0x006, 0x2c7, 0x23e, 0x000,
+ 0x03d, 0x043, 0x0a1, 0x020, 0x1de, 0x007, 0x287, 0x384,
+ 0x003, 0x181, 0x3c4, 0x3f3, 0x052, 0x245, 0x052, 0x3a4,
+ 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5, 0x33c, 0x000, 0x026,
+ 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff, 0x352, 0x3c7, 0x3ff,
+ 0x353, 0x1d3, 0x353, 0x0cf, 0x354, 0x1cb, 0x354, 0x3c5,
+ 0x0c0, 0x354, 0x3c9, 0x222, 0x355, 0x3c5, 0x001, 0x33c,
+ 0x0c0, 0x356, 0x1b0, 0x00a, 0x006, 0x081, 0x18a, 0x247,
+ 0x356, 0x1ab, 0x2c7, 0x357, 0x020, 0x181, 0x006, 0x287,
+ 0x001, 0x025, 0x197, 0x3a4, 0x00f, 0x262, 0x003, 0x001,
+ 0x03d, 0x131, 0x001, 0x020, 0x131, 0x001, 0x021, 0x131,
+ 0x001, 0x022, 0x131, 0x001, 0x025, 0x131, 0x001, 0x026,
+ 0x131, 0x001, 0x027, 0x132, 0x182, 0x390, 0x377, 0x280,
+ 0x04c, 0x042, 0x020, 0x1de, 0x007, 0x044, 0x361, 0x003,
+ 0x000, 0x3d4, 0x327, 0x373, 0x000, 0x3f5, 0x327, 0x372,
+ 0x000, 0x3f1, 0x327, 0x2b3, 0x322, 0x2b2, 0x000, 0x3eb,
+ 0x327, 0x374, 0x000, 0x3e7, 0x327, 0x375, 0x000, 0x3e3,
+ 0x327, 0x376, 0x000, 0x3df, 0x307, 0x369, 0x282, 0x272,
+ 0x369, 0x00e, 0x003, 0x108, 0x04c, 0x000, 0x3d4, 0x364,
+ 0x33c, 0x001, 0x001, 0x01b, 0x364, 0x33b, 0x020, 0x009,
+ 0x016, 0x0a0, 0x307, 0x04f, 0x181, 0x00a, 0x003, 0x0a4,
+ 0x100, 0x00b, 0x004, 0x3a5, 0x080, 0x307, 0x050, 0x194,
+ 0x384, 0x040, 0x225, 0x2c7, 0x392, 0x081, 0x189, 0x314,
+ 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x027, 0x364, 0x33c, 0x020, 0x001, 0x007,
+ 0x347, 0x21d, 0x35a, 0x000, 0x018, 0x347, 0x050, 0x368,
+ 0x3b7, 0x201, 0x1a5, 0x140, 0x04f, 0x00b, 0x00b, 0x364,
+ 0x052, 0x020, 0x001, 0x3f3, 0x2c5, 0x33c, 0x000, 0x005,
+ 0x2a8, 0x2c4, 0x33c, 0x081, 0x189, 0x304, 0x053, 0x001,
+ 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7,
+ 0x027, 0x3c5, 0x080, 0x053, 0x3c4, 0x37f, 0x053, 0x364,
+ 0x33b, 0x020, 0x009, 0x00a, 0x364, 0x33c, 0x020, 0x009,
+ 0x005, 0x044, 0x132, 0x004, 0x020, 0x08d, 0x006, 0x2c7,
+ 0x231, 0x2e4, 0x100, 0x001, 0x045, 0x2e4, 0x001, 0x001,
+ 0x020, 0x3b7, 0x213, 0x084, 0x044, 0x1a9, 0x000, 0x3b7,
+ 0x250, 0x084, 0x044, 0x1a9, 0x000, 0x399, 0x305, 0x3a7,
+ 0x022, 0x044, 0x1a9, 0x000, 0x044, 0x322, 0x001, 0x3c5,
+ 0x002, 0x024, 0x3c5, 0x008, 0x292, 0x000, 0x020, 0x3c4,
+ 0x3fd, 0x024, 0x364, 0x234, 0x002, 0x001, 0x009, 0x3a7,
+ 0x022, 0x399, 0x305, 0x044, 0x1b7, 0x000, 0x3b7, 0x213,
+ 0x084, 0x044, 0x1b7, 0x000, 0x3b7, 0x250, 0x084, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x007, 0x292, 0x044, 0x06e, 0x003,
+ 0x307, 0x231, 0x274, 0x200, 0x001, 0x01b, 0x274, 0x010,
+ 0x001, 0x00b, 0x387, 0x020, 0x245, 0x021, 0x185, 0x245,
+ 0x024, 0x000, 0x00b, 0x387, 0x3df, 0x244, 0x021, 0x185,
+ 0x385, 0x0ff, 0x244, 0x024, 0x044, 0x03a, 0x00a, 0x045,
+ 0x327, 0x230, 0x364, 0x233, 0x004, 0x001, 0x003, 0x045,
+ 0x0c2, 0x22a, 0x347, 0x275, 0x270, 0x0c0, 0x27b, 0x264,
+ 0x010, 0x009, 0x00a, 0x0c1, 0x22a, 0x347, 0x274, 0x270,
+ 0x3d7, 0x200, 0x27b, 0x327, 0x230, 0x264, 0x004, 0x001,
+ 0x012, 0x347, 0x277, 0x270, 0x264, 0x010, 0x009, 0x007,
+ 0x347, 0x276, 0x270, 0x384, 0x00f, 0x112, 0x1b7, 0x000,
+ 0x004, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x384, 0x00f, 0x262,
+ 0x003, 0x001, 0x00b, 0x101, 0x3c2, 0x100, 0x270, 0x34a,
+ 0x225, 0x225, 0x000, 0x3f3, 0x044, 0x237, 0x00a, 0x044,
+ 0x35f, 0x009, 0x364, 0x22a, 0x002, 0x001, 0x006, 0x347,
+ 0x270, 0x272, 0x045, 0x347, 0x270, 0x271, 0x045, 0x044,
+ 0x2f6, 0x002, 0x009, 0x015, 0x307, 0x241, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x3d7, 0x200, 0x27b, 0x044, 0x35f, 0x009, 0x347,
+ 0x241, 0x226, 0x0c1, 0x22a, 0x020, 0x032, 0x009, 0x044,
+ 0x2f6, 0x002, 0x009, 0x014, 0x307, 0x243, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x0c0, 0x27b, 0x044, 0x35f, 0x009, 0x347, 0x243,
+ 0x226, 0x0c2, 0x22a, 0x020, 0x032, 0x009, 0x327, 0x226,
+ 0x2c7, 0x225, 0x3c4, 0x0ff, 0x225, 0x1b7, 0x3a4, 0x007,
+ 0x001, 0x00d, 0x34a, 0x225, 0x225, 0x002, 0x008, 0x131,
+ 0x009, 0x3fa, 0x044, 0x237, 0x00a, 0x045, 0x0c1, 0x26f,
+ 0x000, 0x050, 0x327, 0x01d, 0x2e4, 0x380, 0x009, 0x00a,
+ 0x13a, 0x00b, 0x00a, 0x3c5, 0x008, 0x233, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x233, 0x327, 0x01d, 0x021, 0x2ed, 0x009,
+ 0x364, 0x020, 0x004, 0x009, 0x01a, 0x309, 0x232, 0x00b,
+ 0x009, 0x347, 0x270, 0x271, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x271, 0x270, 0x044, 0x270, 0x009, 0x001, 0x04c,
+ 0x347, 0x270, 0x271, 0x000, 0x018, 0x309, 0x232, 0x001,
+ 0x009, 0x347, 0x270, 0x272, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x272, 0x270, 0x044, 0x270, 0x009, 0x001, 0x089,
+ 0x347, 0x270, 0x272, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x364, 0x26f, 0x002, 0x009, 0x005, 0x3c4, 0x3fe, 0x200,
+ 0x364, 0x020, 0x005, 0x029, 0x2e4, 0x002, 0x364, 0x020,
+ 0x010, 0x009, 0x0e9, 0x364, 0x020, 0x100, 0x009, 0x12b,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3f7, 0x233, 0x364, 0x020,
+ 0x001, 0x009, 0x00c, 0x364, 0x020, 0x004, 0x009, 0x05d,
+ 0x000, 0x3d8, 0x3c5, 0x080, 0x020, 0x364, 0x233, 0x006,
+ 0x001, 0x031, 0x081, 0x189, 0x304, 0x232, 0x009, 0x017,
+ 0x347, 0x271, 0x274, 0x080, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x085, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x081,
+ 0x189, 0x245, 0x232, 0x000, 0x0c8, 0x347, 0x271, 0x276,
+ 0x081, 0x189, 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197,
+ 0x384, 0x007, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x000,
+ 0x005, 0x044, 0x3f3, 0x008, 0x3b7, 0x200, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x203, 0x387, 0x3f7, 0x044,
+ 0x1b8, 0x000, 0x307, 0x020, 0x385, 0x042, 0x384, 0x3ce,
+ 0x247, 0x020, 0x044, 0x044, 0x001, 0x000, 0x383, 0x0a1,
+ 0x1aa, 0x2c5, 0x020, 0x364, 0x233, 0x006, 0x001, 0x039,
+ 0x081, 0x189, 0x304, 0x232, 0x009, 0x017, 0x347, 0x272,
+ 0x275, 0x080, 0x364, 0x233, 0x004, 0x009, 0x003, 0x095,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245,
+ 0x232, 0x000, 0x0ce, 0x347, 0x272, 0x277, 0x081, 0x189,
+ 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197, 0x364, 0x233,
+ 0x002, 0x001, 0x007, 0x193, 0x384, 0x007, 0x385, 0x010,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x000, 0x008, 0x347,
+ 0x272, 0x270, 0x044, 0x013, 0x009, 0x081, 0x18a, 0x208,
+ 0x244, 0x232, 0x0a3, 0x387, 0x3f7, 0x044, 0x1b8, 0x000,
+ 0x307, 0x020, 0x0a4, 0x1a7, 0x3a5, 0x008, 0x285, 0x3b7,
+ 0x304, 0x2a8, 0x284, 0x247, 0x020, 0x0a0, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x044, 0x2f6, 0x002, 0x009, 0x00f,
+ 0x364, 0x2cf, 0x080, 0x009, 0x00a, 0x0c3, 0x26f, 0x044,
+ 0x1b6, 0x001, 0x020, 0x2e4, 0x002, 0x044, 0x051, 0x001,
+ 0x000, 0x308, 0x3c4, 0x3f7, 0x233, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x083, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x3d7, 0x200, 0x27b, 0x0c0, 0x271,
+ 0x3b7, 0x300, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x221, 0x091, 0x189, 0x044, 0x1a9, 0x000, 0x3b7, 0x201,
+ 0x387, 0x0a6, 0x044, 0x1bd, 0x000, 0x3c5, 0x021, 0x020,
+ 0x3c4, 0x3ed, 0x020, 0x3b7, 0x200, 0x387, 0x020, 0x044,
+ 0x020, 0x00a, 0x009, 0x32a, 0x347, 0x271, 0x270, 0x000,
+ 0x055, 0x3a7, 0x071, 0x307, 0x21d, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x233, 0x044, 0x2f6, 0x002, 0x009, 0x007,
+ 0x0c1, 0x2d3, 0x3c5, 0x100, 0x200, 0x081, 0x18a, 0x245,
+ 0x232, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x093, 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x0c0,
+ 0x27b, 0x0c0, 0x272, 0x3b7, 0x300, 0x387, 0x020, 0x044,
+ 0x1a9, 0x000, 0x0a1, 0x397, 0x205, 0x247, 0x27f, 0x044,
+ 0x1bd, 0x000, 0x3d5, 0x204, 0x020, 0x3c4, 0x2ff, 0x020,
+ 0x0a0, 0x387, 0x021, 0x184, 0x044, 0x020, 0x00a, 0x009,
+ 0x336, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3fe, 0x200, 0x3c4, 0x3fe, 0x26f, 0x344, 0x21f, 0x232,
+ 0x0c0, 0x020, 0x0a3, 0x1ab, 0x2c5, 0x020, 0x044, 0x20d,
+ 0x000, 0x044, 0x2f3, 0x000, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x0a9, 0x1ab, 0x09f, 0x18a, 0x208, 0x304,
+ 0x055, 0x225, 0x044, 0x394, 0x00a, 0x307, 0x055, 0x19b,
+ 0x384, 0x003, 0x227, 0x1a3, 0x225, 0x044, 0x0a4, 0x00b,
+ 0x0a7, 0x020, 0x022, 0x008, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x006, 0x387, 0x07f, 0x000, 0x013, 0x399, 0x240,
+ 0x385, 0x07f, 0x306, 0x270, 0x001, 0x06c, 0x399, 0x240,
+ 0x327, 0x270, 0x3a4, 0x380, 0x226, 0x001, 0x015, 0x327,
+ 0x220, 0x324, 0x232, 0x009, 0x019, 0x364, 0x232, 0x001,
+ 0x001, 0x02b, 0x327, 0x01d, 0x307, 0x01b, 0x190, 0x282,
+ 0x003, 0x041, 0x307, 0x01b, 0x191, 0x380, 0x010, 0x282,
+ 0x003, 0x034, 0x000, 0x019, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x00a, 0x327, 0x270, 0x384, 0x07f, 0x3a4, 0x07f,
+ 0x000, 0x008, 0x327, 0x270, 0x3a4, 0x07f, 0x001, 0x005,
+ 0x286, 0x001, 0x02f, 0x307, 0x270, 0x384, 0x07f, 0x386,
+ 0x07f, 0x009, 0x00a, 0x387, 0x080, 0x382, 0x07f, 0x240,
+ 0x270, 0x000, 0x01c, 0x141, 0x270, 0x364, 0x270, 0x07f,
+ 0x009, 0x015, 0x000, 0x013, 0x3c0, 0x040, 0x270, 0x000,
+ 0x00e, 0x307, 0x270, 0x196, 0x384, 0x00f, 0x266, 0x009,
+ 0x001, 0x008, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009,
+ 0x045, 0x309, 0x232, 0x00b, 0x013, 0x347, 0x271, 0x270,
+ 0x374, 0x27b, 0x200, 0x009, 0x005, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x020, 0x0b7, 0x009, 0x327, 0x232,
+ 0x1a2, 0x023, 0x0b7, 0x009, 0x307, 0x270, 0x196, 0x119,
+ 0x021, 0x0b7, 0x009, 0x307, 0x232, 0x181, 0x023, 0x0b7,
+ 0x009, 0x364, 0x270, 0x080, 0x029, 0x0b7, 0x009, 0x083,
+ 0x18c, 0x245, 0x232, 0x3c4, 0x3f7, 0x233, 0x3c4, 0x380,
+ 0x270, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x09c, 0x009, 0x000,
+ 0x000, 0x000, 0x002, 0x000, 0x00a, 0x000, 0x02a, 0x000,
+ 0x0aa, 0x002, 0x0aa, 0x00a, 0x0aa, 0x02a, 0x0aa, 0x06a,
+ 0x0aa, 0x07a, 0x0aa, 0x07e, 0x0aa, 0x07f, 0x0aa, 0x07f,
+ 0x0ba, 0x07f, 0x0be, 0x07f, 0x0fe, 0x07f, 0x0ff, 0x007,
+ 0x000, 0x017, 0x001, 0x027, 0x007, 0x037, 0x00f, 0x047,
+ 0x03f, 0x080, 0x0ff, 0x364, 0x270, 0x07f, 0x307, 0x270,
+ 0x384, 0x07f, 0x009, 0x00d, 0x089, 0x186, 0x306, 0x270,
+ 0x001, 0x007, 0x3c5, 0x001, 0x232, 0x000, 0x005, 0x3c4,
+ 0x3fe, 0x232, 0x347, 0x270, 0x27c, 0x3c4, 0x007, 0x27c,
+ 0x374, 0x27b, 0x200, 0x001, 0x00a, 0x364, 0x233, 0x008,
+ 0x001, 0x005, 0x044, 0x0fe, 0x00a, 0x0a3, 0x325, 0x27b,
+ 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x305, 0x27c, 0x044,
+ 0x1bd, 0x000, 0x307, 0x270, 0x192, 0x384, 0x00f, 0x3b9,
+ 0x200, 0x324, 0x233, 0x001, 0x014, 0x229, 0x387, 0x009,
+ 0x189, 0x395, 0x333, 0x280, 0x051, 0x2c7, 0x27d, 0x1c7,
+ 0x27d, 0x101, 0x051, 0x2c5, 0x27d, 0x000, 0x011, 0x0c0,
+ 0x27d, 0x384, 0x00f, 0x001, 0x00b, 0x349, 0x27d, 0x27d,
+ 0x3c5, 0x001, 0x27d, 0x111, 0x000, 0x3f7, 0x0a2, 0x325,
+ 0x27b, 0x307, 0x27d, 0x044, 0x1bd, 0x000, 0x374, 0x27b,
+ 0x200, 0x001, 0x02c, 0x347, 0x270, 0x27a, 0x3c4, 0x07f,
+ 0x27a, 0x0c0, 0x279, 0x387, 0x009, 0x189, 0x395, 0x353,
+ 0x051, 0x041, 0x101, 0x051, 0x2c7, 0x279, 0x101, 0x043,
+ 0x322, 0x27a, 0x00b, 0x004, 0x000, 0x3f4, 0x1c7, 0x279,
+ 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x0ff,
+ 0x305, 0x279, 0x044, 0x1bd, 0x000, 0x327, 0x220, 0x324,
+ 0x232, 0x009, 0x022, 0x0c0, 0x27e, 0x307, 0x270, 0x197,
+ 0x00a, 0x004, 0x000, 0x005, 0x3c5, 0x018, 0x27e, 0x0a4,
+ 0x222, 0x2c5, 0x27e, 0x0a4, 0x325, 0x27b, 0x044, 0x192,
+ 0x000, 0x384, 0x3e0, 0x3c4, 0x01f, 0x27e, 0x305, 0x27e,
+ 0x044, 0x1bd, 0x000, 0x045, 0x044, 0x1a9, 0x000, 0x083,
+ 0x18c, 0x208, 0x244, 0x232, 0x3c5, 0x001, 0x00c, 0x0a3,
+ 0x325, 0x27b, 0x088, 0x044, 0x1a9, 0x000, 0x387, 0x3f8,
+ 0x044, 0x1b8, 0x000, 0x020, 0x2f6, 0x002, 0x3c7, 0x020,
+ 0x280, 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x247, 0x247,
+ 0x151, 0x280, 0x001, 0x016, 0x0a1, 0x044, 0x182, 0x000,
+ 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x306, 0x247, 0x001,
+ 0x3f1, 0x0a2, 0x364, 0x22b, 0x003, 0x001, 0x003, 0x0a5,
+ 0x020, 0x182, 0x000, 0x1f6, 0x185, 0x1f5, 0x129, 0x1f4,
+ 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff, 0x1f6, 0x189, 0x1f5,
+ 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce, 0x123, 0x0f3, 0x0d2,
+ 0x0ff, 0x364, 0x233, 0x020, 0x009, 0x051, 0x307, 0x272,
+ 0x196, 0x384, 0x00f, 0x0c9, 0x286, 0x242, 0x286, 0x397,
+ 0x00a, 0x189, 0x395, 0x05f, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x00a, 0x051, 0x2c7, 0x284, 0x101, 0x051,
+ 0x2c7, 0x288, 0x3a4, 0x07f, 0x040, 0x307, 0x272, 0x384,
+ 0x07f, 0x222, 0x042, 0x00b, 0x005, 0x101, 0x000, 0x3ed,
+ 0x347, 0x284, 0x285, 0x3c4, 0x00f, 0x284, 0x1d3, 0x285,
+ 0x1d6, 0x288, 0x342, 0x288, 0x286, 0x003, 0x008, 0x156,
+ 0x285, 0x151, 0x286, 0x00b, 0x3fc, 0x362, 0x285, 0x003,
+ 0x003, 0x004, 0x0c3, 0x285, 0x307, 0x285, 0x0a1, 0x184,
+ 0x305, 0x284, 0x044, 0x1bd, 0x000, 0x045, 0x3db, 0x071,
+ 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c, 0x176, 0x1bf, 0x00d,
+ 0x007, 0x006, 0x003, 0x002, 0x005, 0x0b3, 0x156, 0x1fa,
+ 0x27f, 0x080, 0x09e, 0x14b, 0x1f0, 0x27f, 0x080, 0x08b,
+ 0x140, 0x1e9, 0x27f, 0x081, 0x138, 0x1e3, 0x27f, 0x00f,
+ 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080, 0x0a3, 0x145, 0x1e7,
+ 0x27f, 0x080, 0x095, 0x13d, 0x1e0, 0x27f, 0x08b, 0x136,
+ 0x1db, 0x27f, 0x0c5, 0x28b, 0x347, 0x271, 0x281, 0x3c4,
+ 0x07f, 0x281, 0x0c5, 0x282, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x0c2, 0x282, 0x044, 0x1bc, 0x00a, 0x347, 0x283,
+ 0x289, 0x307, 0x241, 0x364, 0x233, 0x002, 0x001, 0x004,
+ 0x307, 0x225, 0x384, 0x0ff, 0x242, 0x283, 0x00b, 0x056,
+ 0x0c3, 0x28b, 0x302, 0x289, 0x397, 0x00a, 0x189, 0x395,
+ 0x0ca, 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x004,
+ 0x051, 0x2c7, 0x28c, 0x2a9, 0x121, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003, 0x007,
+ 0x001, 0x005, 0x101, 0x000, 0x3ed, 0x1d5, 0x28c, 0x347,
+ 0x241, 0x281, 0x364, 0x233, 0x002, 0x001, 0x005, 0x347,
+ 0x225, 0x281, 0x3c4, 0x0ff, 0x281, 0x342, 0x289, 0x281,
+ 0x347, 0x28c, 0x282, 0x366, 0x282, 0x00f, 0x009, 0x006,
+ 0x1c0, 0x281, 0x0c1, 0x282, 0x044, 0x1bc, 0x00a, 0x001,
+ 0x00d, 0x141, 0x28b, 0x362, 0x28b, 0x03f, 0x001, 0x00d,
+ 0x151, 0x283, 0x000, 0x3f5, 0x362, 0x28b, 0x00a, 0x003,
+ 0x004, 0x0ca, 0x28b, 0x397, 0x00a, 0x189, 0x395, 0x0d8,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x013, 0x327,
+ 0x24c, 0x131, 0x003, 0x005, 0x105, 0x000, 0x3fc, 0x051,
+ 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121, 0x040, 0x307, 0x271,
+ 0x384, 0x07f, 0x282, 0x042, 0x003, 0x005, 0x101, 0x000,
+ 0x3f0, 0x1d6, 0x28a, 0x387, 0x00a, 0x189, 0x395, 0x0d3,
+ 0x300, 0x28a, 0x050, 0x247, 0x28a, 0x30a, 0x28b, 0x3b7,
+ 0x201, 0x184, 0x305, 0x28a, 0x044, 0x1bd, 0x000, 0x045,
+ 0x0c1, 0x283, 0x342, 0x282, 0x281, 0x003, 0x006, 0x141,
+ 0x283, 0x000, 0x3f9, 0x045, 0x000, 0x301, 0x103, 0x307,
+ 0x006, 0x10e, 0x21e, 0x33e, 0x13c, 0x038, 0x078, 0x1f8,
+ 0x0f0, 0x000, 0x001, 0x003, 0x002, 0x006, 0x187, 0x227,
+ 0x387, 0x00a, 0x189, 0x395, 0x1c8, 0x300, 0x227, 0x111,
+ 0x050, 0x247, 0x24c, 0x1d7, 0x24c, 0x384, 0x0ff, 0x285,
+ 0x044, 0x226, 0x00a, 0x364, 0x22a, 0x001, 0x001, 0x00e,
+ 0x3d5, 0x200, 0x232, 0x040, 0x1aa, 0x285, 0x3b7, 0x220,
+ 0x044, 0x1bd, 0x000, 0x042, 0x364, 0x22a, 0x002, 0x001,
+ 0x022, 0x040, 0x3a7, 0x073, 0x044, 0x192, 0x000, 0x384,
+ 0x31f, 0x044, 0x226, 0x00a, 0x1a4, 0x285, 0x3a7, 0x073,
+ 0x044, 0x1bd, 0x000, 0x3a7, 0x070, 0x044, 0x192, 0x000,
+ 0x19a, 0x18a, 0x043, 0x285, 0x3a7, 0x070, 0x044, 0x1bd,
+ 0x000, 0x045, 0x040, 0x387, 0x00a, 0x189, 0x395, 0x1d5,
+ 0x300, 0x278, 0x051, 0x042, 0x045, 0x00a, 0x058, 0x026,
+ 0x015, 0x044, 0x032, 0x0c0, 0x278, 0x0a0, 0x397, 0x00a,
+ 0x189, 0x395, 0x231, 0x280, 0x050, 0x247, 0x228, 0x3c4,
+ 0x00f, 0x228, 0x347, 0x228, 0x229, 0x040, 0x307, 0x278,
+ 0x001, 0x008, 0x340, 0x229, 0x228, 0x111, 0x000, 0x3fa,
+ 0x042, 0x193, 0x0c0, 0x227, 0x347, 0x225, 0x226, 0x141,
+ 0x227, 0x366, 0x227, 0x00e, 0x001, 0x00b, 0x342, 0x228,
+ 0x226, 0x007, 0x3f6, 0x140, 0x226, 0x001, 0x011, 0x121,
+ 0x2e2, 0x006, 0x009, 0x3cc, 0x141, 0x278, 0x366, 0x278,
+ 0x005, 0x009, 0x3c4, 0x0a1, 0x000, 0x006, 0x044, 0x1da,
+ 0x00a, 0x0a0, 0x045, 0x044, 0x333, 0x008, 0x020, 0x181,
+ 0x006, 0x345, 0x21d, 0x232, 0x364, 0x23f, 0x001, 0x029,
+ 0x2e4, 0x002, 0x2c7, 0x234, 0x399, 0x200, 0x245, 0x233,
+ 0x364, 0x234, 0x020, 0x009, 0x005, 0x208, 0x244, 0x233,
+ 0x083, 0x18c, 0x208, 0x244, 0x036, 0x2e4, 0x008, 0x001,
+ 0x007, 0x345, 0x220, 0x232, 0x000, 0x007, 0x081, 0x18b,
+ 0x208, 0x244, 0x232, 0x307, 0x235, 0x226, 0x3a4, 0x001,
+ 0x001, 0x010, 0x327, 0x234, 0x2e4, 0x001, 0x009, 0x041,
+ 0x041, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x043,
+ 0x0a3, 0x1ad, 0x3a5, 0x039, 0x326, 0x236, 0x001, 0x02f,
+ 0x3a7, 0x0bf, 0x1a7, 0x3a5, 0x039, 0x326, 0x236, 0x001,
+ 0x026, 0x327, 0x234, 0x307, 0x235, 0x286, 0x384, 0x002,
+ 0x001, 0x01d, 0x2e4, 0x002, 0x009, 0x051, 0x041, 0x0a1,
+ 0x1ad, 0x2c5, 0x024, 0x307, 0x024, 0x19a, 0x002, 0x005,
+ 0x2a8, 0x2c4, 0x024, 0x044, 0x3b5, 0x000, 0x044, 0x0f2,
+ 0x001, 0x044, 0x3f0, 0x000, 0x043, 0x000, 0x067, 0x364,
+ 0x026, 0x001, 0x009, 0x3c6, 0x041, 0x0a0, 0x044, 0x301,
+ 0x000, 0x364, 0x26f, 0x002, 0x021, 0x310, 0x00a, 0x044,
+ 0x290, 0x001, 0x0c1, 0x26f, 0x043, 0x307, 0x020, 0x193,
+ 0x384, 0x007, 0x110, 0x001, 0x00c, 0x364, 0x232, 0x010,
+ 0x009, 0x007, 0x044, 0x044, 0x001, 0x000, 0x3a3, 0x3c4,
+ 0x3fe, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x010, 0x020,
+ 0x3c4, 0x31f, 0x020, 0x000, 0x395, 0x364, 0x026, 0x002,
+ 0x009, 0x3c5, 0x041, 0x044, 0x003, 0x001, 0x043, 0x307,
+ 0x020, 0x197, 0x384, 0x007, 0x110, 0x001, 0x00f, 0x364,
+ 0x232, 0x020, 0x009, 0x00a, 0x044, 0x051, 0x001, 0x3c5,
+ 0x001, 0x027, 0x000, 0x3ab, 0x3c4, 0x3fc, 0x26f, 0x3c5,
+ 0x001, 0x200, 0x3c5, 0x100, 0x020, 0x087, 0x188, 0x208,
+ 0x244, 0x020, 0x000, 0x39b, 0x044, 0x11a, 0x001, 0x307,
+ 0x234, 0x306, 0x235, 0x384, 0x003, 0x021, 0x2e4, 0x002,
+ 0x347, 0x234, 0x235, 0x366, 0x26f, 0x002, 0x021, 0x2e4,
+ 0x002, 0x364, 0x234, 0x003, 0x021, 0x2e4, 0x002, 0x327,
+ 0x21c, 0x324, 0x020, 0x021, 0x385, 0x00a, 0x020, 0x247,
+ 0x002, 0x364, 0x23f, 0x001, 0x021, 0x38e, 0x00a, 0x020,
+ 0x247, 0x002, 0x345, 0x21d, 0x232, 0x020, 0x247, 0x002,
+ 0x2c7, 0x241, 0x307, 0x242, 0x286, 0x001, 0x092, 0x0c1,
+ 0x22a, 0x327, 0x242, 0x326, 0x241, 0x3a4, 0x0ff, 0x001,
+ 0x005, 0x3c4, 0x38f, 0x020, 0x347, 0x241, 0x242, 0x347,
+ 0x241, 0x226, 0x307, 0x21d, 0x304, 0x241, 0x001, 0x00a,
+ 0x0c3, 0x22a, 0x347, 0x241, 0x243, 0x044, 0x39f, 0x002,
+ 0x327, 0x241, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x300,
+ 0x230, 0x2c5, 0x230, 0x044, 0x099, 0x00b, 0x081, 0x18b,
+ 0x304, 0x226, 0x009, 0x00d, 0x364, 0x232, 0x100, 0x009,
+ 0x008, 0x081, 0x18a, 0x208, 0x044, 0x1b8, 0x000, 0x044,
+ 0x186, 0x001, 0x327, 0x226, 0x003, 0x032, 0x044, 0x2f6,
+ 0x002, 0x001, 0x019, 0x044, 0x237, 0x00a, 0x347, 0x241,
+ 0x226, 0x081, 0x306, 0x22a, 0x384, 0x003, 0x001, 0x005,
+ 0x347, 0x243, 0x226, 0x044, 0x032, 0x009, 0x002, 0x02d,
+ 0x000, 0x031, 0x044, 0x237, 0x00a, 0x009, 0x01c, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x001, 0x023,
+ 0x364, 0x233, 0x001, 0x000, 0x01e, 0x045, 0x044, 0x186,
+ 0x001, 0x3c5, 0x003, 0x22a, 0x347, 0x241, 0x243, 0x000,
+ 0x3c7, 0x3a4, 0x3ff, 0x001, 0x00e, 0x327, 0x21c, 0x324,
+ 0x020, 0x009, 0x006, 0x043, 0x3a7, 0x0ff, 0x041, 0x0c0,
+ 0x270, 0x044, 0x082, 0x00b, 0x3c4, 0x3fc, 0x22a, 0x327,
+ 0x21c, 0x324, 0x020, 0x009, 0x010, 0x307, 0x270, 0x003,
+ 0x005, 0x020, 0x181, 0x006, 0x043, 0x3a7, 0x0ff, 0x041,
+ 0x020, 0x181, 0x006, 0x045, 0x2c7, 0x243, 0x307, 0x250,
+ 0x286, 0x001, 0x3de, 0x0c2, 0x22a, 0x041, 0x044, 0x39f,
+ 0x002, 0x043, 0x2c7, 0x226, 0x3a4, 0x0ff, 0x2c7, 0x225,
+ 0x3c4, 0x0ff, 0x230, 0x1a7, 0x2c5, 0x230, 0x307, 0x21d,
+ 0x304, 0x243, 0x001, 0x013, 0x3c4, 0x0ff, 0x243, 0x347,
+ 0x243, 0x241, 0x347, 0x243, 0x242, 0x3c5, 0x003, 0x22a,
+ 0x044, 0x186, 0x001, 0x000, 0x005, 0x044, 0x191, 0x001,
+ 0x327, 0x243, 0x003, 0x394, 0x000, 0x362, 0x364, 0x232,
+ 0x004, 0x001, 0x008, 0x044, 0x232, 0x000, 0x3c4, 0x3fb,
+ 0x232, 0x364, 0x232, 0x008, 0x001, 0x008, 0x044, 0x326,
+ 0x000, 0x3c4, 0x3f7, 0x232, 0x045, 0x3b7, 0x221, 0x083,
+ 0x189, 0x044, 0x1a9, 0x000, 0x083, 0x020, 0x1b7, 0x000,
+ 0x2c7, 0x24d, 0x364, 0x233, 0x002, 0x001, 0x019, 0x3b9,
+ 0x381, 0x121, 0x081, 0x18a, 0x225, 0x304, 0x24d, 0x001,
+ 0x005, 0x044, 0x1ff, 0x00b, 0x140, 0x24d, 0x00b, 0x008,
+ 0x1a3, 0x044, 0x1ff, 0x00b, 0x000, 0x012, 0x307, 0x220,
+ 0x284, 0x001, 0x00d, 0x3c4, 0x33f, 0x232, 0x1b6, 0x3a4,
+ 0x0c0, 0x2c5, 0x232, 0x020, 0x18a, 0x00b, 0x327, 0x24d,
+ 0x364, 0x020, 0x070, 0x001, 0x00a, 0x399, 0x383, 0x101,
+ 0x326, 0x22b, 0x234, 0x001, 0x02e, 0x3c5, 0x080, 0x233,
+ 0x3d5, 0x200, 0x232, 0x364, 0x026, 0x001, 0x001, 0x00a,
+ 0x3c5, 0x004, 0x232, 0x044, 0x20d, 0x000, 0x307, 0x24d,
+ 0x307, 0x24d, 0x384, 0x007, 0x001, 0x09a, 0x113, 0x001,
+ 0x0a8, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8,
+ 0x103, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x021, 0x000,
+ 0x0af, 0x0a7, 0x1a7, 0x324, 0x020, 0x001, 0x014, 0x327,
+ 0x24d, 0x326, 0x22b, 0x397, 0x370, 0x364, 0x233, 0x002,
+ 0x001, 0x006, 0x087, 0x18b, 0x385, 0x070, 0x234, 0x001,
+ 0x02b, 0x3c5, 0x100, 0x233, 0x364, 0x026, 0x002, 0x001,
+ 0x008, 0x3c5, 0x008, 0x232, 0x044, 0x3b5, 0x000, 0x307,
+ 0x24d, 0x384, 0x070, 0x001, 0x091, 0x382, 0x030, 0x001,
+ 0x0a6, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x018, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x024,
+ 0x000, 0x0a4, 0x347, 0x24d, 0x22b, 0x364, 0x233, 0x006,
+ 0x001, 0x033, 0x307, 0x22b, 0x364, 0x233, 0x002, 0x001,
+ 0x023, 0x364, 0x233, 0x080, 0x001, 0x00a, 0x307, 0x24d,
+ 0x197, 0x384, 0x007, 0x044, 0x3a4, 0x008, 0x364, 0x233,
+ 0x100, 0x001, 0x00c, 0x307, 0x24d, 0x19b, 0x384, 0x007,
+ 0x385, 0x010, 0x044, 0x3a4, 0x008, 0x3c4, 0x27f, 0x233,
+ 0x000, 0x00b, 0x3c5, 0x003, 0x22a, 0x197, 0x044, 0x39c,
+ 0x008, 0x003, 0x298, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x021, 0x181, 0x006, 0x045, 0x3b7, 0x301,
+ 0x087, 0x044, 0x1b7, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x3f7, 0x021, 0x000, 0x36c, 0x3b7,
+ 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x105, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000,
+ 0x3c5, 0x008, 0x021, 0x344, 0x21f, 0x021, 0x3b7, 0x250,
+ 0x088, 0x044, 0x1a9, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1a9, 0x000, 0x000, 0x347, 0x3b7, 0x301, 0x387, 0x3c7,
+ 0x044, 0x1b8, 0x000, 0x3a7, 0x060, 0x083, 0x044, 0x1b7,
+ 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x024, 0x000, 0x36f, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x028, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x048, 0x024, 0x3a7, 0x060, 0x083, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1a9,
+ 0x000, 0x000, 0x351, 0x307, 0x22b, 0x284, 0x2a8, 0x2c4,
+ 0x24d, 0x245, 0x24d, 0x2a8, 0x045, 0x2f4, 0x200, 0x001,
+ 0x008, 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287,
+ 0x385, 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010,
+ 0x029, 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384,
+ 0x0ff, 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d,
+ 0x040, 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d,
+ 0x042, 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005,
+ 0x3c5, 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264,
+ 0x080, 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5,
+ 0x080, 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f,
+ 0x02d, 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff,
+ 0x1b8, 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021,
+ 0x3c4, 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a,
+ 0x020, 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x251,
+ 0x22e, 0x364, 0x026, 0x001, 0x021, 0x181, 0x006, 0x041,
+ 0x3a4, 0x01f, 0x001, 0x005, 0x044, 0x0d9, 0x001, 0x043,
+ 0x1b7, 0x001, 0x005, 0x044, 0x182, 0x000, 0x020, 0x181,
+ 0x006, 0x2e4, 0x002, 0x009, 0x01b, 0x3c4, 0x2ff, 0x232,
+ 0x120, 0x001, 0x012, 0x3d5, 0x300, 0x232, 0x044, 0x099,
+ 0x00b, 0x32a, 0x026, 0x00a, 0x008, 0x044, 0x20d, 0x000,
+ 0x044, 0x232, 0x000, 0x020, 0x181, 0x006, 0x042, 0x307,
+ 0x240, 0x20a, 0x040, 0x020, 0x181, 0x006, 0x289, 0x00a,
+ 0x004, 0x2c7, 0x256, 0x22e, 0x364, 0x026, 0x002, 0x021,
+ 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005, 0x044,
+ 0x0de, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044, 0x0e2,
+ 0x001, 0x020, 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009,
+ 0x007, 0x044, 0x340, 0x001, 0x247, 0x249, 0x054, 0x347,
+ 0x249, 0x248, 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018,
+ 0x364, 0x23f, 0x001, 0x001, 0x006, 0x245, 0x057, 0x000,
+ 0x005, 0x208, 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0,
+ 0x364, 0x23f, 0x002, 0x001, 0x009, 0x245, 0x057, 0x044,
+ 0x27d, 0x001, 0x000, 0x005, 0x208, 0x244, 0x057, 0x020,
+ 0x181, 0x006, 0x2c7, 0x246, 0x3b7, 0x300, 0x044, 0x192,
+ 0x000, 0x384, 0x33f, 0x327, 0x246, 0x3a4, 0x0c0, 0x285,
+ 0x3b7, 0x300, 0x044, 0x1bd, 0x000, 0x3b7, 0x200, 0x044,
+ 0x192, 0x000, 0x0a3, 0x1a9, 0x2a8, 0x284, 0x040, 0x307,
+ 0x246, 0x384, 0x030, 0x264, 0x010, 0x001, 0x012, 0x040,
+ 0x041, 0x3b7, 0x201, 0x044, 0x192, 0x000, 0x384, 0x3e0,
+ 0x385, 0x006, 0x044, 0x1bd, 0x000, 0x043, 0x042, 0x185,
+ 0x2a8, 0x284, 0x227, 0x042, 0x285, 0x3b7, 0x200, 0x044,
+ 0x1bd, 0x000, 0x307, 0x246, 0x384, 0x007, 0x044, 0x375,
+ 0x00b, 0x3a7, 0x073, 0x307, 0x246, 0x184, 0x00a, 0x014,
+ 0x044, 0x192, 0x000, 0x384, 0x3e2, 0x327, 0x246, 0x1b5,
+ 0x3a4, 0x01c, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000,
+ 0x000, 0x006, 0x081, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x3c5, 0x001, 0x00c, 0x187, 0x0a7, 0x1a7, 0x2a8,
+ 0x324, 0x00c, 0x225, 0x2c7, 0x00c, 0x045, 0x04e, 0x087,
+ 0x011, 0x054, 0x084, 0x020, 0x01e, 0x00f, 0x018, 0x01d,
+ 0x029, 0x00b, 0x01d, 0x02c, 0x331, 0x04e, 0x020, 0x0d3,
+ 0x000, 0x020, 0x167, 0x00b, 0x020, 0x3f6, 0x00a, 0x020,
+ 0x096, 0x00b, 0x020, 0x0d3, 0x000, 0x020, 0x0d3, 0x000,
+ 0x020, 0x00d, 0x00b, 0x020, 0x179, 0x00b, 0x0c1, 0x244,
+ 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200, 0x020, 0x0d3,
+ 0x000, 0x020, 0x2c3, 0x008, 0x04e, 0x04e, 0x045, 0x0c2,
+ 0x201, 0x045, 0x020, 0x3ed, 0x00a, 0x020, 0x256, 0x002,
+ 0x020, 0x256, 0x002, 0x020, 0x256, 0x002, 0x020, 0x23f,
+ 0x006, 0x020, 0x160, 0x00b, 0x020, 0x28d, 0x00b, 0x020,
+ 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007, 0x020,
+ 0x160, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x2d5, 0x006, 0x020, 0x326,
+ 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007, 0x020,
+ 0x35a, 0x006, 0x020, 0x177, 0x00b, 0x020, 0x38e, 0x006,
+ 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020, 0x31e,
+ 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007, 0x020,
+ 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060, 0x007,
+ 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020, 0x2d1,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252, 0x007,
+ 0x020, 0x293, 0x007, 0x020, 0x2fb, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007, 0x020,
+ 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007, 0x020,
+ 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5, 0x008,
+ 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287, 0x264,
+ 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020,
+ 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004, 0x3a4,
+ 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009, 0x093,
+ 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff, 0x053,
+ 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052,
+ 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de, 0x007,
+ 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c, 0x100,
+ 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c, 0x3d5,
+ 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066, 0x004,
+ 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b, 0x262,
+ 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef,
+ 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009, 0x007,
+ 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200, 0x004,
+ 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001, 0x028,
+ 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344, 0x21f,
+ 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183, 0x19b,
+ 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001, 0x004,
+ 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5, 0x080,
+ 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044, 0x001,
+ 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5, 0x004,
+ 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x097,
+ 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d, 0x001,
+ 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390, 0x1d7,
+ 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000, 0x021,
+ 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000, 0x025,
+ 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000, 0x041,
+ 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000, 0x031,
+ 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007, 0x044,
+ 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072, 0x2c7,
+ 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a, 0x2c7,
+ 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062, 0x2c7,
+ 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a, 0x2c7,
+ 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052, 0x2c7,
+ 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a, 0x307,
+ 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007, 0x006,
+ 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020, 0x1de,
+ 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3, 0x052,
+ 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5,
+ 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff,
+ 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf, 0x354,
+ 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222, 0x355,
+ 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a, 0x006,
+ 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357, 0x020,
+ 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4, 0x00f,
+ 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020, 0x131,
+ 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001, 0x025,
+ 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132, 0x182,
+ 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de, 0x007,
+ 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373, 0x000,
+ 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3, 0x322,
+ 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7, 0x327,
+ 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df, 0x307,
+ 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108, 0x04c,
+ 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b, 0x364,
+ 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f, 0x181,
+ 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5, 0x080,
+ 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7, 0x392,
+ 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5, 0x008,
+ 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364, 0x33c,
+ 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000, 0x018,
+ 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140, 0x04f,
+ 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3, 0x2c5,
+ 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081, 0x189,
+ 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000,
+ 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053, 0x3c4,
+ 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a, 0x364,
+ 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004, 0x020,
+ 0x08d, 0x006, 0x364, 0x026, 0x010, 0x009, 0x0e8, 0x0c9,
+ 0x2ce, 0x0c8, 0x30f, 0x362, 0x2d2, 0x002, 0x009, 0x004,
+ 0x0ce, 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x0af, 0x0c4,
+ 0x303, 0x364, 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303,
+ 0x0c1, 0x310, 0x364, 0x30f, 0x001, 0x009, 0x004, 0x152,
+ 0x310, 0x397, 0x2b2, 0x300, 0x30f, 0x04c, 0x2c7, 0x305,
+ 0x397, 0x2aa, 0x300, 0x303, 0x04c, 0x2c0, 0x305, 0x342,
+ 0x310, 0x305, 0x397, 0x2a1, 0x300, 0x303, 0x327, 0x305,
+ 0x04d, 0x044, 0x1bf, 0x003, 0x08c, 0x364, 0x30f, 0x002,
+ 0x001, 0x003, 0x108, 0x364, 0x30f, 0x004, 0x001, 0x003,
+ 0x101, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x364, 0x2cf, 0x002, 0x001, 0x014, 0x309, 0x013, 0x34b,
+ 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b,
+ 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336, 0x014, 0x397,
+ 0x2a1, 0x300, 0x303, 0x04c, 0x320, 0x310, 0x04d, 0x2e4,
+ 0x100, 0x001, 0x004, 0x000, 0x012, 0x044, 0x3f0, 0x002,
+ 0x044, 0x30e, 0x002, 0x140, 0x00f, 0x009, 0x008, 0x307,
+ 0x00e, 0x302, 0x2cd, 0x00a, 0x3e4, 0x0c4, 0x303, 0x364,
+ 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303, 0x397, 0x2a1,
+ 0x300, 0x303, 0x04c, 0x2c7, 0x305, 0x397, 0x2aa, 0x300,
+ 0x303, 0x04c, 0x2c2, 0x305, 0x342, 0x310, 0x305, 0x397,
+ 0x2b2, 0x300, 0x30f, 0x327, 0x305, 0x04d, 0x362, 0x2d2,
+ 0x002, 0x001, 0x00f, 0x364, 0x2cb, 0x002, 0x009, 0x00a,
+ 0x307, 0x30f, 0x118, 0x390, 0x2b2, 0x327, 0x305, 0x04d,
+ 0x141, 0x30f, 0x000, 0x350, 0x044, 0x275, 0x00b, 0x0c0,
+ 0x2a9, 0x0c0, 0x303, 0x362, 0x303, 0x008, 0x00f, 0x017,
+ 0x397, 0x2b2, 0x300, 0x303, 0x04c, 0x2a7, 0x362, 0x303,
+ 0x004, 0x00f, 0x006, 0x2c0, 0x2a9, 0x000, 0x004, 0x2c2,
+ 0x2a9, 0x141, 0x303, 0x000, 0x3e8, 0x30a, 0x2a9, 0x20a,
+ 0x24a, 0x2a9, 0x044, 0x022, 0x009, 0x3c4, 0x3ef, 0x026,
+ 0x045, 0x307, 0x2b2, 0x300, 0x2b3, 0x247, 0x2a1, 0x362,
+ 0x2a1, 0x000, 0x006, 0x004, 0x141, 0x2a1, 0x30a, 0x2a1,
+ 0x300, 0x2aa, 0x247, 0x2a1, 0x307, 0x2b4, 0x300, 0x2b5,
+ 0x247, 0x2a2, 0x362, 0x2a2, 0x000, 0x006, 0x004, 0x141,
+ 0x2a2, 0x30a, 0x2a2, 0x300, 0x2ab, 0x247, 0x2a2, 0x307,
+ 0x2b6, 0x300, 0x2b7, 0x247, 0x2a3, 0x362, 0x2a3, 0x000,
+ 0x006, 0x004, 0x141, 0x2a3, 0x30a, 0x2a3, 0x300, 0x2ac,
+ 0x247, 0x2a3, 0x307, 0x2b8, 0x300, 0x2b9, 0x247, 0x2a4,
+ 0x362, 0x2a4, 0x000, 0x006, 0x004, 0x141, 0x2a4, 0x30a,
+ 0x2a4, 0x300, 0x2ad, 0x247, 0x2a4, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x044, 0x3f0, 0x002, 0x045, 0x0c0,
+ 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x00c, 0x397, 0x2b2,
+ 0x300, 0x30f, 0x0a0, 0x04d, 0x141, 0x30f, 0x000, 0x3f3,
+ 0x045, 0x364, 0x2d4, 0x002, 0x001, 0x003, 0x045, 0x0c7,
+ 0x2ce, 0x347, 0x290, 0x2fb, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003, 0x309,
+ 0x013, 0x34b, 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336,
+ 0x014, 0x307, 0x2ae, 0x300, 0x2ba, 0x114, 0x247, 0x2a5,
+ 0x307, 0x2af, 0x300, 0x2bd, 0x104, 0x247, 0x2a6, 0x3c7,
+ 0x3ff, 0x301, 0x151, 0x2a6, 0x141, 0x2a5, 0x364, 0x2a6,
+ 0x100, 0x001, 0x004, 0x000, 0x01a, 0x364, 0x2a5, 0x100,
+ 0x001, 0x004, 0x000, 0x013, 0x044, 0x014, 0x003, 0x044,
+ 0x30e, 0x002, 0x307, 0x2cd, 0x0c0, 0x336, 0x302, 0x00e,
+ 0x343, 0x00f, 0x336, 0x00a, 0x3df, 0x3ce, 0x3ff, 0x2fe,
+ 0x141, 0x2a6, 0x00b, 0x004, 0x0c0, 0x2a6, 0x151, 0x2a5,
+ 0x00b, 0x004, 0x0c0, 0x2a5, 0x362, 0x2a6, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a6, 0x362, 0x2a5, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a5, 0x044, 0x014, 0x003, 0x309,
+ 0x303, 0x242, 0x290, 0x0c0, 0x304, 0x362, 0x304, 0x003,
+ 0x00f, 0x02c, 0x340, 0x303, 0x290, 0x362, 0x290, 0x000,
+ 0x00f, 0x004, 0x0c0, 0x290, 0x362, 0x290, 0x0ff, 0x006,
+ 0x004, 0x000, 0x01b, 0x044, 0x034, 0x003, 0x044, 0x1cc,
+ 0x003, 0x044, 0x30e, 0x002, 0x0c0, 0x2ff, 0x044, 0x195,
+ 0x003, 0x140, 0x2ff, 0x001, 0x005, 0x347, 0x290, 0x305,
+ 0x141, 0x304, 0x000, 0x3d3, 0x362, 0x301, 0x000, 0x00f,
+ 0x01b, 0x347, 0x305, 0x301, 0x347, 0x2fb, 0x290, 0x3c6,
+ 0x0ff, 0x01a, 0x307, 0x2ae, 0x300, 0x2be, 0x114, 0x247,
+ 0x2a5, 0x307, 0x2af, 0x300, 0x2c1, 0x104, 0x247, 0x2a6,
+ 0x000, 0x36a, 0x307, 0x305, 0x300, 0x301, 0x24a, 0x290,
+ 0x044, 0x034, 0x003, 0x045, 0x0c0, 0x2e4, 0x0c3, 0x2ce,
+ 0x044, 0x1bf, 0x003, 0x0c0, 0x2ea, 0x044, 0x078, 0x009,
+ 0x347, 0x28f, 0x318, 0x347, 0x28e, 0x319, 0x0c0, 0x31b,
+ 0x0c0, 0x31c, 0x3c7, 0x038, 0x317, 0x364, 0x2d4, 0x020,
+ 0x001, 0x005, 0x347, 0x2c6, 0x317, 0x364, 0x2d4, 0x002,
+ 0x001, 0x005, 0x347, 0x290, 0x317, 0x0c0, 0x31e, 0x0cf,
+ 0x324, 0x364, 0x2d4, 0x004, 0x001, 0x008, 0x347, 0x28f,
+ 0x31e, 0x347, 0x28f, 0x324, 0x0c0, 0x31f, 0x0cf, 0x325,
+ 0x364, 0x2d4, 0x008, 0x001, 0x008, 0x347, 0x28e, 0x31f,
+ 0x347, 0x28e, 0x325, 0x364, 0x2d4, 0x010, 0x001, 0x00d,
+ 0x347, 0x291, 0x320, 0x347, 0x291, 0x326, 0x347, 0x291,
+ 0x31a, 0x000, 0x00a, 0x0cd, 0x291, 0x0cd, 0x31a, 0x0c0,
+ 0x320, 0x0cd, 0x326, 0x044, 0x1bf, 0x00a, 0x364, 0x24d,
+ 0x030, 0x009, 0x028, 0x309, 0x2c7, 0x34b, 0x2c8, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x34e, 0x2c8, 0x337, 0x32c,
+ 0x2c7, 0x282, 0x343, 0x337, 0x336, 0x247, 0x332, 0x347,
+ 0x336, 0x333, 0x307, 0x332, 0x347, 0x333, 0x336, 0x300,
+ 0x2c7, 0x341, 0x2c8, 0x336, 0x247, 0x334, 0x347, 0x336,
+ 0x335, 0x0c0, 0x398, 0x364, 0x2df, 0x002, 0x001, 0x006,
+ 0x0c1, 0x39a, 0x000, 0x004, 0x0c4, 0x39a, 0x347, 0x2cd,
+ 0x397, 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x39a, 0x006,
+ 0x076, 0x044, 0x16d, 0x00a, 0x0cf, 0x28f, 0x362, 0x28f,
+ 0x000, 0x007, 0x065, 0x362, 0x28f, 0x00f, 0x00f, 0x009,
+ 0x08f, 0x302, 0x28f, 0x247, 0x28e, 0x000, 0x004, 0x0c0,
+ 0x28e, 0x362, 0x28f, 0x006, 0x00f, 0x004, 0x0cf, 0x28e,
+ 0x0c1, 0x394, 0x362, 0x28e, 0x00a, 0x006, 0x004, 0x0d1,
+ 0x394, 0x0c0, 0x2a9, 0x307, 0x2a9, 0x302, 0x394, 0x006,
+ 0x03b, 0x347, 0x2a9, 0x313, 0x307, 0x2aa, 0x300, 0x2a9,
+ 0x247, 0x2a1, 0x307, 0x2ab, 0x300, 0x2a9, 0x247, 0x2a2,
+ 0x307, 0x2ac, 0x302, 0x2a9, 0x247, 0x2a3, 0x307, 0x2ad,
+ 0x302, 0x2a9, 0x247, 0x2a4, 0x044, 0x12e, 0x00a, 0x347,
+ 0x31d, 0x290, 0x307, 0x290, 0x302, 0x323, 0x007, 0x00a,
+ 0x044, 0x015, 0x00a, 0x340, 0x329, 0x290, 0x000, 0x3f4,
+ 0x307, 0x314, 0x300, 0x315, 0x247, 0x316, 0x141, 0x2a9,
+ 0x000, 0x3c3, 0x151, 0x28f, 0x000, 0x39a, 0x340, 0x317,
+ 0x398, 0x141, 0x399, 0x000, 0x388, 0x347, 0x398, 0x317,
+ 0x362, 0x39a, 0x004, 0x009, 0x007, 0x307, 0x398, 0x191,
+ 0x247, 0x317, 0x044, 0x177, 0x00a, 0x347, 0x317, 0x2e9,
+ 0x347, 0x31b, 0x2a9, 0x347, 0x2a9, 0x313, 0x347, 0x316,
+ 0x2ea, 0x0c0, 0x395, 0x347, 0x31e, 0x28f, 0x307, 0x28f,
+ 0x302, 0x324, 0x007, 0x093, 0x0c3, 0x2ce, 0x347, 0x325,
+ 0x28e, 0x347, 0x326, 0x291, 0x044, 0x3ad, 0x002, 0x0c3,
+ 0x303, 0x044, 0x3d4, 0x009, 0x347, 0x2a5, 0x2e2, 0x364,
+ 0x2d4, 0x008, 0x001, 0x007, 0x364, 0x2d4, 0x010, 0x009,
+ 0x05d, 0x0cf, 0x32e, 0x362, 0x32e, 0x000, 0x007, 0x02d,
+ 0x364, 0x2d4, 0x008, 0x009, 0x00c, 0x347, 0x32e, 0x28e,
+ 0x362, 0x28e, 0x00f, 0x006, 0x004, 0x0cf, 0x28e, 0x364,
+ 0x2d4, 0x010, 0x009, 0x005, 0x347, 0x32e, 0x291, 0x044,
+ 0x3ad, 0x002, 0x0c0, 0x303, 0x044, 0x3d4, 0x009, 0x307,
+ 0x2a5, 0x302, 0x2e2, 0x00e, 0x004, 0x000, 0x006, 0x151,
+ 0x32e, 0x000, 0x3d2, 0x364, 0x2d4, 0x008, 0x001, 0x006,
+ 0x141, 0x291, 0x000, 0x022, 0x307, 0x28e, 0x302, 0x325,
+ 0x006, 0x01c, 0x307, 0x2a5, 0x302, 0x2e2, 0x00e, 0x016,
+ 0x141, 0x28e, 0x307, 0x28e, 0x302, 0x325, 0x009, 0x004,
+ 0x000, 0x00c, 0x044, 0x034, 0x003, 0x0c0, 0x303, 0x044,
+ 0x3d4, 0x009, 0x000, 0x3e2, 0x362, 0x291, 0x00d, 0x006,
+ 0x004, 0x0cd, 0x291, 0x397, 0x39b, 0x300, 0x28f, 0x327,
+ 0x28e, 0x04d, 0x397, 0x3ab, 0x300, 0x28f, 0x327, 0x291,
+ 0x04d, 0x141, 0x28f, 0x000, 0x36b, 0x044, 0x16d, 0x00a,
+ 0x347, 0x2c7, 0x2c2, 0x347, 0x2c8, 0x2c3, 0x0c4, 0x397,
+ 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x2e0, 0x006, 0x00f,
+ 0x349, 0x2c7, 0x2c7, 0x34b, 0x2c8, 0x2c8, 0x349, 0x397,
+ 0x397, 0x141, 0x399, 0x000, 0x3ef, 0x044, 0x1bf, 0x00a,
+ 0x347, 0x31e, 0x28f, 0x307, 0x28f, 0x302, 0x324, 0x007,
+ 0x055, 0x0c3, 0x2ce, 0x397, 0x39b, 0x300, 0x28f, 0x04c,
+ 0x2c7, 0x28e, 0x397, 0x3ab, 0x300, 0x28f, 0x04c, 0x2c7,
+ 0x291, 0x347, 0x317, 0x290, 0x044, 0x3ad, 0x002, 0x0c0,
+ 0x32e, 0x362, 0x32e, 0x010, 0x00f, 0x023, 0x0c1, 0x30a,
+ 0x0c0, 0x330, 0x362, 0x330, 0x004, 0x00f, 0x013, 0x364,
+ 0x026, 0x010, 0x009, 0x00a, 0x362, 0x32e, 0x002, 0x006,
+ 0x005, 0x044, 0x1e6, 0x00a, 0x141, 0x330, 0x000, 0x3ec,
+ 0x3c4, 0x3ef, 0x026, 0x141, 0x32e, 0x000, 0x3dc, 0x0c3,
+ 0x2ce, 0x044, 0x1bf, 0x003, 0x044, 0x30e, 0x002, 0x044,
+ 0x015, 0x00a, 0x307, 0x314, 0x300, 0x315, 0x247, 0x316,
+ 0x141, 0x28f, 0x000, 0x3a9, 0x044, 0x3b3, 0x002, 0x362,
+ 0x316, 0x006, 0x00f, 0x004, 0x0c0, 0x396, 0x044, 0x177,
+ 0x00a, 0x347, 0x316, 0x2ec, 0x347, 0x2c2, 0x2c7, 0x347,
+ 0x2c3, 0x2c8, 0x045, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x207, 0x384, 0x008, 0x247, 0x304, 0x09c, 0x189, 0x395,
+ 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x3c4, 0x300, 0x01a,
+ 0x140, 0x303, 0x009, 0x010, 0x140, 0x304, 0x009, 0x007,
+ 0x3c5, 0x004, 0x01a, 0x000, 0x013, 0x3c5, 0x020, 0x01a,
+ 0x000, 0x00e, 0x140, 0x304, 0x009, 0x007, 0x3c5, 0x080,
+ 0x01a, 0x000, 0x005, 0x3c5, 0x001, 0x01a, 0x044, 0x1ba,
+ 0x003, 0x3d7, 0x200, 0x013, 0x0c0, 0x014, 0x397, 0x2a5,
+ 0x044, 0x1e9, 0x003, 0x045, 0x0c0, 0x303, 0x0c0, 0x330,
+ 0x0c0, 0x331, 0x0c0, 0x32e, 0x0c0, 0x32f, 0x044, 0x3ad,
+ 0x002, 0x0c0, 0x303, 0x307, 0x303, 0x302, 0x2d1, 0x006,
+ 0x0bb, 0x364, 0x303, 0x001, 0x001, 0x016, 0x307, 0x2ae,
+ 0x302, 0x313, 0x114, 0x302, 0x315, 0x247, 0x2a5, 0x307,
+ 0x2af, 0x302, 0x313, 0x104, 0x300, 0x315, 0x247, 0x2a6,
+ 0x000, 0x014, 0x307, 0x2ae, 0x300, 0x2a9, 0x114, 0x302,
+ 0x314, 0x247, 0x2a5, 0x307, 0x2af, 0x300, 0x2a9, 0x104,
+ 0x300, 0x314, 0x247, 0x2a6, 0x362, 0x2a5, 0x000, 0x00f,
+ 0x003, 0x045, 0x362, 0x2a6, 0x0ff, 0x006, 0x003, 0x045,
+ 0x044, 0x3f0, 0x002, 0x362, 0x303, 0x002, 0x00f, 0x02c,
+ 0x083, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x044, 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x334,
+ 0x347, 0x335, 0x336, 0x302, 0x046, 0x343, 0x047, 0x336,
+ 0x00a, 0x00e, 0x307, 0x046, 0x347, 0x047, 0x336, 0x302,
+ 0x332, 0x343, 0x333, 0x336, 0x002, 0x006, 0x347, 0x323,
+ 0x290, 0x045, 0x307, 0x303, 0x384, 0x001, 0x104, 0x18a,
+ 0x385, 0x101, 0x207, 0x0a0, 0x044, 0x18a, 0x002, 0x044,
+ 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x046, 0x347,
+ 0x047, 0x336, 0x302, 0x332, 0x343, 0x333, 0x336, 0x002,
+ 0x006, 0x347, 0x323, 0x290, 0x045, 0x364, 0x303, 0x001,
+ 0x001, 0x00a, 0x340, 0x00e, 0x330, 0x341, 0x00f, 0x331,
+ 0x000, 0x008, 0x340, 0x00e, 0x32e, 0x341, 0x00f, 0x32f,
+ 0x307, 0x2fd, 0x347, 0x2fe, 0x336, 0x327, 0x32e, 0x347,
+ 0x32f, 0x337, 0x320, 0x330, 0x341, 0x331, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x003, 0x045, 0x141, 0x303,
+ 0x000, 0x343, 0x347, 0x290, 0x317, 0x347, 0x28f, 0x318,
+ 0x347, 0x28e, 0x319, 0x347, 0x291, 0x31a, 0x347, 0x2a9,
+ 0x31b, 0x347, 0x313, 0x31c, 0x347, 0x293, 0x396, 0x307,
+ 0x32e, 0x347, 0x32f, 0x336, 0x300, 0x330, 0x341, 0x331,
+ 0x336, 0x247, 0x2fd, 0x347, 0x336, 0x2fe, 0x307, 0x2fd,
+ 0x347, 0x2fe, 0x336, 0x327, 0x397, 0x0c0, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x019, 0x30a, 0x397, 0x327,
+ 0x330, 0x282, 0x00a, 0x004, 0x141, 0x315, 0x30a, 0x397,
+ 0x327, 0x32e, 0x282, 0x00a, 0x004, 0x141, 0x314, 0x3ce,
+ 0x3ff, 0x2fe, 0x000, 0x2ea, 0x045, 0x364, 0x2d4, 0x002,
+ 0x001, 0x00c, 0x347, 0x317, 0x31d, 0x347, 0x317, 0x323,
+ 0x0c1, 0x329, 0x000, 0x031, 0x362, 0x316, 0x000, 0x00e,
+ 0x007, 0x364, 0x2d4, 0x020, 0x001, 0x01f, 0x307, 0x317,
+ 0x114, 0x247, 0x31d, 0x307, 0x317, 0x104, 0x247, 0x323,
+ 0x0c4, 0x329, 0x362, 0x31d, 0x000, 0x00f, 0x004, 0x0c0,
+ 0x31d, 0x362, 0x323, 0x0b8, 0x006, 0x00f, 0x3c7, 0x0b8,
+ 0x323, 0x000, 0x00a, 0x3c7, 0x038, 0x31d, 0x3c7, 0x0b8,
+ 0x323, 0x0c4, 0x329, 0x045, 0x0c0, 0x314, 0x0c0, 0x315,
+ 0x0c0, 0x316, 0x3ce, 0x3ff, 0x2fe, 0x045, 0x347, 0x319,
+ 0x28e, 0x347, 0x318, 0x28f, 0x347, 0x317, 0x290, 0x347,
+ 0x31a, 0x291, 0x347, 0x396, 0x293, 0x347, 0x31b, 0x2bd,
+ 0x347, 0x2bd, 0x2bc, 0x347, 0x2bc, 0x2bb, 0x347, 0x2bb,
+ 0x2ba, 0x347, 0x2ba, 0x2b5, 0x347, 0x2b5, 0x2b4, 0x347,
+ 0x2b4, 0x2b3, 0x347, 0x2b3, 0x2b2, 0x080, 0x302, 0x31c,
+ 0x247, 0x2c1, 0x347, 0x2c1, 0x2c0, 0x347, 0x2c0, 0x2bf,
+ 0x347, 0x2bf, 0x2be, 0x347, 0x2be, 0x2b9, 0x347, 0x2b9,
+ 0x2b8, 0x347, 0x2b8, 0x2b7, 0x347, 0x2b7, 0x2b6, 0x044,
+ 0x3ad, 0x002, 0x044, 0x022, 0x009, 0x045, 0x309, 0x2c7,
+ 0x34b, 0x2c8, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209,
+ 0x34b, 0x336, 0x336, 0x300, 0x2c7, 0x341, 0x2c8, 0x336,
+ 0x247, 0x332, 0x347, 0x336, 0x333, 0x309, 0x2c7, 0x34b,
+ 0x2c8, 0x336, 0x300, 0x332, 0x341, 0x333, 0x336, 0x247,
+ 0x334, 0x347, 0x336, 0x335, 0x045, 0x3a7, 0x022, 0x090,
+ 0x364, 0x2d4, 0x001, 0x001, 0x00b, 0x364, 0x2cf, 0x001,
+ 0x001, 0x005, 0x044, 0x1a9, 0x000, 0x045, 0x044, 0x1b7,
+ 0x000, 0x362, 0x2f6, 0x002, 0x00f, 0x02b, 0x0c0, 0x304,
+ 0x044, 0x25b, 0x003, 0x347, 0x2a5, 0x2e2, 0x001, 0x007,
+ 0x347, 0x2a6, 0x2e1, 0x009, 0x005, 0x141, 0x2ee, 0x045,
+ 0x364, 0x2f6, 0x001, 0x001, 0x037, 0x0c1, 0x304, 0x044,
+ 0x25b, 0x003, 0x362, 0x2a5, 0x0ff, 0x001, 0x007, 0x362,
+ 0x2a6, 0x0ff, 0x009, 0x028, 0x141, 0x2ee, 0x045, 0x307,
+ 0x2bb, 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2bd, 0x300,
+ 0x2af, 0x247, 0x2a6, 0x347, 0x2a5, 0x2e2, 0x347, 0x2a6,
+ 0x2e1, 0x364, 0x2f6, 0x001, 0x001, 0x00e, 0x307, 0x2be,
+ 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2c0, 0x300, 0x2af,
+ 0x247, 0x2a6, 0x044, 0x1bf, 0x003, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x00f, 0x364, 0x2cf, 0x001, 0x001, 0x007, 0x3d7,
+ 0x200, 0x013, 0x0c0, 0x014, 0x3c5, 0x100, 0x00d, 0x0c6,
+ 0x2ce, 0x34e, 0x306, 0x306, 0x0c2, 0x308, 0x362, 0x308,
+ 0x00d, 0x00e, 0x019, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x04c, 0x2c7, 0x303, 0x34e, 0x306, 0x306, 0x364, 0x306,
+ 0x001, 0x009, 0x005, 0x044, 0x2fa, 0x00a, 0x141, 0x308,
+ 0x000, 0x3e6, 0x140, 0x30a, 0x001, 0x05c, 0x0c0, 0x303,
+ 0x0c0, 0x304, 0x362, 0x304, 0x00b, 0x00e, 0x048, 0x397,
+ 0x294, 0x300, 0x304, 0x04c, 0x2c7, 0x305, 0x362, 0x305,
+ 0x000, 0x00f, 0x007, 0x308, 0x305, 0x101, 0x247, 0x305,
+ 0x307, 0x305, 0x302, 0x303, 0x00f, 0x005, 0x347, 0x305,
+ 0x303, 0x362, 0x305, 0x00f, 0x009, 0x025, 0x141, 0x293,
+ 0x0c0, 0x30a, 0x0c4, 0x2f7, 0x0c0, 0x304, 0x362, 0x304,
+ 0x00b, 0x00e, 0x01c, 0x397, 0x294, 0x300, 0x304, 0x04c,
+ 0x2c7, 0x305, 0x397, 0x294, 0x300, 0x304, 0x329, 0x305,
+ 0x320, 0x305, 0x2aa, 0x2aa, 0x04d, 0x141, 0x304, 0x000,
+ 0x3e7, 0x141, 0x304, 0x000, 0x3b7, 0x362, 0x303, 0x00a,
+ 0x00f, 0x008, 0x151, 0x293, 0x0c0, 0x30a, 0x0c4, 0x2f7,
+ 0x044, 0x06e, 0x003, 0x044, 0x096, 0x003, 0x045, 0x307,
+ 0x2a5, 0x347, 0x2e2, 0x2a5, 0x247, 0x2e2, 0x307, 0x2a6,
+ 0x347, 0x2e1, 0x2a6, 0x247, 0x2e1, 0x044, 0x014, 0x003,
+ 0x045, 0x347, 0x303, 0x304, 0x347, 0x304, 0x30e, 0x3ce,
+ 0x3ff, 0x2fe, 0x152, 0x303, 0x0c0, 0x305, 0x362, 0x305,
+ 0x003, 0x00f, 0x0b9, 0x141, 0x303, 0x362, 0x303, 0x3f1,
+ 0x00f, 0x004, 0x000, 0x0ac, 0x362, 0x303, 0x00f, 0x006,
+ 0x007, 0x347, 0x30c, 0x30d, 0x000, 0x0a6, 0x307, 0x308,
+ 0x112, 0x390, 0x294, 0x327, 0x303, 0x04d, 0x044, 0x096,
+ 0x003, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002, 0x00f,
+ 0x00d, 0x08c, 0x189, 0x395, 0x104, 0x328, 0x308, 0x121,
+ 0x3a4, 0x0ff, 0x000, 0x007, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x30e, 0x002, 0x347,
+ 0x00e, 0x32e, 0x347, 0x00f, 0x32f, 0x364, 0x2f6, 0x001,
+ 0x001, 0x024, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x010, 0x08e, 0x189, 0x395, 0x104, 0x328, 0x308,
+ 0x121, 0x3a4, 0x0ff, 0x044, 0x18a, 0x002, 0x000, 0x005,
+ 0x3c6, 0x0ff, 0x01a, 0x044, 0x30e, 0x002, 0x340, 0x00e,
+ 0x32e, 0x341, 0x00f, 0x32f, 0x140, 0x305, 0x001, 0x007,
+ 0x362, 0x303, 0x3f1, 0x009, 0x005, 0x347, 0x32f, 0x30b,
+ 0x362, 0x305, 0x001, 0x009, 0x005, 0x347, 0x32f, 0x30c,
+ 0x362, 0x305, 0x002, 0x009, 0x005, 0x347, 0x32f, 0x30d,
+ 0x362, 0x32f, 0x000, 0x00f, 0x00d, 0x348, 0x32e, 0x32e,
+ 0x348, 0x32f, 0x32f, 0x141, 0x32e, 0x3c1, 0x000, 0x32f,
+ 0x307, 0x32e, 0x347, 0x32f, 0x336, 0x302, 0x2fd, 0x343,
+ 0x2fe, 0x336, 0x002, 0x00b, 0x347, 0x303, 0x304, 0x347,
+ 0x32e, 0x2fd, 0x347, 0x32f, 0x2fe, 0x140, 0x305, 0x009,
+ 0x007, 0x141, 0x2fd, 0x3c1, 0x000, 0x2fe, 0x141, 0x305,
+ 0x000, 0x346, 0x307, 0x30b, 0x306, 0x30d, 0x003, 0x01d,
+ 0x307, 0x30e, 0x111, 0x247, 0x304, 0x362, 0x30c, 0x000,
+ 0x00f, 0x004, 0x142, 0x304, 0x362, 0x304, 0x3f1, 0x00f,
+ 0x005, 0x3c7, 0x3f1, 0x304, 0x362, 0x304, 0x00f, 0x006,
+ 0x004, 0x0cf, 0x304, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x327, 0x304, 0x04d, 0x045, 0x044, 0x290, 0x001, 0x3c4,
+ 0x3fe, 0x200, 0x020, 0x2e0, 0x002, 0x045, 0x1f6, 0x185,
+ 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff,
+ 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce,
+ 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020, 0x009,
+ 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9, 0x286,
+ 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x3f7, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051, 0x2c7,
+ 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b, 0x005,
+ 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4, 0x00f,
+ 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288, 0x286,
+ 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b, 0x3fc,
+ 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285, 0x307,
+ 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd, 0x000,
+ 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c,
+ 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002, 0x005,
+ 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b, 0x1f0,
+ 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081, 0x138,
+ 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080,
+ 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d, 0x1e0,
+ 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b, 0x347,
+ 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044, 0x154,
+ 0x00b, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364, 0x233,
+ 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff, 0x242,
+ 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289, 0x397,
+ 0x00b, 0x189, 0x395, 0x062, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9, 0x121,
+ 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282,
+ 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000, 0x3ed,
+ 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233, 0x002,
+ 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff, 0x281,
+ 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366, 0x282,
+ 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282, 0x044,
+ 0x154, 0x00b, 0x001, 0x00d, 0x141, 0x28b, 0x362, 0x28b,
+ 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5, 0x362,
+ 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397, 0x00b,
+ 0x189, 0x395, 0x070, 0x044, 0x2ea, 0x001, 0x009, 0x004,
+ 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005, 0x105,
+ 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121,
+ 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003,
+ 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387, 0x00b,
+ 0x189, 0x395, 0x06b, 0x300, 0x28a, 0x050, 0x247, 0x28a,
+ 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a, 0x044,
+ 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282, 0x281,
+ 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045, 0x0c0,
+ 0x244, 0x044, 0x172, 0x000, 0x043, 0x047, 0x0a1, 0x1aa,
+ 0x2c5, 0x233, 0x0c0, 0x244, 0x044, 0x172, 0x000, 0x364,
+ 0x233, 0x001, 0x029, 0x0f4, 0x000, 0x047, 0x000, 0x3e9,
+ 0x364, 0x025, 0x004, 0x009, 0x0d4, 0x307, 0x2cf, 0x18a,
+ 0x003, 0x006, 0x002, 0x08c, 0x000, 0x0bd, 0x364, 0x200,
+ 0x100, 0x009, 0x0c0, 0x0c0, 0x2e6, 0x0c0, 0x2e7, 0x08d,
+ 0x18b, 0x385, 0x090, 0x040, 0x083, 0x040, 0x387, 0x080,
+ 0x044, 0x3e3, 0x001, 0x044, 0x346, 0x002, 0x3c5, 0x001,
+ 0x2cf, 0x0c0, 0x2d8, 0x0c0, 0x2f1, 0x3c4, 0x3ef, 0x027,
+ 0x387, 0x085, 0x187, 0x385, 0x080, 0x247, 0x041, 0x3c5,
+ 0x020, 0x00c, 0x0c0, 0x2f6, 0x044, 0x3b3, 0x002, 0x140,
+ 0x2d3, 0x009, 0x026, 0x364, 0x292, 0x008, 0x009, 0x021,
+ 0x044, 0x16d, 0x009, 0x0d0, 0x312, 0x0c0, 0x30a, 0x0c0,
+ 0x2ee, 0x0d0, 0x2f7, 0x044, 0x253, 0x00b, 0x342, 0x293,
+ 0x312, 0x001, 0x00e, 0x0c1, 0x30a, 0x347, 0x293, 0x312,
+ 0x0c8, 0x2f7, 0x044, 0x253, 0x00b, 0x000, 0x3f1, 0x3c4,
+ 0x1ff, 0x2cf, 0x0c0, 0x30a, 0x307, 0x290, 0x183, 0x305,
+ 0x28f, 0x183, 0x305, 0x28e, 0x247, 0x2f2, 0x0cf, 0x2ce,
+ 0x044, 0x113, 0x003, 0x044, 0x33f, 0x002, 0x055, 0x364,
+ 0x2cf, 0x020, 0x009, 0x005, 0x3c4, 0x3bf, 0x200, 0x3c4,
+ 0x3ee, 0x2cf, 0x364, 0x2df, 0x001, 0x009, 0x04a, 0x364,
+ 0x33c, 0x002, 0x009, 0x045, 0x141, 0x2cf, 0x044, 0x346,
+ 0x002, 0x3c5, 0x002, 0x2cf, 0x347, 0x2d5, 0x2f7, 0x0c4,
+ 0x303, 0x044, 0x08a, 0x009, 0x044, 0x333, 0x008, 0x151,
+ 0x2f7, 0x00b, 0x3f6, 0x347, 0x2d5, 0x2f7, 0x044, 0x253,
+ 0x00b, 0x055, 0x3c4, 0x3fc, 0x2cf, 0x364, 0x2cf, 0x040,
+ 0x009, 0x019, 0x3c4, 0x39d, 0x2cf, 0x0ce, 0x2ce, 0x364,
+ 0x2cf, 0x001, 0x009, 0x007, 0x0cf, 0x2ce, 0x3c4, 0x3fe,
+ 0x2cf, 0x364, 0x2cf, 0x010, 0x009, 0x005, 0x3c4, 0x3bf,
+ 0x200, 0x3c4, 0x1ff, 0x2cf, 0x044, 0x33f, 0x002, 0x020,
+ 0x0d3, 0x000, 0x347, 0x2cb, 0x306, 0x044, 0x1e6, 0x00a,
+ 0x044, 0x333, 0x008, 0x151, 0x2f7, 0x009, 0x3f5, 0x045,
+ 0x397, 0x201, 0x0a0, 0x044, 0x18a, 0x002, 0x020, 0x1bf,
+ 0x003, 0x397, 0x2ba, 0x04c, 0x118, 0x04d, 0x109, 0x272,
+ 0x2c1, 0x00b, 0x3fa, 0x045, 0x397, 0x2b2, 0x04c, 0x2c7,
+ 0x303, 0x101, 0x04c, 0x322, 0x303, 0x00b, 0x009, 0x04c,
+ 0x111, 0x04d, 0x101, 0x327, 0x303, 0x04d, 0x101, 0x272,
+ 0x2c1, 0x00b, 0x3ed, 0x045, 0x2f4, 0x200, 0x001, 0x008,
+ 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287, 0x385,
+ 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010, 0x029,
+ 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384, 0x0ff,
+ 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d, 0x040,
+ 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040, 0x02d,
+ 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d, 0x042,
+ 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005, 0x3c5,
+ 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264, 0x080,
+ 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5, 0x080,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f, 0x02d,
+ 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff, 0x1b8,
+ 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4,
+ 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a, 0x020,
+ 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044,
+ 0x340, 0x001, 0x247, 0x249, 0x054, 0x347, 0x249, 0x248,
+ 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f,
+ 0x001, 0x001, 0x006, 0x245, 0x057, 0x000, 0x005, 0x208,
+ 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f,
+ 0x002, 0x001, 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001,
+ 0x000, 0x005, 0x208, 0x244, 0x057, 0x020, 0x181, 0x006,
+ 0x04e, 0x0ce, 0x045, 0x03c, 0x0f5, 0x04e, 0x0fd, 0x07e,
+ 0x00a, 0x0f9
+
+}; /* end fm10000_serdes_swap_code_prd2 */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.h b/drivers/net/fm10k/switch/fm10k_spico_code.h
new file mode 100644
index 0000000..c172638
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SPICO_CODE_H_
+#define _FM10K_SPICO_CODE_H_
+#include <stdint.h>
+
+extern const uint32_t fm10000_sbus_master_code_version_build_id_prd;
+extern const uint32_t fm10000_sbus_master_code_size_prd;
+extern const uint16_t fm10000_sbus_master_code_prd[];
+
+extern const uint32_t fm10000_serdes_spico_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_spico_code_size_prd2;
+extern const uint16_t fm10000_serdes_spico_code_prd2[];
+
+extern const uint32_t fm10000_serdes_swap_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_swap_code_size_prd2;
+extern const uint16_t fm10000_serdes_swap_code_prd2[];
+
+#endif /* _FM10K_SPICO_CODE_H_ */
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 3/7] net/fm10k: add config ffu statistics support
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 2/7] net/fm10k: add some modules of port Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 4/7] net/fm10k: add flow and switch management Xiaojun Liu
` (4 subsequent siblings)
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
To support switch management, add the following new files:
Add fm10k/switch/fm10k_config.h.
Add fm10k/switch/fm10k_config.c(support switch configuration)
Add fm10k/switch/fm10k_ffu.h
Add fm10k/switch/fm10k_ffu.c(support switch ffu rule)
Add fm10k/switch/fm10k_stats.h
Add fm10k/switch/fm10k_stats.c(support switch statistics)
and modify fm10k/Makefile(add fm10k_config.c,
fm10k_ffu.c, and fm10k_stats.c).
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 3 +
drivers/net/fm10k/switch/fm10k_config.c | 855 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_config.h | 171 +++++
drivers/net/fm10k/switch/fm10k_ffu.c | 1209 ++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++++++
7 files changed, 3768 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 4ec2a80..ed73251 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -90,6 +90,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_config.c b/drivers/net/fm10k/switch/fm10k_config.c
new file mode 100644
index 0000000..46c3fae
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.c
@@ -0,0 +1,855 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <unistd.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+
+#define FM10K_CONFIG_WORD_MAX 10
+#define FM10K_CONFIG_WORD_LEN 20
+#define FM10K_CONFIG_STR_MAX 100
+
+struct fm10k_cfg_key_item {
+ const char *key_str[FM10K_CONFIG_WORD_MAX];
+ uint8_t type;
+};
+
+static const char *fm10k_config_dpdk_conf_file = "/etc/dpdk_fm10k.conf";
+static struct fm10k_dpdk_cfg fm10k_config_dpdk_cfg;
+
+static struct fm10k_cfg_key_item fm10k_config_key_items[] = {
+ /* debug configuration */
+ { {"debug", "print", "enable"},
+ FM10K_CONFIG_DEBUG_ENABLE},
+ { {"debug", "print", "config"},
+ FM10K_CONFIG_DEBUG_CONFIG},
+ { {"debug", "print", "ffu", "init"},
+ FM10K_CONFIG_DEBUG_FFU_INIT},
+ { {"debug", "print", "ffu", "register"},
+ FM10K_CONFIG_DEBUG_FFU_REG},
+ { {"debug", "print", "ffu", "rule"},
+ FM10K_CONFIG_DEBUG_FFU_RULE},
+ { {"debug", "print", "stats", "port"},
+ FM10K_CONFIG_DEBUG_STATS_PORT},
+ { {"debug", "print", "stats", "queue"},
+ FM10K_CONFIG_DEBUG_STATS_QUEUE},
+ { {"debug", "print", "stats", "rule"},
+ FM10K_CONFIG_DEBUG_STATS_FFU},
+ { {"debug", "print", "stats", "detail"},
+ FM10K_CONFIG_DEBUG_STATS_MORE},
+ { {"debug", "print", "stats", "interval"},
+ FM10K_CONFIG_DEBUG_STATS_INTERVAL},
+ /* general configuration */
+ { {"dpdk", "bind", "pf", "number"},
+ FM10K_CONFIG_BIND_PF_NUMBER},
+ { {"extern", "port", "speed"},
+ FM10K_CONFIG_EXT_PORT_SPEED},
+ /* internal redirect configuration */
+ { {"dpdk", "port", "*", "map", "pf"},
+ FM10K_CONFIG_DPDK_PORT_MAP_PF},
+ { {"extern", "port", "*", "map", "pf"},
+ FM10K_CONFIG_EXT_PORT_MAP_PF},
+ /* external redirect configuration */
+ { {"flowset", "start"},
+ FM10K_CONFIG_FLOWSET_START},
+ { {"flowset", "stop"},
+ FM10K_CONFIG_FLOWSET_STOP},
+ { {"flowset", "enable"},
+ FM10K_CONFIG_FLOWSET_ENABLE},
+ { {"flow", "*", "condition", "source", "extern", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT},
+ { {"flow", "*", "condition", "source", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT},
+ { {"flow", "*", "condition", "vlan"},
+ FM10K_CONFIG_FLOW_COND_VLAN},
+ { {"flow", "*", "action", "forward", "extern", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT},
+ { {"flow", "*", "action", "forward", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT},
+ { {"flow", "*", "action", "forward", "vlan"},
+ FM10K_CONFIG_FLOW_ACT_FW_VALN},
+};
+
+static char default_flowset[10] = "default";
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_2pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PF.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF. ",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_4pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 4
+ },
+
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PFs.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF.",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+
+static int
+fm10k_config_conf_line_parser(char *buff, struct fm10k_cfg_config_item *item)
+{
+ char *p;
+ char *endptr;
+ const char *cfg_delims = { " " };
+ const char *key_delims = { "." };
+ struct fm10k_cfg_key_item *key;
+ char cfgs[3][FM10K_CONFIG_STR_MAX];
+ char cmp_keys[FM10K_CONFIG_WORD_MAX][FM10K_CONFIG_WORD_LEN];
+ long key_param = 0;
+ uint16_t i, j;
+ uint8_t val_type = FM10K_CONFIG_VALUE_NULL;
+
+ for (i = 0; i < 3; i++) {
+ if (i == 0)
+ p = strtok(buff, cfg_delims);
+ else
+ p = strtok(NULL, cfg_delims);
+ if (p == NULL)
+ return -1;
+ strncpy(cfgs[i], p, FM10K_CONFIG_STR_MAX);
+ }
+
+ p = strtok(NULL, cfg_delims);
+ if (p)
+ return -1;
+
+ memset(cmp_keys, 0, sizeof(cmp_keys));
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (i == 0)
+ p = strtok(cfgs[0], key_delims);
+ else
+ p = strtok(NULL, key_delims);
+ if (p == NULL)
+ break;
+ strncpy(cmp_keys[i], p, FM10K_CONFIG_WORD_LEN);
+ }
+
+ if (strcmp(cfgs[1], "int") == 0)
+ val_type = FM10K_CONFIG_VALUE_INT;
+ else if (strcmp(cfgs[1], "string") == 0)
+ val_type = FM10K_CONFIG_VALUE_STR;
+ else
+ return -1;
+
+ for (i = 0;
+ i < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ i++) {
+ key = &fm10k_config_key_items[i];
+ for (j = 0; j < FM10K_CONFIG_WORD_MAX; j++) {
+ if (key->key_str[j] &&
+ strlen(cmp_keys[j]) > 0) {
+ if (strcmp(key->key_str[j], "*") == 0) {
+ key_param =
+ strtol(cmp_keys[j], &endptr, 10);
+ if ((key_param == 0 &&
+ endptr == cmp_keys[j]) ||
+ (endptr != cmp_keys[j] &&
+ strlen(endptr) != 0))
+ break;
+ } else if (strcmp(key->key_str[j],
+ cmp_keys[j]) != 0) {
+ break;
+ }
+ } else if (key->key_str[j] == 0 &&
+ strlen(cmp_keys[j]) == 0) {
+ if (val_type == FM10K_CONFIG_VALUE_STR) {
+ item->val.str =
+ malloc(strlen(cfgs[2]) + 1);
+ if (item->val.str == NULL)
+ return -1;
+ strcpy(item->val.str, cfgs[2]);
+ } else {
+ item->val.int64 =
+ strtol(cfgs[2], NULL, 10);
+ }
+ item->type = key->type;
+ item->key_param = key_param;
+ item->val_type = val_type;
+ return 0;
+ }
+ }
+ }
+ return -1;
+}
+
+static bool
+fm10k_config_blank_line_check(char *line)
+{
+ uint16_t i;
+
+ for (i = 0; i < strlen(line); i++) {
+ if (line[i] == ' ' ||
+ line[i] == '\n' ||
+ line[i] == '\t' || line[i] == 0x0d) /* 0d: CR */
+ continue;
+ else
+ return false;
+ }
+ return true;
+}
+
+static int
+fm10k_config_conf_file_load(void)
+{
+ int i = 0;
+ FILE *fp;
+ char buff[255];
+ struct fm10k_cfg_config_item *item;
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "r");
+ if (fp == NULL)
+ return -1;
+
+ fm10k_config_dpdk_cfg.config_list = malloc
+ (sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+ if (fm10k_config_dpdk_cfg.config_list == NULL)
+ return -1;
+ memset(fm10k_config_dpdk_cfg.config_list, 0,
+ sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+
+ while (fgets(buff, sizeof(buff), fp)) {
+ if (buff[0] == '#' || buff[0] == 0)
+ continue;
+
+ if (fm10k_config_blank_line_check(buff))
+ continue;
+
+ item = &fm10k_config_dpdk_cfg.config_list[i++];
+ if (fm10k_config_conf_line_parser(buff, item) < 0) {
+ FM10K_SW_ERR("Unknown configuration: %s", buff);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+struct fm10k_hw*
+fm10k_config_hw_get(int port_no)
+{
+ int hw_port;
+
+ hw_port = fm10k_config_dpdk_cfg.dpdk_port_map[port_no].map_no[0];
+ return fm10k_config_dpdk_cfg.pf_hw[hw_port];
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_current_get(void)
+{
+ return fm10k_config_dpdk_cfg.current;
+}
+
+void
+fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new)
+{
+ fm10k_config_dpdk_cfg.current = new;
+}
+
+bool
+fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow)
+{
+ return (list == flow);
+}
+
+static void
+fm10k_config_flow_list_init(struct fm10k_cfg_flow *list)
+{
+ list->next = list;
+ list->prev = list;
+}
+
+static void
+fm10k_config_flow_list_add(struct fm10k_cfg_flow *new,
+ struct fm10k_cfg_flow *list)
+{
+ struct fm10k_cfg_flow *next = list->next;
+ next->prev = new;
+ new->next = next;
+ new->prev = list;
+ list->next = new;
+}
+
+void
+fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow)
+{
+ fm10k_config_flow_list_add(flow, flowset->flow_head.prev);
+}
+
+void
+fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow)
+{
+ flow->prev->next = flow->next;
+ flow->next->prev = flow->prev;
+ flow->prev = NULL;
+ flow->next = NULL;
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_get(const char *name)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ if (strcmp(flowset->name, name) == 0)
+ break;
+ flowset = flowset->next;
+ }
+
+ if (flowset == NULL) {
+ flowset = malloc(sizeof(struct fm10k_cfg_flowset));
+ if (flowset == NULL)
+ return NULL;
+ strcpy(flowset->name, name);
+ fm10k_config_flow_list_init(&flowset->flow_head);
+ flowset->next = fm10k_config_dpdk_cfg.flowset_head.next;
+ fm10k_config_dpdk_cfg.flowset_head.next = flowset;
+ }
+ return flowset;
+}
+
+
+static int
+fm10k_cfg_flow_item_set(struct fm10k_cfg_flowset *flowset,
+ uint8_t flow_no, uint8_t type, int64_t val)
+{
+ struct fm10k_cfg_flow *tmp;
+ struct fm10k_cfg_flow *flow = NULL;
+
+ if (flowset == NULL)
+ return -1;
+
+ tmp = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, tmp)) {
+ if (tmp->flow_no == flow_no) {
+ flow = tmp;
+ break;
+ }
+ tmp = tmp->next;
+ }
+ if (flow == NULL) {
+ flow = malloc(sizeof(struct fm10k_cfg_flow));
+ memset(flow, 0, sizeof(struct fm10k_cfg_flow));
+ flow->flow_no = flow_no;
+ fm10k_config_flow_list_add_tail(flowset, flow);
+ }
+ if (flow == NULL)
+ return -1;
+
+ switch (type) {
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ flow->src_port.vlan_id = val;
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flow->fw_port[0].vlan_id == 0)
+ flow->fw_port[0].vlan_id = val;
+ else
+ flow->fw_port[1].vlan_id = val;
+ break;
+ default:
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_transfer(void)
+{
+ int i;
+ int offset;
+ uint32_t tmp, data;
+ struct fm10k_cfg_config_item *item;
+ struct fm10k_cfg_flowset *flowset = NULL;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+
+ switch (item->type) {
+ case FM10K_CONFIG_BIND_PF_NUMBER:
+ fm10k_config_dpdk_cfg.pf_num = item->val.int64;
+ break;
+ case FM10K_CONFIG_EXT_PORT_SPEED:
+ fm10k_config_dpdk_cfg.ext_port_speed = item->val.int64;
+ break;
+ case FM10K_CONFIG_DPDK_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.dpdk_port_map[item->key_param];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+ case FM10K_CONFIG_EXT_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.ext_port_map
+ [item->key_param - 1];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+
+ case FM10K_CONFIG_DEBUG_ENABLE:
+ case FM10K_CONFIG_DEBUG_CONFIG:
+ case FM10K_CONFIG_DEBUG_FFU_INIT:
+ case FM10K_CONFIG_DEBUG_FFU_REG:
+ case FM10K_CONFIG_DEBUG_FFU_RULE:
+ case FM10K_CONFIG_DEBUG_STATS_PORT:
+ case FM10K_CONFIG_DEBUG_STATS_QUEUE:
+ case FM10K_CONFIG_DEBUG_STATS_FFU:
+ case FM10K_CONFIG_DEBUG_STATS_MORE:
+ offset = item->type - FM10K_CONFIG_DEBUG_START;
+ tmp = fm10k_config_dpdk_cfg.debug_cfg;
+ data = 1;
+ if (item->val.int64 != 1)
+ fm10k_config_dpdk_cfg.debug_cfg =
+ tmp & ~(data << offset);
+ else
+ fm10k_config_dpdk_cfg.debug_cfg |=
+ (data << offset);
+ break;
+ case FM10K_CONFIG_DEBUG_STATS_INTERVAL:
+ fm10k_config_dpdk_cfg.stats_interval = item->val.int64;
+ break;
+
+ case FM10K_CONFIG_FLOWSET_START:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ flowset = fm10k_config_flowset_get(item->val.str);
+ if (flowset == NULL)
+ return -1;
+ break;
+ case FM10K_CONFIG_FLOWSET_STOP:
+ flowset = NULL;
+ break;
+ case FM10K_CONFIG_FLOWSET_ENABLE:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ fm10k_config_dpdk_cfg.current =
+ fm10k_config_flowset_get(item->val.str);
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flowset == NULL ||
+ fm10k_cfg_flow_item_set(flowset,
+ item->key_param, item->type,
+ item->val.int64) != 0)
+ return -1;
+ break;
+ default:
+ return -1;
+ }
+ }
+ return 0;
+}
+
+static bool
+fm10k_config_conf_file_exist(void)
+{
+ if (access(fm10k_config_dpdk_conf_file, 0) == 0)
+ return true;
+ return false;
+}
+
+static int
+fm10k_config_conf_file_item_create(FILE *fp, char *buff,
+ struct fm10k_cfg_key_item *key,
+ struct fm10k_cfg_config_item *item)
+{
+ int i;
+
+ if (item->type != key->type)
+ return 0;
+
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (key->key_str[i] == 0)
+ break;
+ if (i == 0) {
+ sprintf(buff + strlen(buff),
+ "%s", key->key_str[i]);
+ } else {
+ if (strcmp(key->key_str[i], "*") == 0)
+ sprintf(buff + strlen(buff),
+ ".%u", item->key_param);
+ else
+ sprintf(buff + strlen(buff),
+ ".%s", key->key_str[i]);
+ }
+ }
+ if (item->val_type == FM10K_CONFIG_VALUE_INT)
+ sprintf(buff + strlen(buff),
+ " int %lld\n", (long long)item->val.int64);
+ else if (item->val_type == FM10K_CONFIG_VALUE_STR)
+ sprintf(buff + strlen(buff), " string %s\n", item->val.str);
+ else
+ return -1;
+ fwrite(buff, strlen(buff), 1, fp);
+ FM10K_SW_TRACE("[write] %s", buff);
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_create(void)
+{
+ uint16_t i, j;
+ struct fm10k_cfg_key_item *key;
+ struct fm10k_cfg_config_item *item;
+ FILE *fp;
+ char buff[255] = "";
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "w");
+ if (fp == NULL)
+ return -1;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+ buff[0] = 0;
+ if (strlen(item->describe) > 0)
+ sprintf(buff, "\n\n%s\n", item->describe);
+ for (j = 0;
+ j < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ j++) {
+ key = &fm10k_config_key_items[j];
+ if (fm10k_config_conf_file_item_create
+ (fp, buff, key, item) != 0) {
+ fclose(fp);
+ return -1;
+ }
+ }
+ }
+ fclose(fp);
+ return 0;
+}
+
+void
+fm10k_config_cfg_flowset_show(void)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ printf(" FLOWSET ENABLE: %s\n", fm10k_config_dpdk_cfg.current->name);
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ printf("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ printf(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d & %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ printf(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+}
+
+static void
+fm10k_config_cfg_describe(struct fm10k_switch *sw,
+ struct fm10k_device_info *info)
+{
+ uint16_t i;
+ struct fm10k_cfg_flowset *flowset;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ printf("--- FM10K STATIC CONFIG ---\n");
+ printf(" Card Type: %s\n", info->desc);
+ printf(" PF Max : %d\n", fm10k_config_dpdk_cfg.pf_max);
+ printf(" PF Bind : %d\n", fm10k_config_dpdk_cfg.pf_num);
+ printf(" DEBUG : %#x\n", fm10k_config_dpdk_cfg.debug_cfg);
+ printf(" STATS GAP: %d sec\n", fm10k_config_dpdk_cfg.stats_interval);
+ printf(" EXT PORT speed: %d Gbps\n",
+ fm10k_config_dpdk_cfg.ext_port_speed);
+ printf(" FLOWSET ENABLE: %s\n", fm10k_config_dpdk_cfg.current->name);
+
+ for (i = 0; i < fm10k_config_dpdk_cfg.ext_port_num; i++) {
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ printf(" EXT PORT[%d] MAP: PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0]);
+ else
+ printf(" EXT PORT[%d] MAP: PF%d PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[1]);
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_config_dpdk_cfg.dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ printf(" DPDK PORT[%d] MAP: PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0]);
+ else
+ printf(" DPDK PORT[%d] MAP: PF%d PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[1]);
+ }
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ printf("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ printf(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d & %4s PORT %d "
+ "VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ printf(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+ printf("\n");
+}
+
+int
+fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw)
+{
+ struct fm10k_device_info *info = fm10k_get_device_info(hw);
+
+ fm10k_config_dpdk_cfg.stats_interval = 2;
+ fm10k_config_dpdk_cfg.pf_max = info->num_peps;
+ fm10k_config_dpdk_cfg.ext_port_num = info->num_ext_ports;
+
+ if (!fm10k_config_conf_file_exist()) {
+ if (info->num_epls == 2 && info->num_peps == 2)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_2pep;
+ else if (info->num_epls == 2 && info->num_peps == 4)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_4pep;
+ else
+ return -1;
+
+ fm10k_config_conf_file_create();
+ } else {
+ if (fm10k_config_conf_file_load() < 0)
+ return -1;
+ }
+
+ if (fm10k_config_conf_file_transfer() < 0)
+ return -1;
+ sw->dpdk_cfg = &fm10k_config_dpdk_cfg;
+
+ fm10k_config_cfg_describe(sw, info);
+
+ return 0;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_config.h b/drivers/net/fm10k/switch/fm10k_config.h
new file mode 100644
index 0000000..f79df01
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.h
@@ -0,0 +1,171 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_CONFIG_H_
+#define _FM10K_CONFIG_H_
+
+
+#include <stdint.h>
+#include "fm10k_switch.h"
+
+/* General configuration */
+#define FM10K_CONFIG_TYPE_NULL 0
+#define FM10K_CONFIG_BIND_PF_NUMBER 1
+#define FM10K_CONFIG_EXT_PORT_SPEED 2
+
+/* Internal redirect configuration */
+#define FM10K_CONFIG_DPDK_PORT_MAP_PF 10
+#define FM10K_CONFIG_EXT_PORT_MAP_PF 11
+
+/* Debug configuration */
+#define FM10K_CONFIG_DEBUG_START 50
+#define FM10K_CONFIG_DEBUG_ENABLE (FM10K_CONFIG_DEBUG_START + 0)
+#define FM10K_CONFIG_DEBUG_CONFIG (FM10K_CONFIG_DEBUG_START + 1)
+#define FM10K_CONFIG_DEBUG_FFU_INIT (FM10K_CONFIG_DEBUG_START + 2)
+#define FM10K_CONFIG_DEBUG_FFU_REG (FM10K_CONFIG_DEBUG_START + 3)
+#define FM10K_CONFIG_DEBUG_FFU_RULE (FM10K_CONFIG_DEBUG_START + 4)
+#define FM10K_CONFIG_DEBUG_STATS_PORT (FM10K_CONFIG_DEBUG_START + 5)
+#define FM10K_CONFIG_DEBUG_STATS_QUEUE (FM10K_CONFIG_DEBUG_START + 6)
+#define FM10K_CONFIG_DEBUG_STATS_FFU (FM10K_CONFIG_DEBUG_START + 7)
+#define FM10K_CONFIG_DEBUG_STATS_MORE (FM10K_CONFIG_DEBUG_START + 8)
+#define FM10K_CONFIG_DEBUG_STATS_INTERVAL (FM10K_CONFIG_DEBUG_START + 9)
+
+/* external redirect configuration */
+#define FM10K_CONFIG_FLOWSET_START 80
+#define FM10K_CONFIG_FLOWSET_STOP 81
+#define FM10K_CONFIG_FLOWSET_ENABLE 82
+
+#define FM10K_CONFIG_FLOW_COND_START 100
+#define FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT (FM10K_CONFIG_FLOW_COND_START + 0)
+#define FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT (FM10K_CONFIG_FLOW_COND_START + 1)
+#define FM10K_CONFIG_FLOW_COND_VLAN (FM10K_CONFIG_FLOW_COND_START + 2)
+#define FM10K_CONFIG_FLOW_COND_END (FM10K_CONFIG_FLOW_COND_START + 2)
+
+#define FM10K_CONFIG_FLOW_ACT_START 200
+#define FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT (FM10K_CONFIG_FLOW_ACT_START + 0)
+#define FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT (FM10K_CONFIG_FLOW_ACT_START + 1)
+#define FM10K_CONFIG_FLOW_ACT_FW_VALN (FM10K_CONFIG_FLOW_ACT_START + 2)
+#define FM10K_CONFIG_FLOW_ACT_END (FM10K_CONFIG_FLOW_ACT_START + 2)
+
+#define FM10K_CONFIG_FLOW_NONE_PORT 0
+#define FM10K_CONFIG_FLOW_EXT_PORT 1
+#define FM10K_CONFIG_FLOW_DPDK_PORT 2
+
+#define FM10K_CONFIG_VALUE_NULL 0
+#define FM10K_CONFIG_VALUE_INT 1
+#define FM10K_CONFIG_VALUE_STR 2
+
+/* SWITCH config */
+#define FM10K_CONFIG_PORT_MAP_NULL 0
+#define FM10K_CONFIG_PORT_MAP_PF 1
+#define FM10K_CONFIG_PORT_MAP_PFS 2
+#define FM10K_CONFIG_PORT_MAP_PFSS 3
+
+/* DPDK port */
+#define FM10K_CONFIG_DPDK_NULL 0
+#define FM10K_CONFIG_DPDK_PF 1
+#define FM10K_CONFIG_DPDK_VF 2
+#define FM10K_CONFIG_DPDK_MAX 3
+
+struct fm10k_cfg_port_pf_map {
+ uint8_t type;
+ uint8_t map_no[2];
+};
+
+/* Configuration read from file */
+struct fm10k_cfg_config_item {
+ uint8_t type;
+ uint8_t val_type;
+ uint16_t key_param;
+ const char *describe;
+ union {
+ int64_t int64;
+ char *str;
+ } val;
+};
+
+struct fm10k_hw;
+
+struct fm10k_dpdk_port {
+ uint8_t type;
+ struct fm10k_hw *hw;
+ void *rte_dev;
+ void *flow_list;
+ uint16_t pf_no;
+ uint8_t tx_queue_num;
+ uint8_t rx_queue_num;
+};
+
+
+/* Flow configuration */
+struct fm10k_cfg_port {
+ uint8_t port_type;
+ uint8_t port_no;
+ uint16_t vlan_id;
+};
+
+struct fm10k_cfg_flow {
+ /* set by configuration */
+ struct fm10k_cfg_flow *prev;
+ struct fm10k_cfg_flow *next;
+ uint8_t flow_no; /* only configured flow has this NO. */
+ struct fm10k_cfg_port src_port;
+ struct fm10k_cfg_port fw_port[2];
+ /* set by ffu rule add */
+ uint16_t rule_id;
+};
+
+#define FM10K_CONFIG_FLOWSET_NAME_MAX 256
+struct fm10k_cfg_flowset {
+ char name[FM10K_CONFIG_FLOWSET_NAME_MAX];
+ struct fm10k_cfg_flow flow_head;
+ struct fm10k_cfg_flowset *next;
+};
+
+/* Configuration */
+struct fm10k_dpdk_cfg {
+ uint8_t pf_num; /* configure by conf */
+ uint8_t pf_bind; /* initialize by dpdk */
+ uint8_t pf_max; /* set by card type */
+ uint8_t ext_port_num; /* configure by conf */
+ uint8_t ext_port_speed; /* configure by conf */
+ uint32_t debug_cfg; /* configure by conf */
+ uint32_t stats_interval;/* configure by conf */
+
+ struct fm10k_hw *master_hw; /* initialize by dpdk */
+ struct fm10k_hw *pf_hw[FM10K_SW_PEP_PORTS_MAX]; /* initialize by dpdk */
+
+ /* initialize by dpdk */
+ struct fm10k_dpdk_port ports[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf or default*/
+ struct fm10k_cfg_port_pf_map dpdk_port_map
+ [FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf */
+ struct fm10k_cfg_port_pf_map ext_port_map[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_cfg_config_item *config_list; /* configure by conf */
+ struct fm10k_cfg_flowset flowset_head; /* transfer from conf file */
+ struct fm10k_cfg_flowset *current;
+};
+
+static inline bool
+fm10k_config_check_debug(struct fm10k_dpdk_cfg *cfg, uint16_t dbg)
+{
+ return cfg->debug_cfg & 1 << (dbg - FM10K_CONFIG_DEBUG_START) &&
+ cfg->debug_cfg & 1;
+}
+
+struct fm10k_cfg_flowset *fm10k_config_flowset_get(const char *name);
+struct fm10k_cfg_flowset *fm10k_config_flowset_current_get(void);
+void fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new);
+void fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow);
+void fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow);
+bool fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow);
+
+void fm10k_config_cfg_flowset_show(void);
+struct fm10k_hw *fm10k_config_hw_get(int port_no);
+int fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw);
+
+#endif /* _FM10K_CONFIG_H */
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.c b/drivers/net/fm10k/switch/fm10k_ffu.c
new file mode 100644
index 0000000..c2e8491
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.c
@@ -0,0 +1,1209 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+
+/*
+ * one SLICE for 40bits SEL
+ * SLICE 28 FOR SEL SGLORT(16bits) and VLAN(16bits)
+ * FOR ACT ROUTE
+ * SLICE 29 FOR SEL ETHER_TYPE(16bits)
+ * FOR ACT MIRROR | SET_VLAN
+ * SLICE 30 FOR SEL MPLS_LABEL0(32bits)
+ * SLICE 31 FOR SEL MPLS_LABEL1(32bits)
+ */
+#define FM10K_FFU_SLICE_START 28
+#define FM10K_FFU_SLICE_SGLORT_VID 28
+#define FM10K_FFU_SLICE_ETYPE_VID2 29
+#define FM10K_FFU_SLICE_MPLS_LABEL0 30
+#define FM10K_FFU_SLICE_MPLS_LABEL1 31
+
+#define FM10K_FFU_SLICE_SGLORT_VLAN_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_FFU_SLICE_CFG_START_ACTION | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_ETHER_TYPE_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL0_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL1_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+static uint64_t fm10k_ffu_slice_cfgs[4] = {
+ FM10K_FFU_SLICE_SGLORT_VLAN_CFG,
+ FM10K_FFU_SLICE_ETHER_TYPE_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL0_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL1_CFG
+};
+
+#define FM10K_FFU_INIT_PRINT(cfg, ...) do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_INIT)) \
+ printf(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_RULE_PRINT(cfg, ...) do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_RULE)) \
+ printf(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_CNT_INDEX(x) (FM10K_SW_FFU_CNT_START + (x))
+
+#define FM10K_FFU_FLOW_START 10
+#define FM10K_FFU_MIRROR_PROFILE 1
+
+#define FM10K_FFU_SGLORT_TYPE_NULL 0
+#define FM10K_FFU_SGLORT_TYPE_EPL 1
+#define FM10K_FFU_SGLORT_TYPE_PF 2
+#define FM10K_FFU_SGLORT_TYPE_PFS 3
+#define FM10K_FFU_SGLORT_TYPE_DPDK 4
+
+struct fm10k_ffu_rule_data {
+ uint16_t sglort_type;
+ uint16_t cond_sglort;
+ uint16_t cond_vlan;
+ uint16_t act_dglort;
+ uint16_t act_vlan;
+ uint16_t bypass_dglort;
+ uint16_t bypass_vlan;
+};
+
+
+static uint8_t fm10k_ffu_bitmap[FM10K_FFU_RULE_MAX / 8];
+static uint8_t
+fm10k_ffu_rule_get_bit(int id)
+{
+ int num = id / 8;
+ int offset = id % 8;
+
+ return (fm10k_ffu_bitmap[num] >> offset) & 0x1;
+}
+
+static void
+fm10k_ffu_rule_set_bit(int id, uint8_t bit)
+{
+ int num = id / 8;
+ int offset = id % 8;
+ uint8_t tmp = fm10k_ffu_bitmap[num];
+ uint8_t data = 1;
+
+ if (bit == 0)
+ fm10k_ffu_bitmap[num] = tmp & ~(data << offset);
+ else
+ fm10k_ffu_bitmap[num] |= (data << offset);
+}
+
+static int
+fm10k_ffu_rule_alloc(void)
+{
+ int i;
+
+ for (i = FM10K_FFU_FLOW_START; i < FM10K_FFU_RULE_MAX; i++) {
+ if (fm10k_ffu_rule_get_bit(i) != 0)
+ continue;
+ fm10k_ffu_rule_set_bit(i, 1);
+ return i;
+ }
+ return -1;
+}
+
+static void
+fm10k_ffu_rule_free(int id)
+{
+ fm10k_ffu_rule_set_bit(id, 0);
+}
+
+static void
+fm10k_ffu_always_mismatch(struct fm10k_switch *sw,
+ unsigned int slice, unsigned int idx)
+{
+ uint64_t temp64;
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~0ULL) |
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY_TOP, ~0ULL);
+ fm10k_write_switch_reg128(sw, FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64, temp64);
+}
+
+static void
+fm10k_ffu_route_dglort(struct fm10k_switch *sw, unsigned int slice,
+ unsigned int idx, uint16_t sglort, uint16_t dglort)
+{
+ uint64_t temp64, temp64_2;
+
+ /*
+ * Set the key to exact match on the 16 SGLORT bits and
+ * always match everywhere else.
+ */
+ temp64 = FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, sglort);
+ temp64_2 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~sglort & 0xffff);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64_2, temp64);
+
+ /*
+ * Set the corresponding SRAM entry to ROUTE_GLORT to the
+ * corresponding DGLORT.
+ */
+ temp64 = 0x40;
+ temp64 = temp64 << 32 |
+ FM10K_SW_MAKE_REG_FIELD(FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ /*
+ * 11.5.4.2 FFU_SLICE_SRAM[0..31][0..1023]
+ */
+ temp64_2 =
+ FM10K_SW_FFU_CNT_BANK << (35 - 23) |
+ (FM10K_SW_FFU_CNT_START + idx);
+ temp64 |= temp64_2 << 23;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(slice, idx), temp64);
+}
+
+
+static void
+fm10k_ffu_set_dest_glort_mask(struct fm10k_switch *sw,
+ unsigned int idx, uint16_t dglort, uint64_t dport_mask)
+{
+ uint64_t temp64;
+ unsigned int multiple_dports;
+ unsigned int num_dports;
+ unsigned int amplification_factor;
+ unsigned int hashed_entries;
+ unsigned int i, j;
+
+ multiple_dports = (dport_mask & (dport_mask - 1));
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to port ", __func__, dglort);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ if (multiple_dports) {
+ /*
+ * Create a pair of entries and use the hash value to select
+ * among them.
+ */
+ num_dports = 0;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i)) {
+ num_dports++;
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d", i);
+ }
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, "\n");
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ amplification_factor = 16 / num_dports;
+ hashed_entries = num_dports * amplification_factor;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_HASHED);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_COUNT,
+ hashed_entries);
+ fm10k_write_switch_reg64(sw, FM10K_SW_GLORT_RAM(idx), temp64);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ for (j = 0; j < amplification_factor; j++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ (1ULL << i)));
+ sw->glort_dest_table_idx++;
+ }
+ } else {
+ /*
+ * Set the corresponding entry in the DGLORT map RAM to use
+ * strict indexing straight into the DEST_TABLE, then write
+ * the corresponding destination port in the DEST_TABLE.
+ */
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ dport_mask));
+ sw->glort_dest_table_idx++;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d\n", i);
+ }
+}
+
+
+static uint32_t
+fm10k_ffu_set_dest_glort_multi_cast(struct fm10k_switch *sw,
+ uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2)
+{
+ uint32_t dglort;
+ bool is_new;
+ uint32_t idx;
+ uint64_t temp64;
+
+ dglort =
+ fm10k_switch_multi_glort_get(lport1, lport2,
+ vlan1, vlan2, &is_new);
+
+ if (!is_new)
+ return dglort;
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to (%d:%d) (%d:%d)\n",
+ __func__, dglort, lport1, vlan1, lport2, vlan2);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ idx = sw->glort_cam_ram_idx++;
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ /* GLORT_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * DestMask 47:0 RW 0x0
+ * IP_MulticastIndex59:48 RW 0x0
+ * Reserved 63:60 RSV 0x0
+ */
+ temp64 = sw->mcast_dest_table_idx;
+ temp64 = temp64 << 48 | 1 << lport1 | 1 << lport2;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx), temp64);
+ sw->glort_dest_table_idx++;
+
+ /* MCAST_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * PortMask 47:0 RW 0x0
+ * LenTableIdx 61:48 RW 0x0
+ * Reserved 63:62 RSV 00b
+ */
+ temp64 = sw->mcast_len_table_idx;
+ temp64 = 1 << lport1 | 1 << lport2 | temp64 << 48;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE
+ (sw->mcast_dest_table_idx++), temp64);
+
+ /* MCAST_LEN_TABLE
+ * Field Name Bit(s) Type Default
+ * L3_McastIdx 14:0 RW 0x0
+ * L3_Repcnt 26:15 RW 0x0
+ * Reserved 31:27 RSV 0x0
+ */
+ temp64 =
+ sw->mcast_vlan_table_idx | 1 << 15;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE
+ (sw->mcast_len_table_idx++), temp64);
+
+ /* MCAST_VLAN_TABLE
+ * Field Name Bit(s) Type Default
+ * VID 11:0 RW 0x0
+ * DGLORT 27:12 RW 0x0
+ * ReplaceVID 28 RW 0b
+ * ReplaceDGLORT 29 RW 0b
+ * Reserved 31:30 RSV 00b
+ */
+ temp64 = vlan1 |
+ fm10k_switch_pf_glort_get
+ (lport1) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+ temp64 = vlan2 |
+ fm10k_switch_pf_glort_get
+ (lport2) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+
+ return dglort;
+}
+
+static uint64_t
+fm10k_data64_field64_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+
+ if (start == end) {
+ tmp64 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp64 = tmp64 >> (64 - end + start);
+ }
+ return tmp64;
+}
+
+static uint32_t
+fm10k_data64_field32_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+ uint32_t tmp32;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp32 = tmp64 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+static uint32_t
+fm10k_data32_field_get(uint32_t data, int start, int end)
+{
+ uint32_t tmp32 = data;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp32 = tmp32 << (64 - end);
+ tmp32 = tmp32 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+
+static void
+fm10k_glort_register_dump(struct fm10k_switch *sw)
+{
+ uint32_t i;
+ uint64_t data64;
+ uint32_t data32;
+ uint32_t tmp32;
+
+ if (!fm10k_config_check_debug
+ (sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ printf("----- GLORT -----\n");
+
+ for (i = 0; i < sw->glort_cam_ram_idx; i++) {
+ data32 = fm10k_read_switch_reg(sw, FM10K_SW_GLORT_CAM(i));
+ printf("[%02u]GLORT_CAM %#x Key %#x\n", i, data32,
+ fm10k_data32_field_get(data32, 0, 15));
+ data64 = fm10k_read_switch_reg64(sw, FM10K_SW_GLORT_RAM(i));
+ printf(" GLORT_RAM %#llx Strict %u DestIndex %u\n",
+ (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 0, 1),
+ fm10k_data64_field32_get(data64, 2, 13));
+ tmp32 = fm10k_data64_field32_get(data64, 2, 13);
+ data64 =
+ fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(tmp32));
+ printf(" GLORT_DEST_TABLE[%u] %#llx IP_MulticastIndex %u\n",
+ tmp32, (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 48, 59));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 59);
+ if (tmp32 == 0)
+ continue;
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE(tmp32));
+ printf(" SCHED_MCAST_DEST_TABLE[%u] %#llx PortMask %#llx"
+ " LenTableIdx %u\n", tmp32,
+ (unsigned long long)data64,
+ (unsigned long long)
+ fm10k_data64_field64_get(data64, 0, 47),
+ fm10k_data64_field32_get(data64, 48, 61));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 61);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE(tmp32));
+ printf(" SCHED_MCAST_LEN_TABLE[%u] %#x "
+ "L3_McastIdx %u L3_Repcnt %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 14),
+ fm10k_data32_field_get(data32, 15, 26));
+ tmp32 = fm10k_data32_field_get(data32, 0, 14);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32));
+ printf(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32 + 1));
+ printf(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32 + 1, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ }
+}
+
+static void
+fm10k_ffu_register_dump(struct fm10k_switch *sw)
+{
+ int i, j;
+ uint32_t data[8];
+ uint32_t ffu_valid;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ printf("--------------- FFU REGISTERS DUMP ------------------\n");
+
+ fm10k_read_switch_array(sw, FM10K_SW_FFU_MASTER_VALID, data, 2);
+ printf("FFU_MASTER_VALID: %#x %#x\n", data[0], data[1]);
+ ffu_valid = data[0];
+
+ for (i = 0; i < 32; i++) {
+ if ((ffu_valid & (1 << i)) == 0)
+ continue;
+
+ printf("------ SLICE%d ------\n", i);
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_VALID(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ printf("FFU_SLICE_VALID[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ printf("FFU_SLICE_CASCADE_ACTION[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ for (j = 0; j < 1; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CFG(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ printf("FFU_SLICE_CFG[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ for (j = 0; j < 32; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_TCAM(i, j), data, 4);
+ if ((data[0] != 0 || data[1] != 0 ||
+ data[2] != 0 || data[3] != 0))
+ printf("FFU_SLICE_TCAM[%d][%d]: %#x %#x %#x %#x\n",
+ i, j, data[0], data[1],
+ data[2], data[3]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ printf("FFU_SLICE_SRAM[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ }
+}
+
+
+static void
+fm10k_ffu_mirror_set_action(struct fm10k_switch *sw,
+ uint16_t ffu_slice, uint16_t table_idx)
+{
+ uint64_t data64;
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice, table_idx), 0xffffffffff, 0x1);
+ /* SET FFU RX_MIRROR */
+ data64 = 0x60; /* higher than route, not necessary */
+ data64 = data64 << 32 | 0x2 << 21 | 1 << (8 + 4) | 1 << 4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), data64);
+}
+
+static void
+fm10k_ffu_mirror_set_forward(struct fm10k_switch *sw,
+ int profile,
+ uint16_t vlan,
+ uint16_t dest_lport,
+ uint32_t dest_sglort)
+{
+ uint64_t data64;
+
+ /* RX_MIRROR_CFG */
+ fm10k_write_switch_reg(sw, 0xd50000 + 0xd, profile);
+
+ /* FH_MIRROR_PROFILE_TABLE */
+ fm10k_write_switch_reg(sw,
+ 0xd50000 + 0x1 * profile + 0x40, dest_lport);
+
+ /* MOD_MIRROR_PROFILE_TABLE don't work */
+ data64 = vlan;
+ data64 = (dest_sglort & 0xffff) | data64 << 17;
+ fm10k_write_switch_reg64(sw,
+ 0xe80000 + 0x2 * profile + 0x24000, data64);
+}
+
+
+int
+fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_ext_port,
+ uint16_t dest_ext_port,
+ uint16_t vlan)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+ uint16_t mirror_profile = FM10K_FFU_MIRROR_PROFILE;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_ffu_mirror_set_action(sw, ffu_slice, table_idx);
+
+ fm10k_ffu_mirror_set_forward(sw, mirror_profile,
+ vlan, sw->epl_map[dest_ext_port - 1].logical_port,
+ sw->epl_map[dest_ext_port - 1].glort);
+ return 0;
+}
+
+int
+fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_ext_port)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), 0);
+ return 0;
+}
+
+static void
+fm10k_ffu_logical_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t lport, uint16_t vlan_id)
+{
+ uint64_t data;
+
+ /* 11.21.3.9 INGRESS_VID_TABLE[0..4095] */
+ data = fm10k_read_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000);
+ data |= 1 << lport;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000, data);
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "%s lport:%d, vlan_id:%d, reg:%#llx\n",
+ __func__, lport, vlan_id, (unsigned long long)data);
+}
+
+static void
+fm10k_ffu_glort_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t glort, uint16_t vlan_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ if (sw->pep_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->pep_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->epl_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+}
+
+
+static uint16_t
+fm10k_ffu_dpdk_port_glort_get(struct fm10k_switch *sw, int port)
+{
+ int pf;
+ uint16_t glort = 0;
+
+ if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFS ||
+ sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ glort = fm10k_switch_pfs_glort_get
+ (sw->dpdk_cfg->dpdk_port_map[port].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[port].map_no[1]);
+ } else if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf = sw->dpdk_cfg->dpdk_port_map[port].map_no[0];
+ glort = fm10k_switch_pf_glort_get(pf);
+ }
+ return glort;
+}
+
+static void
+fm10k_ffu_rule_enable_single_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint16_t dglort,
+ uint16_t dvlan)
+{
+ uint64_t temp64;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (dvlan) {
+ /* Force updating VLAN tag if present.
+ * Add if absent. 11.5.3.4
+ */
+ temp64 = 3;
+ sram[sram_idx] =
+ temp64 << 16 | dvlan |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 2) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET);
+ fm10k_ffu_glort_port_vlan_set(sw, dglort, dvlan);
+ sram_idx++;
+ }
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+static void
+fm10k_ffu_rule_enable_multi_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint8_t lport1,
+ uint8_t lport2,
+ uint16_t vlan1,
+ uint16_t vlan2)
+{
+ uint64_t temp64;
+ uint32_t dglort;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ dglort =
+ fm10k_ffu_set_dest_glort_multi_cast(sw,
+ lport1, lport2, vlan1, vlan2);
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ fm10k_ffu_logical_port_vlan_set(sw, lport1, vlan1);
+ fm10k_ffu_logical_port_vlan_set(sw, lport2, vlan2);
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+
+void
+fm10k_ffu_flow_enable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ uint16_t sglort = 0;
+ uint16_t svlan = 0;
+ uint16_t dglort = 0;
+ uint16_t dvlan = 0;
+ uint16_t lport1 = 0, lport2 = 0;
+ uint16_t dvlan2 = 0;
+
+ switch (flow->src_port.port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ sglort = fm10k_switch_epl_glort_get(flow->src_port.port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ sglort =
+ fm10k_ffu_dpdk_port_glort_get(sw,
+ flow->src_port.port_no);
+ break;
+ }
+ switch (flow->fw_port[0].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[0].port_no - 1);
+ lport1 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[0].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[0].port_no);
+ lport1 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[0].port_no);
+ break;
+ }
+ svlan = flow->src_port.vlan_id;
+ dvlan = flow->fw_port[0].vlan_id;
+
+ flow->rule_id = fm10k_ffu_rule_alloc();
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Set rule cond sglort:%#x, vlan:%d ==> "
+ "act dglort:%#x, vlan:%d",
+ sglort, svlan, dglort, dvlan);
+
+ if (flow->fw_port[1].port_type) {
+ switch (flow->fw_port[1].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[1].port_no - 1);
+ lport2 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[1].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[1].port_no);
+ lport2 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[1].port_no);
+ break;
+ }
+ dvlan2 = flow->fw_port[1].vlan_id;
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ " (bypass glort:%#x, vlan:%d)\n",
+ dglort, dvlan2);
+
+ fm10k_ffu_rule_enable_multi_cast(sw,
+ flow->rule_id, sglort, svlan,
+ lport1, lport2, dvlan, dvlan2);
+ } else {
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg, "\n");
+ fm10k_ffu_rule_enable_single_cast(sw,
+ flow->rule_id, sglort, svlan, dglort, dvlan);
+ }
+}
+
+
+void
+fm10k_ffu_flow_disable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ int i;
+
+ if (flow->rule_id == 0)
+ return;
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Remove flow %d rule %d\n",
+ flow->flow_no, flow->rule_id);
+
+ for (i = FM10K_FFU_SLICE_START; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_ffu_always_mismatch(sw, i, flow->rule_id);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, flow->rule_id), 0);
+ }
+ fm10k_ffu_rule_free(flow->rule_id);
+}
+
+
+static int
+fm10k_ffu_configured_flowset_enable(struct fm10k_switch *sw)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flow *flow;
+
+ flowset = fm10k_config_flowset_current_get();
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+
+ return 0;
+}
+
+
+void
+fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flowset *cur_fs;
+ struct fm10k_cfg_flow *flow;
+
+ cur_fs = fm10k_config_flowset_current_get();
+ if (strcmp(cur_fs->name, new_name) == 0)
+ return;
+
+ flowset = fm10k_config_flowset_get(new_name);
+ if (flowset == NULL) {
+ FM10K_SW_ERR("Can not find flowset %s!!\n", new_name);
+ return;
+ }
+
+ /* disable current flowset */
+ flow = cur_fs->flow_head.next;
+ while (!fm10k_config_flow_list_end(&cur_fs->flow_head, flow)) {
+ fm10k_ffu_flow_disable(sw, flow);
+ flow = flow->next;
+ }
+
+ /* enable new flowset */
+ fm10k_config_flowset_current_set(flowset);
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+}
+
+
+int
+fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg)
+{
+ int ret = 0;
+ uint64_t data64;
+ uint16_t i, j;
+ uint32_t sglort = 0, dglort = 0;
+ uint16_t table_idx = 0;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_START;
+
+ sw->mcast_dest_table_idx = 1;
+ sw->mcast_len_table_idx = 1;
+ sw->mcast_vlan_table_idx = 1;
+
+ for (i = 0;
+ i < sizeof(fm10k_ffu_slice_cfgs) / sizeof(uint64_t);
+ i++) {
+ for (j = 0; j < FM10K_SW_FFU_NUM_SCENARIOS; j++) {
+ data64 = fm10k_ffu_slice_cfgs[i];
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CFG
+ (FM10K_FFU_SLICE_START + i, j),
+ data64);
+ }
+ FM10K_FFU_INIT_PRINT(cfg, "SET slice %d cfg = %#llx\n",
+ FM10K_FFU_SLICE_START + i,
+ (unsigned long long)data64);
+ }
+
+ for (table_idx = 0;
+ table_idx < FM10K_SW_FFU_SLICE_TCAM_ENTRIES / 2;
+ table_idx++)
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ fm10k_ffu_always_mismatch(sw, i, table_idx);
+
+ /*
+ * Create a TCAM entry to match each SGLORT that might be used, and
+ * set the corresponding SRAM action entries to ROUTE_GLORT to the
+ * corresponding DGLORT (see above table).
+ * SGLORT ROUTE is always the first slice
+ */
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pf_glort_get
+ (cfg->ext_port_map[i].map_no[0]);
+ } else if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ }
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, sglort, dglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ table_idx = FM10K_FFU_EXT_PORT_RULE_EGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, dglort, sglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ }
+
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i),
+ 0xf0f000f);
+ /* Set slice 0 to valid for all scenarios */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_VALID(i),
+ FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS);
+ }
+
+ /* Mark slice 0 as valid and all chunks as invalid */
+ data64 = 0;
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ data64 |= FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(i);
+ fm10k_write_switch_reg64(sw, FM10K_SW_FFU_MASTER_VALID, data64);
+
+ /*
+ * Set up the DGLORT map according to the desired
+ * SGLORT -> { DGLORT, logical_port } map.
+ */
+ for (i = 0; i < sw->info->num_peps; i++) {
+ if (sw->pep_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->pep_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->epl_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->epl_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ uint64_t dport_mask = 0;
+
+ if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ dport_mask =
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[0]].logical_port) |
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[1]].logical_port);
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++, dglort, dport_mask);
+ }
+ }
+
+ /* Ensure the rest of the DGLORT TCAM won't match */
+ for (table_idx = sw->glort_cam_ram_idx;
+ table_idx < FM10K_SW_GLORT_CAM_ENTRIES;
+ table_idx++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_GLORT_CAM(table_idx),
+ FM10K_SW_GLORT_CAM_MATCH_NONE);
+
+ ret = fm10k_ffu_configured_flowset_enable(sw);
+
+ fm10k_ffu_register_dump(sw);
+ fm10k_glort_register_dump(sw);
+ return ret;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.h b/drivers/net/fm10k/switch/fm10k_ffu.h
new file mode 100644
index 0000000..9d46007
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_FFU_H_
+#define _FM10K_FFU_H_
+
+
+#include <stdint.h>
+
+#define FM10K_FFU_EXT_PORT_RULE_INGRESS(i) ((i) * 2)
+#define FM10K_FFU_EXT_PORT_RULE_EGRESS(i) ((i) * 2 + 1)
+#define FM10K_FFU_RULE_MAX FM10K_SW_FFU_RULE_MAX
+
+struct fm10k_switch;
+struct fm10k_dpdk_cfg;
+struct fm10k_cfg_flow;
+
+int fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg);
+
+int fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_dpdk_port, u16 dest_dpdk_port, uint16_t vlan);
+int fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_dpdk_port);
+
+void fm10k_ffu_flow_enable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flow_disable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name);
+
+#endif /* _FM10K_FFU_H */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
new file mode 100644
index 0000000..4172b22
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -0,0 +1,1242 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_stats.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+
+
+#define FM10K_MAX_VLAN_COUNTER 63
+#define FM10K_NB_RX_STATS_BANKS 6
+#define FM10K_BINS_PER_RX_STATS_BANK 16
+#define FM10K_WORDS_PER_RX_STATS_COUNTER 4
+
+/* Max expected entries in read stats scatter gather list */
+#define MAX_STATS_SGLIST 128
+#define FM10K_RX_STATS_BASE (0xE00000)
+#define FM10K_MOD_BASE (0xE80000)
+#define FM10K_CM_APPLY_BASE (0xD40000)
+#define FM10K_EPL_BASE (0x0E0000)
+
+#define FM10K_RX_STATS_BANK(index1, index0, word) \
+ ((0x001000) * ((index1) - 0) + \
+ (0x000004) * ((index0) - 0) + \
+ (word) + (0x000000) + \
+ (FM10K_RX_STATS_BASE))
+#define FM10K_MOD_STATS_BANK_FRAME(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x025000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_MOD_STATS_BANK_BYTE(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x026000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_CM_APPLY_DROP_COUNT(index, word) \
+ ((0x000002) * ((index) - 0) + \
+ (word) + (0x000880) + \
+ (FM10K_CM_APPLY_BASE))
+#define FM10K_MAC_OVERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000021) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_JABBER_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000022) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000023) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_RUNT_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000024) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_OVERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000025) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000026) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_CODE_ERROR_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000027) + \
+ (FM10K_EPL_BASE))
+
+/* Rx Bank Definitions */
+#define FM10K_RX_STAT_BANK_TYPE 0
+#define FM10K_RX_STAT_BANK_SIZE 1
+#define FM10K_RX_STAT_BANK_PRI 2
+#define FM10K_RX_STAT_BANK_FWD_1 3
+#define FM10K_RX_STAT_BANK_FWD_2 4
+#define FM10K_RX_STAT_BANK_VLAN 5
+
+/* FM10K_RX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_RX_STAT_NONIP_L2UCAST 0
+#define FM10K_RX_STAT_NONIP_L2MCAST 1
+#define FM10K_RX_STAT_NONIP_L2BCAST 2
+#define FM10K_RX_STAT_IPV4_L2UCAST 3
+#define FM10K_RX_STAT_IPV4_L2MCAST 4
+#define FM10K_RX_STAT_IPV4_L2BCAST 5
+#define FM10K_RX_STAT_IPV6_L2UCAST 6
+#define FM10K_RX_STAT_IPV6_L2MCAST 7
+#define FM10K_RX_STAT_IPV6_L2BCAST 8
+#define FM10K_RX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_RX_STAT_CLASS_BASED_PAUSE 10
+#define FM10K_RX_STAT_FRAMING_ERR 11
+#define FM10K_RX_STAT_FCS_ERR 12
+
+/* FM10K_RX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_RX_STAT_LEN_LT_64 0
+#define FM10K_RX_STAT_LEN_EQ_64 1
+#define FM10K_RX_STAT_LEN_65_127 2
+#define FM10K_RX_STAT_LEN_128_255 3
+#define FM10K_RX_STAT_LEN_256_511 4
+#define FM10K_RX_STAT_LEN_512_1023 5
+#define FM10K_RX_STAT_LEN_1024_1522 6
+#define FM10K_RX_STAT_LEN_1523_2047 7
+#define FM10K_RX_STAT_LEN_2048_4095 8
+#define FM10K_RX_STAT_LEN_4096_8191 9
+#define FM10K_RX_STAT_LEN_8192_10239 10
+#define FM10K_RX_STAT_LEN_GE_10240 11
+
+/* FM10K_RX_STAT_BANK_PRI Bin Definitions */
+#define FM10K_RX_STAT_PRI_0 0
+#define FM10K_RX_STAT_PRI_1 1
+#define FM10K_RX_STAT_PRI_2 2
+#define FM10K_RX_STAT_PRI_3 3
+#define FM10K_RX_STAT_PRI_4 4
+#define FM10K_RX_STAT_PRI_5 5
+#define FM10K_RX_STAT_PRI_6 6
+#define FM10K_RX_STAT_PRI_7 7
+#define FM10K_RX_STAT_PRI_8 8
+#define FM10K_RX_STAT_PRI_9 9
+#define FM10K_RX_STAT_PRI_10 10
+#define FM10K_RX_STAT_PRI_11 11
+#define FM10K_RX_STAT_PRI_12 12
+#define FM10K_RX_STAT_PRI_13 13
+#define FM10K_RX_STAT_PRI_14 14
+#define FM10K_RX_STAT_PRI_15 15
+
+/* FM10K_RX_STAT_BANK_FWD_1 Bin Definitions */
+#define FM10K_RX_STAT_FID_FORWARDED 0
+#define FM10K_RX_STAT_FLOOD_FORWARDED 1
+#define FM10K_RX_STAT_SPECIALLY_HANDLED 2
+#define FM10K_RX_STAT_PARSER_ERROR_DROP 3
+#define FM10K_RX_STAT_ECC_ERROR_DROP 4
+#define FM10K_RX_STAT_TRAPPED 5
+#define FM10K_RX_STAT_PAUSE_DROPS 6
+#define FM10K_RX_STAT_STP_DROPS 7
+#define FM10K_RX_STAT_SECURITY_VIOLATIONS 8
+#define FM10K_RX_STAT_VLAN_TAG_DROPS 9
+#define FM10K_RX_STAT_VLAN_INGRESS_DROPS 10
+#define FM10K_RX_STAT_VLAN_EGRESS_DROPS 11
+#define FM10K_RX_STAT_GLORT_MISS_DROPS 12
+#define FM10K_RX_STAT_FFU_DROPS 13
+#define FM10K_RX_STAT_TRIGGER_DROPS 14
+#define FM10K_RX_STAT_OVERFLOW4 15
+
+/* FM10K_RX_STAT_BANK_FWD_2 Bin Definitions */
+#define FM10K_RX_STAT_POLICER_DROPS 0
+#define FM10K_RX_STAT_TTL_DROPS 1
+#define FM10K_RX_STAT_CM_PRIV_DROPS 2
+#define FM10K_RX_STAT_CM_SMP0_DROPS 3
+#define FM10K_RX_STAT_CM_SMP1_DROPS 4
+#define FM10K_RX_STAT_CM_RX_HOG_0_DROPS 5
+#define FM10K_RX_STAT_CM_RX_HOG_1_DROPS 6
+#define FM10K_RX_STAT_CM_TX_HOG_0_DROPS 7
+#define FM10K_RX_STAT_CM_TX_HOG_1_DROPS 8
+/* Bin 9 reserved */
+#define FM10K_RX_STAT_TRIGGER_REDIRECTS 10
+#define FM10K_RX_STAT_FLOOD_CONTROL_DROPS 11
+#define FM10K_RX_STAT_GLORT_FORWARDED 12
+#define FM10K_RX_STAT_LOOPBACK_SUPPRESS 13
+#define FM10K_RX_STAT_OTHERS 14
+#define FM10K_RX_STAT_OVERFLOW5 15
+
+/* FM10K_RX_STAT_BANK_VLAN Bin definitions */
+#define FM10K_RX_STAT_VLAN_UCAST 0
+#define FM10K_RX_STAT_VLAN_MCAST 1
+#define FM10K_RX_STAT_VLAN_BCAST 2
+
+/* Tx Bank Definitions */
+#define FM10K_TX_STAT_BANK_TYPE 0
+#define FM10K_TX_STAT_BANK_SIZE 1
+
+/* FM10K_TX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_TX_STAT_L2UCAST 0
+#define FM10K_TX_STAT_L2MCAST 1
+#define FM10K_TX_STAT_L2BCAST 2
+#define FM10K_TX_STAT_ERR_SENT 3
+#define FM10K_TX_STAT_TIMEOUT_DROP 4
+#define FM10K_TX_STAT_ERR_DROP 5
+#define FM10K_TX_STAT_ECC_DROP 6
+#define FM10K_TX_STAT_LOOPBACK_DROP 7
+#define FM10K_TX_STAT_TTL1_DROP 8
+#define FM10K_TX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_TX_STAT_CLASS_BASED_PAUSE 10
+
+/* FM10K_TX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_TX_STAT_LEN_LT_64 0
+#define FM10K_TX_STAT_LEN_EQ_64 1
+#define FM10K_TX_STAT_LEN_65_127 2
+#define FM10K_TX_STAT_LEN_128_255 3
+#define FM10K_TX_STAT_LEN_256_511 4
+#define FM10K_TX_STAT_LEN_512_1023 5
+#define FM10K_TX_STAT_LEN_1024_1522 6
+#define FM10K_TX_STAT_LEN_1523_2047 7
+#define FM10K_TX_STAT_LEN_2048_4095 8
+#define FM10K_TX_STAT_LEN_4096_8191 9
+#define FM10K_TX_STAT_LEN_8192_10239 10
+#define FM10K_TX_STAT_LEN_GE_10240 11
+
+
+/* This structure is used to build a table of all rx / tx counters */
+struct fm10k_port_count_mapping {
+ /* Bank in which the counter is located */
+ uint32_t bank;
+
+ /* Bin in which the counter is located */
+ uint32_t bin;
+
+ /*
+ * Offset (in bytes) of a the frame counter in
+ * the fm10k_counters structure
+ */
+ uint32_t frame_offset;
+
+ /*
+ * Offset (in bytes) of a the byte counter in
+ * the fm10k_counters structure
+ */
+ uint32_t byte_offset;
+};
+
+
+/*
+ * Table of all RX port counters in the RX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping rx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FRAMING_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FCS_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors),
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors_octets)},
+
+ /* Size Bank */
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_64_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_octets)},
+
+ /* priority Bank */
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_0,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[0]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[0])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_1,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[1]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[1])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_2,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[2]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[2])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_3,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[3]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[3])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_4,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[4]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[4])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_5,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[5]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[5])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_6,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[6]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[6])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_7,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[7]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[7])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_8,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[8]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[8])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_9,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[9]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[9])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_10,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[10]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[10])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_11,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[11]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[11])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_12,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[12]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[12])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_13,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[13]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[13])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_14,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[14]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[14])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_15,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[15]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[15])},
+
+ /* Forwarding Bank */
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FID_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FLOOD_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SPECIALLY_HANDLED,
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_pkts),
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PARSER_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_ECC_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parity_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parity_error_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRAPPED,
+ offsetof(struct fm10k_port_counters, cnt_trapped_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trapped_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PAUSE_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_STP_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SECURITY_VIOLATIONS,
+ offsetof(struct fm10k_port_counters, cnt_security_violation_pkts),
+ offsetof(struct fm10k_port_counters, cnt_security_violation_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_TAG_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_INGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_EGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_GLORT_MISS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FFU_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRIGGER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_POLICER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TTL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_PRIV_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TRIGGER_REDIRECTS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_FLOOD_CONTROL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_GLORT_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_LOOPBACK_SUPPRESS,
+ offsetof(struct fm10k_port_counters, cnt_loopback_drops_pkts),
+ offsetof(struct fm10k_port_counters, cnt_loopback_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_OTHERS,
+ offsetof(struct fm10k_port_counters, cnt_other_pkts),
+ offsetof(struct fm10k_port_counters, cnt_other_octets)},
+
+}; /* end rxPortCntMapTable */
+
+
+/*
+ * Table of all TX port counters in the TX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping tx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_SENT,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TIMEOUT_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ECC_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_LOOPBACK_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TTL1_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_octets)},
+
+ /* Size Bank */
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_64_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_octets)},
+
+}; /* end tx_port_cnt_map_table */
+
+
+/*
+ * Get a 32bit EPL counter by adding it to the scatter gather
+ * list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+static inline void
+fm10k_get_epl_port_stat32(struct fm10k_scatter_gather_entry *sg_list,
+ struct fm10k_port_counters *counters,
+ int *sg_list_cnt, int epl, int lane)
+{
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_oversized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_JABBER_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_jabber_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_UNDERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_undersized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_RUNT_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_fragment_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERRUN_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_over_run_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_CODE_ERROR_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_code_errors;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+}
+
+
+static void
+fm10k_read_scatter_gather(struct fm10k_switch *sw,
+ int n_entries,
+ struct fm10k_scatter_gather_entry *sg_list)
+{
+ int i;
+
+ for (i = 0; i < n_entries; i++) {
+ uint32_t addr = sg_list[i].addr;
+ uint32_t count = sg_list[i].count;
+ uint32_t *data = sg_list[i].data;
+
+ fm10k_read_switch_array(sw, addr, data, count);
+ }
+}
+
+
+static int
+fm10k_get_port_counters(struct fm10k_switch *sw,
+ struct fm10k_ext_port *ext_port,
+ struct fm10k_port_counters *counters)
+{
+ int phys_port;
+ int epl;
+ int lane;
+ struct timeval ts;
+ bool valid_ip_stats = true;
+ uint32_t i;
+ struct fm10k_scatter_gather_entry sg_list[MAX_STATS_SGLIST];
+ int sg_list_cnt = 0;
+ /* Temporary bin array to retrieve 128bit
+ * port counters (frame + bytes).
+ */
+ uint32_t cnt_rx_port_stats_bank
+ [FM10K_NB_RX_STATS_BANKS]
+ [FM10K_BINS_PER_RX_STATS_BANK]
+ [FM10K_WORDS_PER_RX_STATS_COUNTER];
+ /*
+ * Fill the counters structure with 0 to assure
+ * all fields not explicitly set below, which will
+ * be the fields not supported by the FM10000,
+ * will be 0 on return.
+ */
+ memset(counters, 0, sizeof(*counters));
+
+ phys_port = ext_port->portno;
+ epl = ext_port->eplno;
+ lane = ext_port->first_lane;
+
+ counters->cnt_version = FM10K_STATS_VERSION;
+
+ /*
+ * Reading counters for each RX bank
+ *
+ * Because the RX_STATS_BANK register contains
+ * both, frame count and byte count in a 128bit
+ * register, we first store the data in a temporary
+ * array.
+ * 1. Fill Scatter Gather to retrieve each bin
+ * counter from the rx bank and store in a temp
+ * array.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 128bit read of an rx counter to the scatter gather
+ * list.
+ * NOTE: Read values will be stored in the temporary
+ * array switchExt->cnt_rx_port_stats_bank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [out] - Array to store the
+ * 128bit values sgList
+ * [out] - scatter gather array to hold the reads sgListCnt
+ * [out] - number of entry in the sgList
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_RX_STATS_BANK(bank, (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data = cnt_rx_port_stats_bank[bank][bin];
+ sg_list[sg_list_cnt].count = 4;
+ sg_list_cnt++;
+}
+
+ /*
+ * 2. Fill in the scatter gather for tx bank
+ * counters. They will directly be stored
+ * in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 64bit read of a tx counter to the scatter gather list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+ uint32_t bank = tx_port_cnt_map_table[i].bank;
+ uint32_t bin = tx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = tx_port_cnt_map_table[i].frame_offset;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_MOD_STATS_BANK_FRAME(bank,
+ (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)(((uint8_t *)counters) + (frame_offset));
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ }
+
+ /*
+ * 3. Fill in the scatter gather for TX CM DROP
+ * and for EPL counters. They will directly be
+ * stored in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ sg_list[sg_list_cnt].addr =
+ FM10K_CM_APPLY_DROP_COUNT(phys_port, 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_cm_drop_pkts;
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ /* EPL Counters */
+ fm10k_get_epl_port_stat32(sg_list, counters, &sg_list_cnt, epl, lane);
+
+ /*
+ * 4. Execute scatter gather read.
+ */
+ if (sg_list_cnt >= MAX_STATS_SGLIST) {
+ /*
+ * Pretty static. Mainly to warn if something new added,
+ * but the array size is not adjust accordingly
+ */
+ FM10K_SW_ERR("Scatter list array %d for port %d overflow.\n",
+ sg_list_cnt, phys_port);
+ return -1;
+ }
+
+ /*
+ * Taking lock to protect temporary structures used to
+ * store 128b counters
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ /* now get the stats in one shot, optimized for fibm */
+ fm10k_read_scatter_gather(sw, sg_list_cnt, sg_list);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ counters->timestamp = ts.tv_sec * 1000000 + ts.tv_usec;
+
+ /*
+ * 5. Retrieve the frame/byte counts from 128bit
+ * registers stored in temporary array and set
+ * the proper fm_portCounter structure members.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Update the counter variable related to a 128bit HW counter
+ * NOTE: Read values will be retrieved from the temporary
+ * array switchExt->cnt_rx_PortStatsBank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [in] - Array to read the
+ * 128bit values from
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = rx_port_cnt_map_table[i].frame_offset;
+ uint32_t byte_offset = rx_port_cnt_map_table[i].byte_offset;
+
+ *((uint64_t *)(((uint8_t *)counters) + frame_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][1]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][0])));
+
+ *((uint64_t *)(((uint8_t *)counters) + byte_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][3]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][2])));
+ }
+
+ /*
+ * 6. Set some counters that are not available
+ * in HW but can be computed from two or more
+ * HW counters.
+ */
+ /*
+ * If IP parsing is enabled then the IP stats will be read and will be
+ * valid. Else all traffic, whether IP or not, is counted as _nonip.
+ */
+ if (valid_ip_stats) {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip +
+ counters->cnt_rx_ucst_pkts_ipv4 +
+ counters->cnt_rx_ucst_pkts_ipv6;
+
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip +
+ counters->cnt_rx_mcst_pkts_ipv4 +
+ counters->cnt_rx_mcst_pkts_ipv6;
+
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip +
+ counters->cnt_rx_bcst_pkts_ipv4 +
+ counters->cnt_rx_bcst_pkts_ipv6;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_octets_ipv4 =
+ counters->cnt_rx_ucst_octets_ipv4 +
+ counters->cnt_rx_mcst_octets_ipv4 +
+ counters->cnt_rx_bcst_octets_ipv4;
+
+ counters->cnt_rx_octets_ipv6 =
+ counters->cnt_rx_ucst_octets_ipv6 +
+ counters->cnt_rx_mcst_octets_ipv6 +
+ counters->cnt_rx_bcst_octets_ipv6;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip +
+ counters->cnt_rx_octets_ipv4 +
+ counters->cnt_rx_octets_ipv6;
+ } else {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip;
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip;
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip;
+ }
+
+ counters->cnt_trigger_drop_redir_pkts =
+ counters->cnt_trigger_redir_pkts +
+ counters->cnt_trigger_drop_pkts;
+
+ /* Emulate Tx _octets counter using the sum of all size bins. */
+ counters->cnt_tx_octets += counters->cnt_tx_minto63_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_64_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_65to127_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_128to255_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_256to511_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_512to1023_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1024to1522_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1523to2047_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_2048to4095_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_4096to8191_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_8192to10239_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_10240tomax_octets;
+
+ return 0;
+} /* end fm10k_get_port_counters */
+
+
+struct fm10k_stats_data {
+ uint64_t rx_pkts;
+ uint64_t tx_pkts;
+ uint64_t rx_drop;
+};
+
+
+static uint64_t
+fm10k_stats_ffu_count_get(struct fm10k_switch *sw, int table_id)
+{
+ uint64_t data;
+
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x4000 * FM10K_SW_FFU_CNT_BANK +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x8000);
+ } else {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x800 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x10000);
+ }
+ return data;
+}
+
+static void
+fm10k_stats_epl_ffu_print(struct fm10k_switch *sw, int epl)
+{
+ int table_id;
+ uint64_t data;
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_INGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ printf(" FFU[%d] ingress %-12llu\n",
+ table_id, (unsigned long long)(data));
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_EGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ printf(" FFU[%d] egress %-12llu\n",
+ table_id, (unsigned long long)(data));
+}
+
+#define FM10K_STATS_RULE_NUM_MAX 100
+static uint16_t fm10k_stats_rule_list[FM10K_STATS_RULE_NUM_MAX];
+void
+fm10k_stats_rule_count_reg(uint16_t rule_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0) {
+ fm10k_stats_rule_list[i] = rule_id;
+ break;
+ }
+ }
+}
+
+void
+fm10k_stats_ffu_count_print(struct fm10k_switch *sw)
+{
+ int i, rule_id;
+ uint64_t data;
+ static uint64_t ffu_count[FM10K_STATS_RULE_NUM_MAX];
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0)
+ continue;
+
+ rule_id = fm10k_stats_rule_list[i];
+ data = fm10k_stats_ffu_count_get(sw, rule_id);
+ printf("FFU[%d] count %-12llu\n",
+ rule_id,
+ (unsigned long long)
+ (data - ffu_count[rule_id]));
+ ffu_count[rule_id] = data;
+ }
+}
+
+void
+fm10k_stats_epl_port_print(struct fm10k_switch *sw)
+{
+ int i, lport;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_stats_data data;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ lport = sw->epl_map[i].logical_port;
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ data.rx_pkts =
+ counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts =
+ counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ printf("EPL port %-2d lport %-5d tx_pkt %-12llu rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, lport,
+ (unsigned long long)
+ (data.tx_pkts - last_data[i].tx_pkts),
+ (unsigned long long)
+ (data.rx_pkts - last_data[i].rx_pkts),
+ (unsigned long long)
+ (data.rx_drop - last_data[i].rx_drop));
+ last_data[i] = data;
+
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU))
+ fm10k_stats_epl_ffu_print(sw, i);
+ }
+}
+
+
+void
+fm10k_stats_dpdk_port_print(struct fm10k_switch *sw)
+{
+ int i, j, lport, pf_no;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_PEPS_MAX];
+ static struct fm10k_stats_data last_queue_data[FM10K_SW_PEPS_MAX][4];
+ struct fm10k_stats_data data, mydata;
+ char pf_ports[10];
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].hw == NULL)
+ continue;
+ if (sw->dpdk_cfg->ports[i].type != FM10K_CONFIG_DPDK_PF)
+ continue;
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ memset(&mydata, 0, sizeof(mydata));
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ for (j = 0; j < 2; j++) {
+ pf_no =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[j];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters
+ (sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ }
+ } else if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ } else {
+ continue;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ sprintf(pf_ports, "%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0]);
+ else
+ sprintf(pf_ports, "%d/%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1]);
+
+ printf("DPDK port %-2d pf %-5s tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, pf_ports,
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_QUEUE))
+ continue;
+ memset(&mydata, 0, sizeof(mydata));
+ for (j = 0;
+ j < sw->dpdk_cfg->ports[i].tx_queue_num;
+ j++) {
+ struct fm10k_hw *hw = sw->dpdk_cfg->ports[i].hw;
+ uint16_t queue_id = j, pf_no;
+
+ fm10k_switch_dpdk_hw_queue_map(hw, queue_id,
+ sw->dpdk_cfg->ports[i].tx_queue_num,
+ &hw, &queue_id);
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ data.tx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPTC(queue_id));
+ data.rx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPRC(queue_id));
+ data.rx_drop =
+ FM10K_READ_REG(hw, FM10K_QPRDC(queue_id));
+ mydata.tx_pkts += data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop;
+ printf(" queue %d(%d:%d) tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n", j,
+ pf_no, queue_id,
+ (unsigned long long)(data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts),
+ (unsigned long long)(data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts),
+ (unsigned long long)(data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop));
+ last_queue_data[pf_no][queue_id] = data;
+ }
+ printf(" total tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+ }
+}
+
+
+static void
+fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
+{
+ unsigned int i;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ uint64_t *pdata;
+
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ rx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ printf("port %d rx bank %d idx %d pkt %llu\n", lport,
+ rx_port_cnt_map_table[i].bank,
+ rx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ tx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ printf("port %d tx bank %d idx %d pkt %llu\n",
+ lport,
+ tx_port_cnt_map_table[i].bank,
+ tx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+}
+
+void
+fm10k_stats_port_bank_print(struct fm10k_switch *sw)
+{
+ int i;
+
+ for (i = 0;
+ i < FM10K_SW_EPLS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->epl_map[i].logical_port);
+ }
+ for (i = 0;
+ i < FM10K_SW_PEPS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->pep_map[i].logical_port);
+ }
+}
+
+
+void *
+fm10k_switch_process_stats(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ usec_delay(5000000);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_ENABLE))
+ return NULL;
+
+ if (!sw->dpdk_cfg->stats_interval)
+ return NULL;
+
+ while (1) {
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_PORT)) {
+ printf("--- port statistic ---\n");
+ fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU)) {
+ printf("--- ffu statistic ---\n");
+ fm10k_stats_ffu_count_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_MORE)) {
+ printf("--- detail statistic ---\n");
+ fm10k_stats_port_bank_print(sw);
+ }
+ usec_delay(1000000 * sw->dpdk_cfg->stats_interval);
+ }
+ return NULL;
+}
+
diff --git a/drivers/net/fm10k/switch/fm10k_stats.h b/drivers/net/fm10k/switch/fm10k_stats.h
new file mode 100644
index 0000000..5180312
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.h
@@ -0,0 +1,257 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#ifndef _fm10k_stats_h_
+#define _fm10k_stats_h_
+
+
+#include <stdint.h>
+
+
+#define FM10K_STATS_VERSION (1 << 4)
+
+
+struct fm10k_scatter_gather_entry {
+ uint32_t addr;
+ uint32_t count;
+ uint32_t *data;
+
+};
+
+
+struct fm10k_port_counters {
+ uint64_t cnt_version;
+ uint64_t cnt_rx_ucst_pkts;
+ uint64_t cnt_rx_ucst_pkts_nonip;
+ uint64_t cnt_rx_ucst_pkts_ipv4;
+ uint64_t cnt_rx_ucst_pkts_ipv6;
+ uint64_t cnt_rx_bcst_pkts;
+ uint64_t cnt_rx_bcst_pkts_nonip;
+ uint64_t cnt_rx_bcst_pkts_ipv4;
+ uint64_t cnt_rx_bcst_pkts_ipv6;
+ uint64_t cnt_rx_mcst_pkts;
+ uint64_t cnt_rx_mcst_pkts_nonip;
+ uint64_t cnt_rx_mcst_pkts_ipv4;
+ uint64_t cnt_rx_mcst_pkts_ipv6;
+ uint64_t cnt_rx_pause_pkts;
+ uint64_t cnt_rx_cbpause_pkts;
+ uint64_t cnt_rx_fcs_errors;
+ uint64_t cnt_rx_symbol_errors;
+ uint64_t cnt_rx_framesize_errors;
+ uint64_t cnt_rx_framing_error_pkts;
+ uint64_t cnt_rx_minto63_pkts;
+ uint64_t cnt_rx_64_pkts;
+ uint64_t cnt_rx_65to127_pkts;
+ uint64_t cnt_rx_128to255_pkts;
+ uint64_t cnt_rx_256to511_pkts;
+ uint64_t cnt_rx_512to1023_pkts;
+ uint64_t cnt_rx_1024to1522_pkts;
+ uint64_t cnt_rx_1523to2047_pkts;
+ uint64_t cnt_rx_2048to4095_pkts;
+ uint64_t cnt_rx_4096to8191_pkts;
+ uint64_t cnt_rx_8192to10239_pkts;
+ uint64_t cnt_rx_10240tomax_pkts;
+ uint64_t cnt_rx_minto63_octets;
+ uint64_t cnt_rx_64_octets;
+ uint64_t cnt_rx_65to127_octets;
+ uint64_t cnt_rx_128to255_octets;
+ uint64_t cnt_rx_256to511_octets;
+ uint64_t cnt_rx_512to1023_octets;
+ uint64_t cnt_rx_1024to1522_octets;
+ uint64_t cnt_rx_1523to2047_octets;
+ uint64_t cnt_rx_2048to4095_octets;
+ uint64_t cnt_rx_4096to8191_octets;
+ uint64_t cnt_rx_8192to10239_octets;
+ uint64_t cnt_rx_10240tomax_octets;
+ uint64_t cnt_rx_octets_nonip;
+ uint64_t cnt_rx_octets_ipv4;
+ uint64_t cnt_rx_octets_ipv6;
+ uint64_t cnt_rx_ucst_octets_nonip;
+ uint64_t cnt_rx_ucst_octets_ipv4;
+ uint64_t cnt_rx_ucst_octets_ipv6;
+ uint64_t cnt_rx_bcst_octets_nonip;
+ uint64_t cnt_rx_bcst_octets_ipv4;
+ uint64_t cnt_rx_bcst_octets_ipv6;
+ uint64_t cnt_rx_mcst_octets_nonip;
+ uint64_t cnt_rx_mcst_octets_ipv4;
+ uint64_t cnt_rx_mcst_octets_ipv6;
+ uint64_t cnt_rx_pause_octets;
+ uint64_t cnt_rx_cbpause_octets;
+ uint64_t cnt_rx_fcs_errors_octets;
+ uint64_t cnt_rx_framing_error_octets;
+ uint64_t cnt_rx_good_octets;
+ uint64_t cnt_rx_bad_octets;
+ uint64_t cnt_rx_priority_pkts[16];
+ uint64_t cnt_rx_invalid_priority_pkts;
+ uint64_t cnt_rx_priority_octets[16];
+ uint64_t cnt_rx_invalid_priority_octets;
+ uint64_t cnt_fid_forwarded_pkts;
+ uint64_t cnt_flood_forwarded_pkts;
+ uint64_t cnt_glort_switched_pkts;
+ uint64_t cnt_glort_routed_pkts;
+ uint64_t cnt_specially_handled_pkts;
+ uint64_t cnt_parse_err_drop_pkts;
+ uint64_t cnt_parity_error_pkts;
+ uint64_t cnt_trapped_pkts;
+ uint64_t cnt_pause_drop_pkts;
+ uint64_t cnt_stp_drop_pkts;
+ uint64_t cnt_stp_ingress_drops_pkts;
+ uint64_t cnt_stp_egress_drops_pkts;
+ uint64_t cnt_reserved_trap_pkts;
+ uint64_t cnt_security_violation_pkts;
+ uint64_t cnt_vlan_tag_drop_pkts;
+ uint64_t cnt_vlan_ingressbv_pkts;
+ uint64_t cnt_vlan_egressbv_pkts;
+ uint64_t cnt_loopback_drops_pkts;
+ uint64_t cnt_glort_miss_drop_pkts;
+ uint64_t cnt_ffu_drop_pkts;
+ uint64_t cnt_invalid_drop_pkts;
+ uint64_t cnt_policer_drop_pkts;
+ uint64_t cnt_ttl_drop_pkts;
+ uint64_t cnt_global_wmdrop_pkts;
+ uint64_t cnt_rx_mpdrop_pkts;
+ uint64_t cnt_rx_hogdrop_pkts;
+ uint64_t cnt_tx_hogdrop_pkts;
+ uint64_t cnt_other_pkts;
+ uint64_t cnt_flood_control_drop_pkts;
+ uint64_t cnt_cmpriv_drop_pkts;
+ uint64_t cnt_smp0_drop_pkts;
+ uint64_t cnt_smp1_drop_pkts;
+ uint64_t cnt_rx_hog0_drop_pkts;
+ uint64_t cnt_rx_hog1_drop_pkts;
+ uint64_t cnt_tx_hog0_drop_pkts;
+ uint64_t cnt_tx_hog1_drop_pkts;
+ uint64_t cnt_rate_limit0_drop_pkts;
+ uint64_t cnt_rate_limit1_drop_pkts;
+ uint64_t cnt_bad_smp_drop_pkts;
+ uint64_t cnt_trigger_drop_redir_pkts;
+ uint64_t cnt_trigger_drop_pkts;
+ uint64_t cnt_trigger_redir_pkts;
+ uint64_t cnt_glort_forwarded_pkts;
+ uint64_t cnt_trigger_mirrored_pkts;
+ uint64_t cnt_broadcast_drop_pkts;
+ uint64_t cnt_dlf_drop_pkts;
+ uint64_t cnt_rx_cm_drop_pkts;
+ uint64_t cnt_fid_forwarded_octets;
+ uint64_t cnt_flood_forwarded_octets;
+ uint64_t cnt_specially_handled_octets;
+ uint64_t cnt_parse_err_drop_octets;
+ uint64_t cnt_parity_error_octets;
+ uint64_t cnt_trapped_octets;
+ uint64_t cnt_pause_drop_octets;
+ uint64_t cnt_stp_drop_octets;
+ uint64_t cnt_security_violation_octets;
+ uint64_t cnt_vlan_tag_drop_octets;
+ uint64_t cnt_vlan_ingressbv_octets;
+ uint64_t cnt_vlan_egressbv_octets;
+ uint64_t cnt_loopback_drop_octets;
+ uint64_t cnt_glort_miss_drop_octets;
+ uint64_t cnt_ffu_drop_octets;
+ uint64_t cnt_policer_drop_octets;
+ uint64_t cnt_ttl_drop_octets;
+ uint64_t cnt_other_octets;
+ uint64_t cnt_flood_control_drop_octets;
+ uint64_t cnt_cmpriv_drop_octets;
+ uint64_t cnt_smp0_drop_octets;
+ uint64_t cnt_smp1_drop_octets;
+ uint64_t cnt_rx_hog0_drop_octets;
+ uint64_t cnt_rx_hog1_drop_octets;
+ uint64_t cnt_tx_hog0_drop_octets;
+ uint64_t cnt_tx_hog1_drop_octets;
+ uint64_t cnt_trigger_drop_octets;
+ uint64_t cnt_trigger_redir_octets;
+ uint64_t cnt_glort_forwarded_octets;
+ uint64_t cnt_tx_ucst_pkts;
+ uint64_t cnt_tx_bcst_pkts;
+ uint64_t cnt_tx_mcst_pkts;
+ uint64_t cnt_tx_ucst_pkts_nonip;
+ uint64_t cnt_tx_bcst_pkts_nonip;
+ uint64_t cnt_tx_mcst_pkts_nonip;
+ uint64_t cnt_tx_ucst_pkts_ip;
+ uint64_t cnt_tx_bcst_pkts_ip;
+ uint64_t cnt_tx_mcst_pkts_ip;
+ uint64_t cnt_tx_pause_pkts;
+ uint64_t cnt_tx_cbpause_pkts;
+ uint64_t cnt_tx_fcs_err_drop_pkts;
+ uint64_t cnt_tx_framing_error_pkts;
+ uint64_t cnt_tx_error_sent_pkts;
+ uint64_t cnt_tx_error_drop_pkts;
+ uint64_t cnt_tx_timeout_pkts;
+ uint64_t cnt_tx_outofmem_err_pkts;
+ uint64_t cnt_tx_unrepair_ecc_pkts;
+ uint64_t cnt_tx_loopback_pkts;
+ uint64_t cnt_tx_ttl_drop_pkts;
+ uint64_t cnt_tx_minto63_pkts;
+ uint64_t cnt_tx_64_pkts;
+ uint64_t cnt_tx_65to127_pkts;
+ uint64_t cnt_tx_128to255_pkts;
+ uint64_t cnt_tx_256to511_pkts;
+ uint64_t cnt_tx_512to1023_pkts;
+ uint64_t cnt_tx_1024to1522_pkts;
+ uint64_t cnt_tx_1523to2047_pkts;
+ uint64_t cnt_tx_2048to4095_pkts;
+ uint64_t cnt_tx_4096to8191_pkts;
+ uint64_t cnt_tx_8192to10239_pkts;
+ uint64_t cnt_tx_10240tomax_pkts;
+ uint64_t cnt_tx_minto63_octets;
+ uint64_t cnt_tx_64_octets;
+ uint64_t cnt_tx_65to127_octets;
+ uint64_t cnt_tx_128to255_octets;
+ uint64_t cnt_tx_256to511_octets;
+ uint64_t cnt_tx_512to1023_octets;
+ uint64_t cnt_tx_1024to1522_octets;
+ uint64_t cnt_tx_1523to2047_octets;
+ uint64_t cnt_tx_2048to4095_octets;
+ uint64_t cnt_tx_4096to8191_octets;
+ uint64_t cnt_tx_8192to10239_octets;
+ uint64_t cnt_tx_10240tomax_octets;
+ uint64_t cnt_tx_ucst_octets_nonip;
+ uint64_t cnt_tx_bcst_octets_nonip;
+ uint64_t cnt_tx_mcst_octets_nonip;
+ uint64_t cnt_tx_ucst_octets_ip;
+ uint64_t cnt_tx_bcst_octets_ip;
+ uint64_t cnt_tx_mcst_octets_ip;
+ uint64_t cnt_tx_ucst_octets;
+ uint64_t cnt_tx_mcst_octets;
+ uint64_t cnt_tx_bcst_octets;
+ uint64_t cnt_tx_fcs_err_drop_octets;
+ uint64_t cnt_tx_octets;
+ uint64_t cnt_tx_error_octets;
+ uint64_t cnt_tx_framing_error_octets;
+ uint64_t cnt_tx_pause_octets;
+ uint64_t cnt_tx_cbpause_octets;
+ uint64_t cnt_tx_fcs_errored_octets;
+ uint64_t cnt_tx_error_sent_octets;
+ uint64_t cnt_tx_timeout_octets;
+ uint64_t cnt_tx_outofmem_err_octets;
+ uint64_t cnt_tx_unrepair_ecc_octets;
+ uint64_t cnt_tx_loopback_octets;
+ uint64_t cnt_tx_ttl_drop_octets;
+ uint64_t cnt_tx_priority_octets[16];
+ uint64_t cnt_under_run_pkts;
+ uint64_t cnt_over_run_pkts;
+ uint64_t cnt_rx_fragment_pkts;
+ uint64_t cnt_rx_undersized_pkts;
+ uint64_t cnt_rx_jabber_pkts;
+ uint64_t cnt_corrupted_pkts;
+ uint64_t cnt_code_errors;
+ uint64_t cnt_rx_oversized_pkts;
+ uint64_t cnt_tx_fcs_errored_pkts;
+ uint64_t cnt_stats_drop_count_tx;
+ uint64_t cnt_stats_drop_count_rx;
+ uint64_t cnt_tx_mirror_pkts;
+ uint64_t cnt_tx_mirror_octets;
+ uint64_t cnt_tx_cmdrop_pkts;
+ uint64_t timestamp;
+};
+
+void fm10k_stats_rule_count_reg(uint16_t rule_id);
+void *fm10k_switch_process_stats(void *ctx_);
+
+void fm10k_stats_epl_port_print(struct fm10k_switch *sw);
+void fm10k_stats_dpdk_port_print(struct fm10k_switch *sw);
+void fm10k_stats_ffu_count_print(struct fm10k_switch *sw);
+void fm10k_stats_port_bank_print(struct fm10k_switch *sw);
+
+#endif /* _fm10k_stats_h */
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 4/7] net/fm10k: add flow and switch management
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
` (2 preceding siblings ...)
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 3/7] net/fm10k: add config ffu statistics support Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 5/7] net/fm10k: add switch initialization Xiaojun Liu
` (3 subsequent siblings)
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
To support switch management, add the following new files:
Add fm10k/switch/fm10k_flow.h.
Add fm10k/switch/fm10k_flow.c(support dpdk flow operations)
Add fm10k/switch/fm10k_switch.c(support switch management)
Modify fm10k/Makefile(add fm10k_flow.c
and fm10k_switch.c).
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 2 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 +++++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_switch.c | 2562 +++++++++++++++++++++++++++++++
4 files changed, 3462 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index ed73251..ab263c5 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -93,6 +93,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_switch.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_flow.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_flow.c b/drivers/net/fm10k/switch/fm10k_flow.c
new file mode 100644
index 0000000..353f021
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.c
@@ -0,0 +1,872 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <sys/queue.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_eth_ctrl.h>
+#include <rte_tailq.h>
+#include <rte_flow_driver.h>
+
+#include "fm10k_flow.h"
+#include "fm10k_switch.h"
+#include "fm10k_ffu.h"
+#include "fm10k_config.h"
+
+
+static int fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static struct rte_flow *fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static int fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error);
+static int fm10k_flow_flush(struct rte_eth_dev *dev,
+ struct rte_flow_error *error);
+static int fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error);
+
+const struct rte_flow_ops fm10k_flow_ops = {
+ .validate = fm10k_flow_validate,
+ .create = fm10k_flow_create,
+ .destroy = fm10k_flow_destroy,
+ .flush = fm10k_flow_flush,
+};
+
+union fm10k_filter_t cons_filter;
+enum rte_filter_type fm10k_cons_filter_type = RTE_ETH_FILTER_NONE;
+
+/**
+ * MPLS filter configuration.
+ */
+enum fm10k_mpls_type {
+ FM10K_MPLS_TYPE_UNI,
+ FM10K_MPLS_TYPE_MULTI,
+};
+
+enum fm10k_mpls_action {
+ FM10K_MPLS_ACTION_DROP,
+ FM10K_MPLS_ACTION_QUEUE,
+};
+
+struct fm10k_mpls_filter_conf {
+ enum fm10k_mpls_type mpls_type; /**< mandatory for MPLS */
+ uint32_t mpls_header; /**< MPLS header */
+ uint32_t mpls_header_mask; /**< MPLS header mask */
+ enum fm10k_mpls_action mpls_action;
+ uint16_t queue;
+ uint8_t ffu_id;
+ uint8_t ffu_prio;
+};
+
+/**
+ * VLAN filter configuration.
+ */
+struct fm10k_vlan_filter_conf {
+ int ffu_id;
+ uint8_t ffu_prio;
+ uint8_t is_ingress;
+ uint8_t port;
+ uint8_t in_ext_port;
+ uint8_t out_ext_port;
+ uint16_t in_vlan;
+ uint16_t out_vlan;
+};
+
+
+union fm10k_filter_t {
+ struct fm10k_mpls_filter_conf mpls_filter;
+ struct fm10k_vlan_filter_conf vlan_filter;
+};
+
+typedef int (*parse_filter_t)(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+struct fm10k_valid_pattern {
+ enum rte_flow_item_type *items;
+ parse_filter_t parse_filter;
+};
+
+static enum rte_flow_item_type pattern_mpls_1[] = {
+ RTE_FLOW_ITEM_TYPE_ETH,
+ RTE_FLOW_ITEM_TYPE_MPLS,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_1[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_2[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static int fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+static struct fm10k_valid_pattern fm10k_supported_patterns[] = {
+ /* MPLS */
+ { pattern_mpls_1, fm10k_flow_parse_mpls_filter },
+ /* VLAN */
+ { pattern_vlan_1, fm10k_flow_parse_vlan_filter },
+ /* VLAN */
+ { pattern_vlan_2, fm10k_flow_parse_vlan_filter },
+};
+
+static const struct rte_flow_action *
+fm10k_next_item_of_action(const struct rte_flow_action *actions,
+ uint32_t *index)
+{
+ static const struct rte_flow_action *act;
+
+ act = actions + *index;
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID) {
+ (*index)++;
+ act = actions + *index;
+ }
+ return act;
+}
+
+/* Find the first VOID or non-VOID item pointer */
+static const struct rte_flow_item *
+fm10k_find_first_item(const struct rte_flow_item *item, bool is_void)
+{
+ bool is_find;
+
+ while (item->type != RTE_FLOW_ITEM_TYPE_END) {
+ if (is_void)
+ is_find = item->type == RTE_FLOW_ITEM_TYPE_VOID;
+ else
+ is_find = item->type != RTE_FLOW_ITEM_TYPE_VOID;
+ if (is_find)
+ break;
+ item++;
+ }
+ return item;
+}
+
+/* Skip all VOID items of the pattern */
+static void
+fm10k_pattern_skip_void_item(struct rte_flow_item *items,
+ const struct rte_flow_item *pattern)
+{
+ uint32_t cpy_count = 0;
+ const struct rte_flow_item *pb = pattern, *pe = pattern;
+
+ for (;;) {
+ /* Find a non-void item first */
+ pb = fm10k_find_first_item(pb, false);
+ if (pb->type == RTE_FLOW_ITEM_TYPE_END) {
+ pe = pb;
+ break;
+ }
+
+ /* Find a void item */
+ pe = fm10k_find_first_item(pb + 1, true);
+
+ cpy_count = pe - pb;
+ rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+
+ items += cpy_count;
+
+ if (pe->type == RTE_FLOW_ITEM_TYPE_END) {
+ pb = pe;
+ break;
+ }
+
+ pb = pe + 1;
+ }
+ /* Copy the END item. */
+ rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+}
+
+/* Check if the pattern matches a supported item type array */
+static bool
+fm10k_match_pattern(enum rte_flow_item_type *item_array,
+ struct rte_flow_item *pattern)
+{
+ struct rte_flow_item *item = pattern;
+
+ while ((*item_array == item->type) &&
+ (*item_array != RTE_FLOW_ITEM_TYPE_END)) {
+ item_array++;
+ item++;
+ }
+
+ return (*item_array == RTE_FLOW_ITEM_TYPE_END &&
+ item->type == RTE_FLOW_ITEM_TYPE_END);
+}
+
+/* Find if there's parse filter function matched */
+static parse_filter_t
+fm10k_find_parse_filter_func(struct rte_flow_item *pattern, uint32_t *idx)
+{
+ parse_filter_t parse_filter = NULL;
+ uint8_t i = *idx;
+
+ for (; i < RTE_DIM(fm10k_supported_patterns); i++) {
+ if (fm10k_match_pattern(fm10k_supported_patterns[i].items,
+ pattern)) {
+ parse_filter = fm10k_supported_patterns[i].parse_filter;
+ break;
+ }
+ }
+
+ *idx = ++i;
+
+ return parse_filter;
+}
+
+/* Parse attributes */
+static int
+fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ /* Not supported */
+ if (attr->group) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+ attr, "Not support group.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+/*
+ * MPLS
+ */
+/* 1. Last in item should be NULL as range is not supported.
+ * 2. Supported filter types: MPLS label.
+ * 3. Mask of fields which need to be matched should be
+ * filled with 1.
+ * 4. Mask of fields which needn't to be matched should be
+ * filled with 0.
+ */
+static int
+fm10k_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_mpls *mpls_spec;
+ const struct rte_flow_item_mpls *mpls_mask;
+ const struct rte_flow_item_eth *eth_spec;
+ enum rte_flow_item_type item_type;
+ const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
+ uint32_t label_be = 0;
+ uint32_t be_mask = 0;
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_ETH:
+ if (!item->spec || item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid ETH item");
+ return -rte_errno;
+ }
+ eth_spec =
+ (const struct rte_flow_item_eth *)item->spec;
+
+ if (rte_be_to_cpu_16(eth_spec->type) == 0x8847) {
+ filter->mpls_type = FM10K_MPLS_TYPE_UNI;
+ } else if (rte_be_to_cpu_16(eth_spec->type) == 0x8848) {
+ filter->mpls_type = FM10K_MPLS_TYPE_MULTI;
+ } else {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Invalid ETH item");
+ return -rte_errno;
+ }
+ break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ mpls_spec =
+ (const struct rte_flow_item_mpls *)item->spec;
+ mpls_mask =
+ (const struct rte_flow_item_mpls *)item->mask;
+
+ if (!mpls_spec || !mpls_mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS item");
+ return -rte_errno;
+ }
+
+ if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS label mask");
+ return -rte_errno;
+ }
+ rte_memcpy(((uint8_t *)&label_be + 1),
+ mpls_spec->label_tc_s, 3);
+ rte_memcpy(((uint8_t *)&be_mask + 1),
+ mpls_mask->label_tc_s, 3);
+ filter->mpls_header =
+ rte_be_to_cpu_32(label_be) >> 4;
+ filter->mpls_header_mask =
+ rte_be_to_cpu_32(be_mask) >> 4;
+
+ fm10k_cons_filter_type = RTE_ETH_FILTER_TUNNEL;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* MPLS action only supports QUEUE or DROP. */
+static int
+fm10k_flow_parse_mpls_action(const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ const struct rte_flow_action_queue *act_q;
+ uint32_t index = 0;
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_QUEUE &&
+ act->type != RTE_FLOW_ACTION_TYPE_DROP) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ if (act->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->mpls_action = FM10K_MPLS_ACTION_QUEUE;
+ filter->queue = act_q->index;
+ } else {
+ filter->mpls_action = FM10K_MPLS_ACTION_DROP;
+ }
+
+ /* Check if the next non-void item is END */
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ struct fm10k_mpls_filter_conf *mpls_filter =
+ &filter->mpls_filter;
+ int ret;
+
+ ret = fm10k_flow_parse_mpls_pattern(dev, pattern,
+ error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_mpls_action(actions, error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+
+/*
+ * VLAN
+ */
+static int
+fm10k_flow_parse_vlan_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_vlan *vlan_spec;
+ const struct rte_flow_item_phy_port *pp_spec;
+ enum rte_flow_item_type item_type;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan pattern ffu id %d", filter->ffu_id);
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_VLAN:
+ vlan_spec =
+ (const struct rte_flow_item_vlan *)item->spec;
+
+ if (!vlan_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid VLAN item");
+ return -rte_errno;
+ }
+ break;
+
+ case RTE_FLOW_ITEM_TYPE_PHY_PORT:
+ pp_spec =
+ (const struct rte_flow_item_phy_port *)item->spec;
+ if (!pp_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid PHY PORT item");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_flow_parse_vlan_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ uint32_t index = 0;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan action name %s ffu id %d",
+ dev->device->name, filter->ffu_id);
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK &&
+ act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+ if (act->type == RTE_FLOW_ACTION_TYPE_END)
+ return 0;
+
+ index++;
+ /* Check if the next non-void item is END */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ int ret;
+ struct fm10k_vlan_filter_conf *vlan_filter =
+ &filter->vlan_filter;
+
+ ret = fm10k_flow_parse_vlan_pattern(dev, pattern,
+ error, vlan_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_vlan_action(dev, actions, error, vlan_filter);
+ if (ret)
+ return ret;
+
+ if (attr->ingress)
+ vlan_filter->is_ingress = 1;
+ else if (attr->egress)
+ vlan_filter->is_ingress = 0;
+ vlan_filter->ffu_prio = attr->priority;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+/*
+ *
+ */
+static int
+fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct rte_flow_item *items; /* internal pattern w/o VOID items */
+ parse_filter_t parse_filter;
+ uint32_t item_num = 0; /* non-void item number of pattern*/
+ uint32_t i = 0;
+ bool flag = false;
+ int ret = -1;
+
+ if (!pattern) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "NULL pattern.");
+ return -rte_errno;
+ }
+
+ if (!actions) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_NUM,
+ NULL, "NULL action.");
+ return -rte_errno;
+ }
+
+ if (!attr) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR,
+ NULL, "NULL attribute.");
+ return -rte_errno;
+ }
+
+ /* Get the non-void item number of pattern */
+ while ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_END) {
+ if ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_VOID)
+ item_num++;
+ i++;
+ }
+ item_num++;
+
+ items = rte_zmalloc("fm10k_pattern",
+ item_num * sizeof(struct rte_flow_item), 0);
+ if (!items) {
+ rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "No memory for PMD internal items.");
+ return -ENOMEM;
+ }
+
+ fm10k_pattern_skip_void_item(items, pattern);
+
+ i = 0;
+ do {
+ parse_filter = fm10k_find_parse_filter_func(items, &i);
+ if (!parse_filter && !flag) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ pattern, "Unsupported pattern");
+ rte_free(items);
+ return -rte_errno;
+ }
+ if (parse_filter)
+ ret = parse_filter(dev, attr, items, actions,
+ error, &cons_filter);
+ flag = true;
+ } while ((ret < 0) && (i < RTE_DIM(fm10k_supported_patterns)));
+
+ rte_free(items);
+
+ return ret;
+}
+
+static struct fm10k_cfg_flow *
+fm10k_flow_cfg_transfer(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ int i;
+ u8 port_id;
+ int set_port_num = 0, set_vlan_num = 0;
+ u16 fw_port_id = 0, bp_port_id = 0;
+ u16 filter_vlan_id = 0, fw_vlan_id = 0, bp_vlan_id = 0;
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_cfg_flow *cf;
+
+ cf = rte_zmalloc("fm10k_rule", sizeof(struct fm10k_cfg_flow), 0);
+ if (!cf) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return NULL;
+ }
+ memset(cf, 0, sizeof(struct fm10k_cfg_flow));
+
+ port_id = fm10k_switch_dpdk_port_no_get(hw);
+ for (i = 0; i < 4; i++) {
+ if (pattern[i].type == RTE_FLOW_ITEM_TYPE_VLAN) {
+ filter_vlan_id =
+ rte_be_to_cpu_16
+ (((const struct rte_flow_item_vlan *)
+ pattern[i].spec)->tci);
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_PHY_PORT) {
+ if (set_port_num)
+ bp_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ else
+ fw_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ set_port_num++;
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_END) {
+ break;
+ }
+ }
+
+ for (i = 0; i < 3; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_MARK) {
+ if (set_vlan_num)
+ bp_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ else
+ fw_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ set_vlan_num++;
+ } else if (actions[i].type == RTE_FLOW_ACTION_TYPE_END) {
+ break;
+ }
+ }
+
+ if (attr->ingress && !attr->egress) {
+ /* this port is DPDK port and it is destination port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = fw_port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && attr->egress) {
+ /* this port is DPDK port and it is source port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && !attr->egress) {
+ /* two ports are external port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else {
+ /* two ports are DPDK port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ }
+
+ if (set_port_num == 2 && set_vlan_num == 2) {
+ cf->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[1].port_no = bp_port_id;
+ cf->fw_port[1].vlan_id = bp_vlan_id;
+ }
+
+ return cf;
+}
+
+
+static struct rte_flow *
+fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ struct fm10k_switch *sw = fm10k_switch_get();
+ struct fm10k_cfg_flow *cf;
+ int ret;
+
+ flow = rte_zmalloc("fm10k_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return flow;
+ }
+
+ ret = fm10k_flow_validate(dev, attr, pattern, actions, error);
+ if (ret < 0)
+ return NULL;
+
+ cf = fm10k_flow_cfg_transfer(dev, attr, pattern, actions, error);
+ if (!cf)
+ goto free_flow;
+
+ flow->rule = cf;
+ fm10k_ffu_flow_enable(sw, cf);
+ fm10k_config_flow_list_add_tail(fm10k_config_flowset_current_get(), cf);
+
+ TAILQ_INSERT_TAIL((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw), flow, node);
+ return flow;
+
+free_flow:
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to create flow.");
+ rte_free(flow);
+ return NULL;
+}
+
+static int
+fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret = 0;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (flow->rule) {
+ fm10k_config_flow_list_delete
+ ((struct fm10k_cfg_flow *)flow->rule);
+ fm10k_ffu_flow_disable(sw,
+ (struct fm10k_cfg_flow *)flow->rule);
+ }
+
+ if (!ret) {
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow->rule);
+ rte_free(flow);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to destroy flow.");
+ }
+ return ret;
+}
+
+
+static int
+fm10k_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ void *temp;
+ int ret = 0;
+ struct fm10k_cfg_flow *cf;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ /* Delete flows in flow list. */
+ TAILQ_FOREACH_SAFE(flow,
+ (struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ node, temp) {
+ cf = flow->rule;
+ if (cf) {
+ fm10k_config_flow_list_delete(cf);
+ fm10k_ffu_flow_disable(sw, cf);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "No such rule in flow.");
+ }
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+void
+fm10k_flow_list_init(void *flow_list)
+{
+ TAILQ_INIT((struct fm10k_flow_list *)flow_list);
+}
+
+/* Flow operations */
+const struct rte_flow_ops *
+fm10k_flow_ops_get(void)
+{
+ return &fm10k_flow_ops;
+}
+
+
diff --git a/drivers/net/fm10k/switch/fm10k_flow.h b/drivers/net/fm10k/switch/fm10k_flow.h
new file mode 100644
index 0000000..c538544
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_FLOW_H_
+#define _FM10K_SW_FLOW_H_
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+/*
+ * Struct to store flow created.
+ */
+struct rte_flow {
+ TAILQ_ENTRY(rte_flow) node;
+ enum rte_filter_type filter_type;
+ void *rule;
+};
+
+TAILQ_HEAD(fm10k_flow_list, rte_flow);
+
+#endif /* _FM10K_SW_FLOW_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.c b/drivers/net/fm10k/switch/fm10k_switch.c
new file mode 100644
index 0000000..c3887d0
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.c
@@ -0,0 +1,2562 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_ethdev.h>
+#include <rte_ethdev_pci.h>
+
+#include <rte_malloc.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_serdes.h"
+#include "fm10k_i2c.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+#include "fm10k_config.h"
+
+static struct fm10k_device_info fm10k_device_table[] = {
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4,
+ "Silicom PE3100G2DQiR-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4,
+ "Silicom PE3100G2DQiRL-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4,
+ "Silicom PE3100G2DQiRM-QX4/QS4/QL4", 2, 100, 2, 4 },
+};
+
+
+static struct fm10k_switch fm10k_sw;
+
+#define FM10K_AM_TIMEOUT 16384
+#define FM10K_COMP_PPM_SCALE 1000000
+
+#define FM10K_LED_POLL_INTERVAL_MS 500
+#define FM10K_LED_BLINKS_PER_SECOND 2
+
+/*
+ * GLORT MAP
+ */
+#define FM10K_SW_EPLA_GLORT 0x10
+#define FM10K_SW_EPLB_GLORT 0x20
+#define FM10K_SW_PEP01_GLORT 0x30
+#define FM10K_SW_PEP23_GLORT 0x40
+#define FM10K_SW_PEP45_GLORT 0x50
+#define FM10K_SW_PEP67_GLORT 0x60
+
+/*
+ * logical port number
+ */
+#define FM10K_SW_EPLA_LOGICAL_PORT 1
+#define FM10K_SW_EPLB_LOGICAL_PORT 2
+
+#define FM10K_SW_PEP01_LOGICAL_PORT 3
+#define FM10K_SW_PEP23_LOGICAL_PORT 4
+#define FM10K_SW_PEP45_LOGICAL_PORT 5
+#define FM10K_SW_PEP67_LOGICAL_PORT 6
+
+/*
+ * physical port number
+ */
+#define FM10K_SW_EPLA_PHYSICAL_PORT 0
+#define FM10K_SW_EPLB_PHYSICAL_PORT 4
+
+#define FM10K_SW_PEP01_PHYSICAL_PORT 36
+#define FM10K_SW_PEP23_PHYSICAL_PORT 40
+#define FM10K_SW_PEP45_PHYSICAL_PORT 44
+#define FM10K_SW_PEP67_PHYSICAL_PORT 48
+
+static struct fm10k_sw_port_map fm10k_pep_port_map[FM10K_SW_PEPS_SUPPORTED] = {
+ {
+ FM10K_SW_PEP01_GLORT,
+ FM10K_SW_PEP01_LOGICAL_PORT,
+ FM10K_SW_PEP01_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP23_GLORT,
+ FM10K_SW_PEP23_LOGICAL_PORT,
+ FM10K_SW_PEP23_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP45_GLORT,
+ FM10K_SW_PEP45_LOGICAL_PORT,
+ FM10K_SW_PEP45_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP67_GLORT,
+ FM10K_SW_PEP67_LOGICAL_PORT,
+ FM10K_SW_PEP67_PHYSICAL_PORT
+ },
+};
+
+static struct fm10k_sw_port_map fm10k_epl_port_map[FM10K_SW_EPLS_SUPPORTED] = {
+ {
+ FM10K_SW_EPLA_GLORT,
+ FM10K_SW_EPLA_LOGICAL_PORT,
+ FM10K_SW_EPLA_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_EPLB_GLORT,
+ FM10K_SW_EPLB_LOGICAL_PORT,
+ FM10K_SW_EPLB_PHYSICAL_PORT
+ },
+};
+
+/*
+ * use epl as external port map, only support QUAD_ON
+ */
+struct fm10k_sched_prog {
+ uint8_t idle;
+ uint8_t phys; /* physical port */
+ uint8_t log; /* logical port */
+ uint8_t quad; /* now, only support QUAD_ON */
+/* convert entry to idle if EPL in quad port mode */
+#define FM10K_SW_QUAD_OFF 0
+/* always use quad port mode */
+#define FM10K_SW_QUAD_ON 1
+/* use quad port mode if link speed is 40/100G */
+#define FM10K_SW_QUAD_40_100 2
+};
+
+static struct fm10k_sched_prog
+ fm10k_sched_prog[FM10K_SW_PEPS_SUPPORTED + FM10K_SW_EPLS_SUPPORTED + 1];
+
+uint32_t
+fm10k_switch_pf_logical_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].logical_port;
+}
+
+uint32_t
+fm10k_switch_epl_logical_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].logical_port;
+}
+
+uint32_t
+fm10k_switch_vf_glort_get(uint8_t vf_no)
+{
+ return FM10K_SW_VF_GLORT_START + vf_no;
+}
+
+uint32_t
+fm10k_switch_pf_glort_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].glort;
+}
+
+uint32_t
+fm10k_switch_epl_glort_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].glort;
+}
+
+uint32_t
+fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2)
+{
+ uint8_t idx;
+ if (pf1 > pf2)
+ idx = (pf2 & 0xf) | (pf1 << 4 & 0xf0);
+ else
+ idx = (pf1 & 0xf) | (pf2 << 4 & 0xf0);
+ return FM10K_SW_PFS_GLORT_START + idx;
+}
+
+struct fm10k_multi_glort {
+ uint8_t lport1;
+ uint8_t lport2;
+ uint16_t vlan1;
+ uint16_t vlan2;
+} fm10k_multi_glorts[FM10K_SW_FFU_RULE_MAX];
+
+uint32_t
+fm10k_switch_multi_glort_get(uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (lport1 == fm10k_multi_glorts[i].lport1 &&
+ lport2 == fm10k_multi_glorts[i].lport2 &&
+ vlan1 == fm10k_multi_glorts[i].vlan1 &&
+ vlan2 == fm10k_multi_glorts[i].vlan2) {
+ if (p_new != NULL)
+ *p_new = false;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (fm10k_multi_glorts[i].lport1 == 0 &&
+ fm10k_multi_glorts[i].lport2 == 0 &&
+ fm10k_multi_glorts[i].vlan1 == 0 &&
+ fm10k_multi_glorts[i].vlan2 == 0) {
+ fm10k_multi_glorts[i].lport1 = lport1;
+ fm10k_multi_glorts[i].lport2 = lport2;
+ fm10k_multi_glorts[i].vlan1 = vlan1;
+ fm10k_multi_glorts[i].vlan2 = vlan2;
+ if (p_new != NULL)
+ *p_new = true;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ return 0;
+}
+
+
+/*
+ * Note that for physical port numbers, the initial values used for
+ * the EPL entries assume EPL[A] is EPL[0] and EPL[B] is EPL[1].
+ * fm10k_switch_determine_epls() will update these physical port
+ * numbers based on the actual A and B indices.
+ */
+static void
+fm10k_switch_set_sched_prog(void)
+{
+ int i;
+ int start = 0;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_sched_prog[i].idle = 0;
+ fm10k_sched_prog[i].phys = fm10k_epl_port_map[i].physical_port;
+ fm10k_sched_prog[i].log = fm10k_epl_port_map[i].logical_port;
+ fm10k_sched_prog[i].quad = FM10K_SW_QUAD_ON;
+ }
+ start += FM10K_SW_EPLS_SUPPORTED;
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ fm10k_sched_prog[start + i].idle = 0;
+ fm10k_sched_prog[start + i].phys =
+ fm10k_pep_port_map[i].physical_port;
+ fm10k_sched_prog[start + i].log =
+ fm10k_pep_port_map[i].logical_port;
+ fm10k_sched_prog[start + i].quad = FM10K_SW_QUAD_40_100;
+ }
+ start += FM10K_SW_PEPS_SUPPORTED;
+ fm10k_sched_prog[start].idle = 1;
+}
+
+struct fm10k_device_info*
+fm10k_get_device_info(struct fm10k_hw *hw)
+{
+ unsigned int i;
+ struct fm10k_device_info *info;
+ uint16_t pci_vendor = hw->vendor_id;
+ uint16_t pci_device = hw->device_id;
+ uint16_t pci_subvendor = hw->subsystem_vendor_id;
+ uint16_t pci_subdevice = hw->subsystem_device_id;
+
+ if (pci_vendor != FM10K_SW_VENDOR_ID_INTEL ||
+ pci_device != FM10K_SW_DEV_ID_FM10K)
+ return (NULL);
+
+ for (i = 0;
+ i < sizeof(fm10k_device_table) /
+ sizeof(fm10k_device_table[0]);
+ i++) {
+ info = &fm10k_device_table[i];
+ if (pci_subvendor == info->subvendor &&
+ pci_subdevice == info->subdevice) {
+ return info;
+ }
+ }
+
+ return NULL;
+}
+
+
+static void
+fm10k_switch_determine_epls(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t phys;
+
+ sw->epla_no = 0;
+ sw->eplb_no = 6;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ sw->epla_no = 1;
+ sw->eplb_no = 6;
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE325G2DSIR):
+ sw->epla_no = 1;
+ sw->eplb_no = 7;
+ break;
+ }
+
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ if (fm10k_sched_prog[i].idle)
+ continue;
+
+ phys = fm10k_sched_prog[i].phys;
+ if (phys <= 3) {
+ /* Substitute actual epla phys port number */
+ fm10k_sched_prog[i].phys = sw->epla_no * 4 + phys;
+ } else if (phys >= 4 && phys <= 7) {
+ /* Substitute actual eplb phys port number */
+ fm10k_sched_prog[i].phys = sw->eplb_no * 4 + (phys - 4);
+ }
+ }
+}
+
+static void
+fm10k_hw_eicr_disable_source(struct fm10k_switch *sw, unsigned int source)
+{
+ unsigned int shift;
+
+ switch (source) {
+ case FM10K_SW_EICR_PCA_FAULT:
+ shift = FM10K_SW_EICR_PCA_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_THI_FAULT:
+ shift = FM10K_SW_EICR_THI_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_FUM_FAULT:
+ shift = FM10K_SW_EICR_FUM_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAILBOX:
+ shift = FM10K_SW_EICR_MAILBOX_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_READY:
+ shift = FM10K_SW_EICR_SWITCH_READY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_NREADY:
+ shift = FM10K_SW_EICR_SWITCH_NREADY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_INT:
+ shift = FM10K_SW_EICR_SWITCH_INT_SHIFT;
+ break;
+ case FM10K_SW_EICR_SRAM_ERROR:
+ shift = FM10K_SW_EICR_SRAM_ERROR_SHIFT;
+ break;
+ case FM10K_SW_EICR_VFLR:
+ shift = FM10K_SW_EICR_VFLR_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAX_HOLD_TIME:
+ shift = FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT;
+ break;
+ default:
+ return;
+ }
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_DISABLE << (shift * 2));
+ fm10k_write_flush(sw);
+}
+
+
+unsigned int
+fm10k_switch_eplidx_to_eplno(struct fm10k_switch *sw, unsigned int eplidx)
+{
+ return eplidx ? sw->eplb_no : sw->epla_no;
+}
+
+
+static uint16_t
+fm10k_switch_ppm_to_tx_clk_compensation_timeout(uint32_t pcs, uint32_t num_ppm)
+{
+ unsigned int scale;
+ unsigned int ppm;
+ unsigned int timeout;
+ unsigned int am_ppm;
+
+ if (num_ppm == 0 || pcs == FM10K_SW_EPL_PCS_SEL_DISABLE) {
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER)
+ return (FM10K_AM_TIMEOUT / 2);
+ else
+ return (0);
+ }
+
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER) {
+ am_ppm = 1000000 / FM10K_AM_TIMEOUT;
+ scale = FM10K_COMP_PPM_SCALE / 2;
+ } else {
+ am_ppm = 0;
+ scale = FM10K_COMP_PPM_SCALE;
+ }
+
+ ppm = num_ppm + am_ppm;
+ timeout = scale / ppm;
+
+ if (timeout >= 0xffff)
+ return 0xffff;
+ else
+ return timeout;
+}
+
+
+static int
+fm10k_switch_configure_epls(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ext_ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_device_info *cfg = sw->info;
+ u32 mac_cfg[FM10K_SW_MAC_CFG_ARRAY_SIZE];
+ u32 data, pcs, qpl;
+ u32 pcstmp;
+ unsigned int i, j;
+ unsigned int dic_enable;
+ unsigned int anti_bubble_wm;
+ unsigned int rate_fifo_wm;
+ unsigned int rate_fifo_slow_inc, rate_fifo_fast_inc;
+ int error;
+ u16 timeout;
+
+ cfg = fm10k_get_device_info(hw);
+ if (cfg == NULL)
+ return -1;
+
+ /*
+ * Assumptions:
+ * - All external interfaces are the same speed
+ * - 1G/10G ports are packed into the minimum number of EPLs
+ * - quad-mode EPLs use lane 0 as master
+ * - The lowest numbered PEPs are used
+ * - PEPs are always used in x8 mode.
+ */
+ switch (cfg->ext_port_speed) {
+ case 10:
+ pcs = FM10K_SW_EPL_PCS_SEL_10GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 1;
+ anti_bubble_wm = 5;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 12;
+ rate_fifo_fast_inc = 13;
+ break;
+ /* XXX what is the physical config for 25G? */
+ case 25:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 0;
+ anti_bubble_wm = 6;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 32;
+ rate_fifo_fast_inc = 33;
+ break;
+ case 40:
+ pcs = FM10K_SW_EPL_PCS_SEL_40GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 5;
+ rate_fifo_slow_inc = 51;
+ rate_fifo_fast_inc = 52;
+ break;
+ case 100:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 129;
+ rate_fifo_fast_inc = 130;
+ break;
+ default:
+ error = -1;
+ goto done;
+ }
+
+ /*
+ * EPL_CFG_A
+ *
+ * Mark all used lanes as active and all unused lanes as
+ * inactive. Adjust timeout and skew tolerance values for EPLs that
+ * are used.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ data = fm10k_read_switch_reg(sw, FM10K_SW_EPL_CFG_A(i));
+ data &= ~FM10K_SW_EPL_CFG_A_ACTIVE_QUAD;
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < ext_ports->num_ports; j++) {
+ port = &ext_ports->ports[j];
+ if (port->eplno == i)
+ data |= port->is_quad ?
+ FM10K_SW_EPL_CFG_A_ACTIVE_QUAD :
+ FM10K_SW_EPL_CFG_A_ACTIVE
+ (port->first_lane);
+ }
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_TIMEOUT, 19, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_SKEW_TOLERANCE, 38, data);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_A(i), data);
+ }
+
+ /*
+ * EPL_CFG_B
+ *
+ * Disable all unused lanes and configure all used ones
+ * appropriately. For EPLs used in quad port mode, the master lane
+ * is the only one that gets configured.
+ */
+ data = 0;
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++) {
+ if (j < ext_ports->ports_per_epl)
+ pcstmp = pcs;
+ else
+ pcstmp = FM10K_SW_EPL_PCS_SEL_DISABLE;
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j, pcstmp, j, j);
+ }
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE, qpl);
+ } else {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++)
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j,
+ FM10K_SW_EPL_PCS_SEL_DISABLE, j, j);
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE,
+ FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_B(i), data);
+ }
+
+ /*
+ * MAC_CFG, LINK_RULES and PCS_ML_BASER_CFG
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_read_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ /* dic enable */
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_IDLE_ENABLE_DIC,
+ dic_enable, mac_cfg);
+
+ /* ifg */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_IDLE_MIN_IFG_BYTES,
+ 12, mac_cfg);
+
+ /* tx pad size: (64 bytes + 8 preamble) / 4 */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_MIN_COLUMNS,
+ 18, mac_cfg);
+
+ /* tx clock compensation */
+ timeout =
+ fm10k_switch_ppm_to_tx_clk_compensation_timeout(pcs,
+ 100);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE,
+ (timeout > 0), mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT,
+ timeout, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_PREAMBLE_MODE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MIN_FRAME_LENGTH,
+ 64, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MAX_FRAME_LENGTH,
+ FM10K_SW_PACKET_SIZE_MAX, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_RX_IGNORE_IFG_ERRORS,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FCS_MODE,
+ FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_IEEE_1588_ENABLE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_DRAIN_MODE,
+ FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIMEOUT,
+ 100, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIME_SCALE,
+ 3, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIMEOUT,
+ 180, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIME_SCALE,
+ 1, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIMEOUT,
+ 20, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIME_SCALE,
+ 1, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LPI_AUTOMATIC,
+ 1, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LP_IDLE_REQUEST,
+ 0, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FAULT_MODE,
+ FM10K_SW_TX_MAC_FAULT_MODE_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_ANTI_BUBBLE_WATERMARK,
+ anti_bubble_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_WATERMARK,
+ rate_fifo_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_FAST_INC,
+ rate_fifo_fast_inc, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_SLOW_INC,
+ rate_fifo_slow_inc, mac_cfg);
+
+ fm10k_write_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j));
+
+ /* link-up debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_UP,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_UP,
+ 30, data);
+
+ /* link-down debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_DOWN,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_DOWN,
+ 5, data);
+
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_HEARTBEAT_TIME_SCALE,
+ cfg->ext_port_speed == 10 ? 4 : 0, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j), data);
+
+ /* XXX add 10GBASER config */
+ }
+
+ if (cfg->ext_port_speed != 10)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PCS_ML_BASER_CFG(i), 0x00003fff);
+ }
+
+
+ /*
+ * LANE_CFG, LANE_SERDES_CFG, LANE_ENERGY_DETECT_CFG,
+ * LANE_SIGNAL_DETECT_CFG, and EPL_FIFO_ERROR_STATUS
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_CFG(i, j), 0);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_CFG(i, j), 0x1106);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j));
+
+ data |=
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY;
+
+ data &=
+ ~FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j), data);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j));
+
+ data &=
+ ~(FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY);
+
+ data |=
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j), data);
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+ }
+
+ error = fm10k_epl_serdes_reset_and_load_all(sw);
+ if (error)
+ goto done;
+
+ /*
+ * EPL_FIFO_ERROR_STATUS LINK_IP
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IP(i, j), FM10K_SW_MASK32(31, 0));
+ }
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+done:
+ return (error);
+}
+
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+
+int
+fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ /* source port is external port number */
+ if (src_port < 0 || src_port == dest_port)
+ return -1;
+
+ return fm10k_ffu_mirror_set(sw, src_port, dest_port, vlan);
+}
+
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (src_port < 0)
+ return -1;
+
+ return fm10k_ffu_mirror_reset(sw, src_port);
+}
+
+
+typedef struct {
+ u8 lport;
+ u8 has_ftag;
+} fm10k_sw_lport;
+
+static int
+fm10k_switch_init(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ u32 data;
+ unsigned int num_lports = sw->info->num_peps + FM10K_SW_EPLS_SUPPORTED;
+ fm10k_sw_lport all_lports[num_lports];
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ unsigned int is_quad;
+ unsigned int table_idx;
+ int error;
+ u32 watermark;
+ u64 data64, data64_2;
+
+ /*
+ * Build list of all logical ports that might appear in the
+ * scheduler program. Note that for any particular card
+ * configuration, not all of these logical ports may be used.
+ */
+ table_idx = 0;
+ for (i = 0; i < sw->info->num_peps; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->pep_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 1;
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->epl_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 0;
+ }
+
+ if (table_idx != num_lports) {
+ FM10K_SW_ERR("fm10k switch lport table construction error");
+ return -1;
+ }
+
+ /*
+ * Reset the switch to get to the default state
+ */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+
+ usec_delay(100);
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ usec_delay(1000);
+
+ /* Clear memories */
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL);
+ fm10k_write_flush(sw);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_EPL);
+ fm10k_write_flush(sw);
+ usec_delay(800);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL, 0);
+ fm10k_write_flush(sw);
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~(FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET);
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ /* ensure switch reset is deasserted for at least 100ns */
+ usec_delay(1);
+
+ sw->epl_sbus = fm10k_sbus_attach(sw, "EPL", FM10K_SW_SBUS_EPL_CFG);
+ if (sw->epl_sbus == NULL) {
+ error = -1;
+ goto done;
+ }
+
+ /* Clear non-BIST accessible pause state memories */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 0), 0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 1), 0);
+ }
+
+ /* Initialize RXQ_MCAST list */
+ for (i = 0; i < FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(i),
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE, i) |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES));
+ }
+ /* Initialize TXQ list */
+ for (i = 0; i < FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_HEAD_PERQ_HEAD, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL0_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL0_PERQ_TAIL, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL1_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL1_PERQ_TAIL, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_TXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES));
+ }
+ /* Initialize free segment list */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_SSCHED_RX_PERPORT(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SSCHED_RX_PERPORT_NEXT, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_FREELIST_INIT_ENTRIES -
+ FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_LOGICAL_PORTS_MAX));
+ }
+ /* Disable switch scan chain */
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCAN_DATA_IN,
+ FM10K_SW_SCAN_DATA_IN_UPDATE_NODES |
+ FM10K_SW_SCAN_DATA_IN_PASSTHRU);
+
+ error = fm10k_switch_configure_epls(hw, sw);
+ if (error)
+ goto done;
+
+ /*
+ * XXX for now configure store-and-forward between PEPs and external
+ * ports regardless of relative speeds
+ */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SAF_MATRIX(all_lports[i].lport),
+ FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS);
+ }
+
+ /* Disable MTU violation trap */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SYS_CFG_1);
+ data &= ~FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS;
+ fm10k_write_switch_reg(sw, FM10K_SW_SYS_CFG_1, data);
+
+ /* Disable ingress VLAN filtering and learning */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_3(all_lports[i].lport), 0);
+ }
+ /*
+ * Make all ports members of every VLAN, and configure every VLAN to
+ * use MST instance 0.
+ */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ fm10k_write_switch_reg128(sw, FM10K_SW_INGRESS_VID_TABLE(i),
+ FM10K_SW_INGRESS_VID_TABLE_REFLECT, data64);
+ }
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_EGRESS_VID_TABLE_ENTRIES; i++)
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_EGRESS_VID_TABLE(i), 0, data64);
+
+ // Init MOD_VLAN_TAG_VID1_MAP
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ data64 = i;
+ data64 = data64 << 48;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * i + 0x20000, data64);
+ }
+
+ /* Configure MST instance 0 to forward for all ports */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_MST_TABLE_FORWARDING
+ (all_lports[i].lport);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_EGRESS_MST_TABLE(0), data64);
+
+ data64 = 0;
+ data64_2 = 0;
+ for (i = 0; i < num_lports; i++) {
+ if (all_lports[i].lport <
+ FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) {
+ data64 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ } else {
+ data64_2 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ }
+ }
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(0, 0), data64);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(1, 0), data64_2);
+
+ for (i = 0; i < num_lports; i++) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(0) |
+ FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport), data64);
+
+ /*
+ * Configure tags for f-tagged lports
+ */
+ if (all_lports[i].has_ftag) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport),
+ data64);
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_MOD_PER_PORT_CFG_2_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport),
+ data64);
+ }
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3;
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport), data64);
+ }
+
+ /*
+ * Assign default SGLORTs for the EPL ports in the PARSER config.
+ * This isn't necessary for the PEPs as for those, as all frames
+ * ingressing from PEPs are tagged with an SGLORT that is derived
+ * from a per-tx-queue setting.
+ *
+ * The PORT_CFG_ISL register offset is determined by logical port
+ * number and the register contents determined by the corresponding
+ * SGLORT.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_ISL
+ (fm10k_sw.epl_map[i].logical_port),
+ FM10K_SW_MAKE_REG_FIELD
+ (PORT_CFG_ISL_SGLORT,
+ fm10k_sw.epl_map[i].glort));
+ }
+
+ /*
+ * FFU counter, 11.10.3.5 POLICER_CFG[0..3]
+ */
+ fm10k_write_switch_reg64(sw,
+ 0xE40000 + 0x2 * FM10K_SW_FFU_CNT_BANK + 0x11000,
+ FM10K_SW_POLICER_LAST);
+
+ /*
+ * 11.10.3.1 POLICER_CFG_4K[0..1][0..4095]
+ * 11.10.3.2 POLICER_CFG_512[0..1][0..511]
+ */
+ for (i = 0; i < FM10K_SW_FFU_CNT_MAX; i++) {
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x2000 * FM10K_SW_FFU_CNT_BANK +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x0);
+ } else {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x400 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x4000);
+ }
+ }
+
+ /*
+ * FFU
+ *
+ * The FFU is programmed to match on SGLORT and to set the DGLORT to
+ * a unique value corresponding to the given SGLORT. One TCAM entry
+ * is required per host port per PEP and one per external interface.
+ * Only slice 0 is used.
+ */
+ if (fm10k_ffu_init(sw, sw->dpdk_cfg) < 0)
+ return -1;
+
+ /*
+ * Program the segment scheduler (see tables at top)
+ *
+ * The TX and RX schedules are the same. Page 0 is used.
+ */
+ if (cfg->ext_port_speed == 40 || cfg->ext_port_speed == 100)
+ is_quad = 1;
+ else
+ is_quad = 0;
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ /*
+ * In addition to explicit idle cycles, non-quad port
+ * entries are converted to idle cycles if the interfaces
+ * are configured in quad-port mode.
+ */
+ if (fm10k_sched_prog[i].idle ||
+ (!fm10k_sched_prog[i].quad && is_quad))
+ data = FM10K_SW_SCHED_SCHEDULE_IDLE;
+ else
+ data = FM10K_SW_SCHED_SCHEDULE_ENTRY
+ (fm10k_sched_prog[i].phys,
+ fm10k_sched_prog[i].log,
+ (fm10k_sched_prog[i].quad == FM10K_SW_QUAD_40_100) ?
+ is_quad : fm10k_sched_prog[i].quad);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RX_SCHEDULE(0, i), data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TX_SCHEDULE(0, i), data);
+ }
+
+ fm10k_write_switch_reg(sw, FM10K_SW_SCHED_SCHEDULE_CTRL,
+ FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_RX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1) |
+ FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_TX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1));
+
+ /* Per 5.7.10.4 */
+ watermark = FM10K_SW_MEM_POOL_SEGS_MAX -
+ ((sw->info->num_peps +
+ FM10K_SW_EPLS_SUPPORTED * FM10K_SW_EPL_LANES) *
+ FM10K_SW_HOWMANY(FM10K_SW_PACKET_SIZE_MAX,
+ FM10K_SW_MEM_POOL_SEG_SIZE, FM10K_SW_MEM_POOL_SEG_SIZE)) -
+ FM10K_SW_MEM_POOL_SEGS_RSVD;
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_WM,
+ FM10K_SW_MAKE_REG_FIELD(CM_GLOBAL_WM_WATERMARK, watermark));
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_CFG,
+ FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN |
+ FM10K_SW_MAKE_REG_FIELD
+ (CM_GLOBAL_CFG_NUM_SWEEPER_PORTS,
+ FM10K_SW_LOGICAL_PORTS_MAX));
+
+ /* Configure stats counters */
+ data = FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS;
+ data64 =
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 |
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8;
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_RX_STATS_CFG(all_lports[i].lport),
+ data |
+ ((all_lports[i].has_ftag) ?
+ FM10K_SW_MAKE_REG_FIELD
+ (RX_STATS_CFG_PER_FRAME_ADJUSTMENT,
+ FM10K_SW_FTAG_SIZE) :
+ 0));
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_STATS_CFG(all_lports[i].lport),
+ data64);
+ }
+
+ /* Transition switch to ready */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+
+ done:
+ return (error);
+}
+
+
+static void
+fm10k_switch_leds_init(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t addr;
+ uint32_t data;
+ int max;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ /* MODE1 - put into sleep mode to ensure it is brought out
+ * of sleep without violating the oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x00);
+
+ /* Wait for oscillator to stabilize after coming out of sleep */
+ fm10k_udelay(500);
+
+ /* MODE2 - group control is blinking, open drain outputs,
+ * OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x01, 0x20);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x02, 0xff);
+
+ /* PWM1 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x03, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x13, max - 1);
+
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ addr = 0x6a;
+ for (i = 0; i < 2; i++) {
+ /* MODE1 - put into sleep mode to ensure it is
+ * brought out of sleep without violating the
+ * oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x00);
+
+ /* MODE2 - group control is blinking, open drain
+ * outputs, OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x20);
+
+ /* Wait for oscillator to stabilize
+ * after coming out of sleep
+ */
+ fm10k_udelay(500);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x02, 0xff);
+
+ /* PWM3 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x05, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, addr, 0x13, max - 1);
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x88);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * PCA9538 initialization
+ * only differences from defaults are noted
+ */
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x65;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0xf0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x66;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ /* set LEC_CFG */
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+
+ /* port from rdifd, LED */
+ fm10k_gpio_output_set(sw, 4, 0);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to operate LEDs for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+
+static unsigned int
+fm10k_switch_pca9635_led_bits(uint8_t led_flags)
+{
+ unsigned int bits;
+
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE)
+ bits = 0x3; /* group blink */
+ else
+ bits = 0x1; /* full on */
+ } else {
+ bits = 0; /* off */
+ }
+ return (bits);
+}
+
+static void
+fm10k_switch_process_leds(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_ports *ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ unsigned int num_ports = ports->num_ports;
+ uint32_t data;
+ uint8_t update_port[num_ports];
+ uint8_t led_flags = 0, read;
+ uint8_t addr;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ if (sw->master_hw->sw_addr == NULL) {
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ return;
+ }
+
+ for (i = 0; i < num_ports; i++) {
+ port = &ports->ports[i];
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_EPL_LED_STATUS(port->eplno));
+ led_flags =
+ ((data & FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP
+ (port->first_lane)) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_UP : 0) |
+ ((data &
+ (FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING
+ (port->first_lane) |
+ FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING
+ (port->first_lane))) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE : 0);
+ update_port[i] = (led_flags != port->last_led_flags);
+ port->last_led_flags = led_flags;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ if (!(update_port[0] || update_port[1])) {
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (ports->ports[0].last_led_flags) |
+ (fm10k_switch_pca9635_led_bits
+ (ports->ports[1].last_led_flags) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* XXX will need to update for QSFPs operating
+ * as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (update_port[i] == 0)
+ continue;
+
+ addr = (i == 0) ? 0x6a : 0x69;
+
+ port = &ports->ports[i];
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (port->last_led_flags);
+
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ led_flags = 0;
+ addr = 0x62;
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ case 25:
+ led_flags |= 0x6 << (4 * i); /* 100G */
+ break;
+ case 40:
+ case 10:
+ led_flags |= 0x4 << (4 * i); /* 40G */
+ break;
+ default:
+ led_flags = 0;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ }
+
+ if (update_port[0] || update_port[1]) {
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ if (update_port[0])
+ led_flags |= read & 0xf0;
+ else
+ led_flags |= read & 0xf;
+ fm10k_i2c_write16(sw->i2c, addr, 0x1, led_flags);
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /* XXX will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = (i == 0) ? 0x62 : 0x66;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ led_flags = 0x08; /* 100G */
+ break;
+ case 40:
+ led_flags = 0x02; /* 40G */
+ break;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x1, led_flags);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+static void *
+fm10k_switch_leds_update(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ while (sw->detaching == 0) {
+ fm10k_switch_process_leds(ctx);
+ usec_delay(FM10K_LED_POLL_INTERVAL_MS * 1000);
+ }
+ return NULL;
+}
+
+static void
+fm10k_switch_leds_off(struct fm10k_switch *sw, struct fm10k_ext_ports *ports)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ uint8_t led_flags;
+ uint8_t addr;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits(0) |
+ (fm10k_switch_pca9635_led_bits(0) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* XXX will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ addr = 0x6a;
+ led_flags = fm10k_switch_pca9635_led_bits(0);
+ for (i = 0; i < 2; i++) {
+ port = &ports->ports[i];
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+
+static void *
+fm10k_switch_process_intr(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_ext_ports *ports;
+ uint64_t gid;
+ uint64_t reenable_mask;
+ uint16_t epl_mask;
+
+ while (1) {
+ sem_wait(&sw->intr_tq);
+ /*
+ * Mask off all global interrupt detect interrupts
+ * toward PCIe to ensure that the PEP sees the
+ * interrupt condition clear so that all subsequent
+ * events will be recognized. The same result could
+ * be achieved by masking off all block-internal
+ * interrupt sources in all blocks prior to handling
+ * any global interrupt detect interrupts, but the
+ * approach taken here makes the interrupt processing
+ * easier to decompose and increases the likelihood that
+ * more work will be coalesced into fewer interrupts.
+ */
+ FM10K_SW_TRACE("masking off PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ /*
+ * sw->ext_ports will be set to NULL during detach
+ * to indicate that we are to no longer process EPL
+ * interrupts.
+ */
+ ports = sw->ext_ports;
+ /*
+ * sw->sm_mailbox_enabled will be set to 0 during
+ * detach to indicate that we are to no longer process
+ * PEP interrupts.
+ */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ FM10K_SW_INTERRUPT_MASK_PCIE_ALL);
+ gid = fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ reenable_mask = 0;
+ if (ports) {
+ epl_mask = fm10k_ext_ports_epl_intrs(ports, gid);
+ reenable_mask |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (INTERRUPT_MASK_PCIE_EPL, epl_mask);
+ }
+
+ if (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C)
+ fm10k_i2c_intr(sw->i2c);
+
+ reenable_mask |= FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+
+ FM10K_SW_TRACE("re-enabling PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ ~reenable_mask);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ }
+ return NULL;
+}
+
+void
+fm10k_switch_intr(struct fm10k_hw *hw)
+{
+ if (hw)
+ sem_post(&fm10k_sw.intr_tq);
+}
+
+
+static void
+fm10k_switch_enable_interrupts(struct fm10k_switch *sw)
+{
+ uint64_t data64;
+ unsigned int i;
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno));
+
+ /* enable interrupts from all EPLs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_EPL,
+ ~sw->ext_ports->epl_mask, data64);
+
+ /* enable interrupts from all non-master PEPs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_PCIE,
+ ~sw->pep_mask, data64);
+
+ /* enable I2C interrupt */
+ data64 &= ~FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno), data64);
+
+ /* enable outbound mailbox interrupts from all non-master PEPs */
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (sw->pep_mask & (1 << i))
+ fm10k_write_switch_reg(sw, FM10K_SW_PCIE_GLOBAL(i, IM),
+ ~FM10K_SW_IM_MAILBOX);
+ }
+ FM10K_WRITE_REG(sw->master_hw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_FIELD(SWITCH_INT, ENABLE));
+}
+
+
+static void
+fm10k_switch_detach(struct fm10k_hw *hw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_switch *sw = &fm10k_sw;
+
+ if (hw == NULL || sw == NULL)
+ return;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->detaching = 1;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (sw->ext_ports) {
+ ports = sw->ext_ports;
+ /*
+ * Ensure that any further switch interrupts will not
+ * process sw->ext_ports before detaching them.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->ext_ports = NULL;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_switch_leds_off(sw, ports);
+ fm10k_ext_ports_detach(ports);
+ }
+
+ if (sw->epl_sbus)
+ fm10k_sbus_detach(sw->epl_sbus);
+
+ /*
+ * Detach i2c after ext_ports so ext_ports can use i2c to disable
+ * phys.
+ */
+ if (sw->i2c)
+ fm10k_i2c_detach(sw->i2c);
+
+ fm10k_hw_eicr_disable_source(sw, FM10K_SW_EICR_SWITCH_INT);
+}
+
+
+static unsigned int
+fm10k_switch_get_fabric_clock(struct fm10k_switch *sw)
+{
+ uint32_t pll_fabric;
+ unsigned int freq;
+
+ pll_fabric = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_LOCK);
+
+ switch (FM10K_SW_REG_FIELD(pll_fabric, PLL_FABRIC_FREQSEL)) {
+ case FM10K_SW_PLL_FABRIC_FREQSEL_CTRL: {
+ uint32_t ctrl;
+ unsigned int refdiv, fbdiv4, fbdiv255;
+
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_CTRL);
+ refdiv = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_REFDIV);
+ fbdiv4 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV4);
+ fbdiv255 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV255);
+
+ freq = (15625 * 4 * fbdiv255 * (1 + fbdiv4)) / (refdiv * 100);
+ break;
+ }
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F600:
+ freq = 600;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F500:
+ freq = 500;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F400:
+ freq = 400;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F300:
+ freq = 300;
+ break;
+ default:
+ freq = 0;
+ break;
+ }
+
+ return freq;
+}
+
+static unsigned int
+fm10k_switch_get_sku(struct fm10k_switch *sw)
+{
+ uint32_t fuse_data;
+
+ fuse_data = fm10k_read_switch_reg(sw, FM10K_SW_FUSE_DATA_0);
+ return FM10K_SW_REG_FIELD(fuse_data, FUSE_SKU);
+}
+
+static const char *
+fm10k_switch_get_sku_name(unsigned int sku)
+{
+ const char *name;
+
+ switch (sku) {
+ case FM10K_SW_FUSE_SKU_FM10840:
+ name = "FM10840";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10420:
+ name = "FM10420";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10064:
+ name = "FM10064";
+ break;
+ default:
+ name = "FM10xxx-unknown";
+ break;
+ }
+
+ return name;
+}
+
+
+static void
+fm10k_switch_describe(struct fm10k_switch *sw)
+{
+ unsigned int i, pair;
+ unsigned int reset_state, active_state;
+ uint32_t device_cfg, resets;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("switch: device is %s, fabric clock %u MHz",
+ fm10k_switch_get_sku_name(fm10k_switch_get_sku(sw)),
+ fm10k_switch_get_fabric_clock(sw));
+
+ device_cfg = fm10k_read_switch_reg(sw, FM10K_SW_DEVICE_CFG);
+ FM10K_SW_INFO("switch: 100G Ethernet is %s",
+ (device_cfg & FM10K_SW_DEVICE_CFG_PCIE_100G_DIS) ?
+ "disabled" : "enabled");
+ switch (FM10K_SW_REG_FIELD(device_cfg, DEVICE_CFG_FEATURE)) {
+ case FM10K_SW_DEVICE_CFG_PCIE_FULL:
+ FM10K_SW_INFO("switch: PEPs 0-7 support 2x4 and 1x8 "
+ "modes, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_HALF:
+ FM10K_SW_INFO("switch: PEPs 0-3 support 2x4 and 1x8 "
+ "modes, PEPs 4 and 6 are 1x4, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_BASIC:
+ FM10K_SW_INFO("switch: PEP 0 supports, PEP 8 is x1, "
+ "only one can be used");
+ break;
+ }
+
+ resets = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (!(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_EN(i)))
+ continue;
+
+ pair = i / 2;
+ reset_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_RESET(i));
+ active_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_ACTIVE(i));
+ if (pair < FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS &&
+ !(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(pair))) {
+ if (i % 2 == 0)
+ FM10K_SW_INFO("switch: PEP[%u,%u] is enabled in 1x8 "
+ "mode (Reset=%u, Active=%u)", i, i + 1,
+ reset_state, active_state);
+ else
+ FM10K_SW_INFO("switch: PEP[%u] unexpectedly enabled when "
+ "PEP[%u,%u] is in 1x8 mode (Reset=%u, "
+ "Active=%u)", i, i - 1, i, reset_state,
+ active_state);
+ } else {
+ FM10K_SW_INFO("switch: PEP[%u] is enabled in 1x4 "
+ "mode (Reset=%u, Active=%u)", i, reset_state,
+ active_state);
+ }
+ }
+}
+
+
+static struct fm10k_switch *
+fm10k_switch_attach(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ int i;
+ int error = 0;
+
+ /*
+ * XXX apparently no way to determine one's own PEP number.
+ */
+ switch (sw->info->num_peps) {
+ case 1:
+ sw->pepno = 0;
+ break;
+
+ case 2:
+ /*
+ * XXX assumption is using even numbered PEPs
+ * starting with 0, and the highest numbered PEP
+ * is the master
+ */
+ sw->pepno = 2;
+ for (i = 0; i < sw->info->num_peps - 1; i++)
+ sw->pep_mask |= (1 << (i * 2));
+ break;
+
+ case 4:
+ sw->pepno = 6;
+ sw->pep_mask = 0x7f;
+ break;
+
+ default:
+ sw->pepno = 0;
+ }
+
+ /* When not zero, initialize serdes in 'near loopback' mode */
+ sw->serdes_loopback = 0;
+
+ pthread_mutex_init(&sw->lock, NULL);
+
+ fm10k_switch_determine_epls(sw);
+
+ sw->ext_ports = fm10k_ext_ports_attach(sw);
+ if (sw->ext_ports == NULL)
+ goto fail;
+
+ /* label all external ports with their logical port numbers */
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ sw->ext_ports->ports[i].lport = sw->epl_map[i].logical_port;
+
+ /* interrupt */
+ sem_init(&sw->intr_tq, 0, 0);
+ pthread_create(&sw->intr_task, NULL, fm10k_switch_process_intr, sw);
+
+ fm10k_switch_enable_interrupts(sw);
+ fm10k_switch_describe(sw);
+
+ sw->i2c = fm10k_i2c_attach(sw);
+ if (sw->i2c == NULL)
+ goto fail;
+
+ error = fm10k_switch_init(hw, sw);
+ if (error)
+ goto fail;
+
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ fm10k_ext_port_up(&sw->ext_ports->ports[i]);
+
+ usec_delay(10000);
+ fm10k_switch_leds_init(sw);
+
+ pthread_create(&sw->led_task, NULL, &fm10k_switch_leds_update, sw);
+ pthread_create(&sw->stats_task, NULL, &fm10k_switch_process_stats, sw);
+ sw->inited = 1;
+ return sw;
+fail:
+ if (sw != NULL)
+ fm10k_switch_detach(hw);
+ return NULL;
+}
+
+struct fm10k_switch*
+fm10k_switch_get(void)
+{
+ return &fm10k_sw;
+}
+
+static int
+fm10k_switch_dpdk_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].pf_no == pf_no)
+ return i;
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_mapped_ext_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ if (map->map_no[0] == pf_no)
+ return i;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ if (map->map_no[0] == pf_no || map->map_no[1] == pf_no)
+ return i;
+ }
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_start(struct fm10k_switch *sw,
+ struct fm10k_hw *master_hw, eth_fm10k_dev_init_half_func *func)
+{
+ int i, j;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_cfg_port_pf_map *dpdk_map;
+ int dpdk_port_no, ext_port_no = 0;
+ int pf_no;
+
+ sw->info = fm10k_get_device_info(master_hw);
+ sw->master_hw = master_hw;
+ sw->pep_map = fm10k_pep_port_map;
+ sw->epl_map = fm10k_epl_port_map;
+
+ sw->info->ext_port_speed = sw->dpdk_cfg->ext_port_speed;
+ fm10k_switch_set_sched_prog();
+ fm10k_switch_attach(master_hw, &fm10k_sw);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type != FM10K_CONFIG_DPDK_PF ||
+ map->type != FM10K_CONFIG_PORT_MAP_PFS)
+ continue;
+
+ for (j = 0; j < 2; j++) {
+ pf_no = map->map_no[j];
+ dpdk_port_no =
+ fm10k_switch_dpdk_port_get(sw, pf_no);
+ dpdk_map =
+ &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ dpdk_map->type = FM10K_CONFIG_PORT_MAP_PFSS;
+ dpdk_map->map_no[0] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ dpdk_map->map_no[1] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1];
+ }
+ }
+ }
+
+ /* do initialize all ports, after switch is ready */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ uint32_t sglort;
+ struct fm10k_dpdk_port *port = &sw->dpdk_cfg->ports[i];
+ struct fm10k_hw *hw = port->hw;
+
+ if (hw == NULL)
+ break;
+
+ map = &sw->dpdk_cfg->ext_port_map[ext_port_no];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_pf_glort_get(pf_no);
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ ext_port_no =
+ fm10k_switch_mapped_ext_port_get
+ (sw, pf_no);
+ sglort =
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]);
+ } else {
+ FM10K_SW_ERR("Unknown mapped port type %d!",
+ port->type);
+ return -1;
+ }
+ hw->mac.dglort_map = sglort | 0xffff0000;
+ }
+ func(hw);
+ }
+ return 0;
+}
+
+
+static int
+fm10k_switch_dpdk_port_reg(struct fm10k_switch *sw,
+ struct fm10k_hw *hw, void *rte_dev, uint8_t is_pf, bool master)
+{
+ int i;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg == NULL && is_pf) {
+ if (fm10k_config_init(sw, hw) < 0)
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ if (master && port->hw &&
+ port->type == FM10K_CONFIG_DPDK_PF) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (hw->mac.addr[5] - port->hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ if (port->hw == NULL) {
+ port->hw = hw;
+ port->rte_dev = rte_dev;
+ fm10k_flow_list_init(&port->flow_list);
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (is_pf) {
+ sw->dpdk_cfg->pf_bind++;
+ port->type = FM10K_CONFIG_DPDK_PF;
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ } else {
+ port->type = FM10K_CONFIG_DPDK_VF;
+ }
+ if (master)
+ sw->dpdk_cfg->master_hw = hw;
+ if (sw->dpdk_cfg->master_hw) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (sw->dpdk_cfg->master_hw->mac.addr[5] -
+ hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ return 0;
+ }
+ }
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == NULL)
+ break;
+
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == hw)
+ return i;
+ }
+ return -1;
+}
+
+void *
+fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].rte_dev;
+}
+
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].flow_list;
+}
+
+static int
+fm10k_switch_dpdk_cfg_check(struct fm10k_switch *sw)
+{
+ int i;
+ bool need_default = true;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg->master_hw == NULL) {
+ FM10K_SW_ERR("Master PF is not bound!!!");
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF &&
+ map->type != FM10K_CONFIG_PORT_MAP_NULL) {
+ need_default = false;
+ break;
+ }
+ }
+
+ if (need_default) {
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ }
+ }
+
+ return 0;
+}
+
+
+static void
+fm10k_switch_dpdk_cfg_describe(struct fm10k_switch *sw)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ printf("--- FM10K DYNAMIC CONFIG ---\n");
+ printf(" PF Bind : %d\n", sw->dpdk_cfg->pf_bind);
+
+ printf("--- PF ---\n");
+ for (i = 0; i < FM10K_SW_PEP_PORTS_MAX; i++) {
+ uint8_t *mac;
+ struct rte_eth_dev *dev;
+ struct rte_pci_device *pdev;
+
+ if (sw->dpdk_cfg->pf_hw[i] == NULL)
+ continue;
+ dev = (struct rte_eth_dev *)
+ fm10k_switch_dpdk_port_rte_dev_get
+ (sw->dpdk_cfg->pf_hw[i]);
+ pdev = RTE_ETH_DEV_TO_PCI(dev);
+ mac = sw->dpdk_cfg->pf_hw[i]->mac.addr;
+ printf(" PF%d : Logical %d Glort %#x "
+ "PCI Addr %02x:%02x.%x "
+ "MAC Addr %02x:%02x:%02x:%02x:%02x:%02x\n",
+ i, fm10k_pep_port_map[i].logical_port,
+ fm10k_switch_pf_glort_get(i), pdev->addr.bus,
+ pdev->addr.devid,
+ pdev->addr.function,
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+ }
+
+ printf("--- DPDK PORT ---\n");
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].type == FM10K_CONFIG_DPDK_NULL)
+ break;
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ break;
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ printf(" DPDK PORT%d : Map to PF%d\n",
+ i, map->map_no[0]);
+ } else {
+ printf(" DPDK PORT%d : Map to PF%d&PF%d Glort %#x\n",
+ i,
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+ }
+
+ printf("--- EXT PORT ---\n");
+ for (i = 0; i < sw->dpdk_cfg->ext_port_num; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ printf(" EXT PORT%d : Logical %d Glort %#x", i,
+ fm10k_epl_port_map[i].logical_port,
+ fm10k_switch_epl_glort_get(i));
+
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ printf("\n");
+ else if (map->type == FM10K_CONFIG_PORT_MAP_PF)
+ printf(" Map to PF%d\n", map->map_no[0]);
+ else
+ printf(" Map to PF%d&PF%d Glort %#x\n",
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+}
+
+
+int
+fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func)
+{
+ int ret;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (fm10k_switch_dpdk_port_reg(sw, hw, rte_dev, is_pf, master) != 0) {
+ FM10K_SW_ERR("Register ports failed!!!");
+ return -1;
+ }
+
+ /*
+ * After all pfs are started
+ * start switch here
+ */
+ if (fm10k_sw.dpdk_cfg->pf_max != 0 &&
+ fm10k_sw.dpdk_cfg->pf_bind == fm10k_sw.dpdk_cfg->pf_num) {
+ if (fm10k_switch_dpdk_cfg_check(&fm10k_sw) != 0)
+ return -1;
+
+ ret = fm10k_switch_start(&fm10k_sw,
+ fm10k_sw.dpdk_cfg->master_hw, func);
+ fm10k_switch_dpdk_cfg_describe(&fm10k_sw);
+ return ret;
+ }
+ return 0;
+}
+
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw)
+{
+ if (hw)
+ return;
+}
+
+
+/*
+ * for multi-host, only dpdk port 0 and 1 are real rx/tx packets
+ * so we need init/setup/start dpdk port queue for them.
+ * for dpdk port 2/3, we need init/setup except start.
+ * we map the pf queue to 0/1 dpdk queue first, then map
+ * the other pf queue to 2/3 dpdk queue.
+ */
+int
+fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue)
+{
+ int idx, pf_no;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ FM10K_SW_ERR("Not support yet!!!");
+ return -1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[0];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue / 2;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = (max_queue + 1) / 2 + queue / 2;
+ return 0;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_NULL) {
+ FM10K_SW_ERR("Unmapped dpdk port %d!!!", dpdk_port_no);
+ return -1;
+ }
+ FM10K_SW_ERR("Unknown mapped type!!!");
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw, struct fm10k_hw *hw_list[])
+{
+ int pf_no, pf_no_ext;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+ map = &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ hw_list[0] = hw;
+ hw_list[1] = NULL;
+ return 1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ hw_list[1] = NULL;
+ return 1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 2;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFSS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 0;
+ }
+
+ hw_list[0] = NULL;
+ hw_list[1] = NULL;
+ return 0;
+}
+
+void
+fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].tx_queue_num = num;
+}
+
+void
+fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].rx_queue_num = num;
+}
+
+int
+fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ return sw->dpdk_cfg->ports[port_no].pf_no;
+}
+
+
+void
+fm10k_switch_flowset_switchto(const char *name)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_ffu_flowset_switch(sw, name);
+}
+
+void
+fm10k_switch_show_port(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
+}
+
+void
+fm10k_switch_show_ffu(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_ffu_count_print(sw);
+}
+
+void
+fm10k_switch_show_bank(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_port_bank_print(sw);
+}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 5/7] net/fm10k: add switch initialization
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
` (3 preceding siblings ...)
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 4/7] net/fm10k: add flow and switch management Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 6/7] net/fm10k: add mirror and filter ctrl Xiaojun Liu
` (2 subsequent siblings)
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
Modify fm10k/fm10k_ethdev.c.
Split dev init to 2 parts.
First only register the port in switch
management; second init hook will be
called after all the pf are registered
and switch initialization. It will finish
dev init. Also add switch interrupt support.
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/fm10k_ethdev.c | 182 +++++++++++++++++++++++++++++++++++++--
1 file changed, 176 insertions(+), 6 deletions(-)
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 99c4366..4c81952 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -13,6 +13,10 @@
#include "fm10k.h"
#include "base/fm10k_api.h"
+#ifdef ENABLE_FM10K_MANAGEMENT
+#include "switch/fm10k_regs.h"
+#include "switch/fm10k_switch.h"
+#endif
/* Default delay to acquire mailbox lock */
#define FM10K_MBXLOCK_DELAY_US 20
@@ -39,6 +43,10 @@
#define GLORT_PF_MASK 0xFFC0
#define GLORT_FD_MASK GLORT_PF_MASK
#define GLORT_FD_INDEX GLORT_FD_Q_BASE
+#ifdef ENABLE_FM10K_MANAGEMENT
+/* When the switch is ready, the status will be changed */
+static int fm10k_switch_ready;
+#endif
int fm10k_logtype_init;
int fm10k_logtype_driver;
@@ -2588,6 +2596,9 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
int status_mbx;
s32 err;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ uint32_t writeback = 0;
+#endif
if (hw->mac.type != fm10k_mac_pf)
return;
@@ -2601,11 +2612,20 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle switch up/down */
- if (cause & FM10K_EICR_SWITCHNOTREADY)
- PMD_INIT_LOG(ERR, "INT: Switch is not ready");
+ if (cause & FM10K_EICR_SWITCHNOTREADY) {
+ PMD_INIT_LOG(INFO, "INT: Switch is not ready");
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_ready = 0;
+ writeback |= FM10K_EICR_SWITCHNOTREADY;
+#endif
+ }
if (cause & FM10K_EICR_SWITCHREADY) {
PMD_INIT_LOG(INFO, "INT: Switch is ready");
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_ready = 1;
+ writeback |= FM10K_EICR_SWITCHREADY;
+#endif
if (dev_info->sm_down == 1) {
fm10k_mbx_lock(hw);
@@ -2656,6 +2676,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle mailbox message */
+#ifndef ENABLE_FM10K_MANAGEMENT
fm10k_mbx_lock(hw);
err = hw->mbx.ops.process(hw, &hw->mbx);
fm10k_mbx_unlock(hw);
@@ -2663,10 +2684,33 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
if (err == FM10K_ERR_RESET_REQUESTED) {
PMD_INIT_LOG(INFO, "INT: Switch is down");
dev_info->sm_down = 1;
- _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
NULL);
}
+#else
+ if (cause & FM10K_EICR_MAILBOX) {
+ fm10k_mbx_lock(hw);
+ err = hw->mbx.ops.process(hw, &hw->mbx);
+ fm10k_mbx_unlock(hw);
+ writeback |= FM10K_EICR_MAILBOX;
+ if (err == FM10K_ERR_RESET_REQUESTED) {
+ PMD_INIT_LOG(INFO, "INT: Switch is down");
+ dev_info->sm_down = 1;
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
+ NULL);
+ }
+ }
+
+ /* Handle switch interrupt */
+ if (cause & FM10K_SW_EICR_SWITCH_INT)
+ fm10k_switch_intr(hw);
+#endif
+
/* Handle SRAM error */
if (cause & FM10K_EICR_SRAMERROR) {
PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
@@ -2678,15 +2722,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
/* Todo: print out error message after shared code updates */
}
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Clear these 3 events if having any */
cause &= FM10K_EICR_SWITCHNOTREADY | FM10K_EICR_MAILBOX |
FM10K_EICR_SWITCHREADY;
if (cause)
FM10K_WRITE_REG(hw, FM10K_EICR, cause);
+#else
+ if (writeback)
+ FM10K_WRITE_REG(hw, FM10K_EICR, writeback);
+#endif
/* Re-enable interrupt from device side */
- FM10K_WRITE_REG(hw, FM10K_ITR(0), FM10K_ITR_AUTOMASK |
+#ifndef ENABLE_FM10K_MANAGEMENT
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_ITR_AUTOMASK |
FM10K_ITR_MASK_CLEAR);
+#else
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_SW_ITR_AUTO_MASK |
+ FM10K_SW_MAKE_REG_FIELD(ITR_MASK, FM10K_SW_ITR_MASK_W_ENABLE));
+#endif
/* Re-enable interrupt from host side */
rte_intr_ack(dev->intr_handle);
}
@@ -3071,13 +3127,87 @@ static void __attribute__((cold))
info->sm_down = false;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+static int eth_fm10k_dev_init_hook(struct fm10k_hw *hw)
+{
+ int i, switch_ready;
+ struct rte_eth_dev *dev =
+ (struct rte_eth_dev *)fm10k_switch_dpdk_port_rte_dev_get(hw);
+
+ /* Make sure Switch Manager is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ switch_ready = 0;
+
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ fm10k_mbx_lock(hw);
+ switch_ready = fm10k_switch_ready;
+ fm10k_mbx_unlock(hw);
+ if (switch_ready)
+ break;
+ /* Delay some time to acquire async LPORT_MAP info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (switch_ready == 0) {
+ PMD_INIT_LOG(ERR, "switch is not ready");
+ return -1;
+ }
+ }
+
+ /*
+ * Below function will trigger operations on mailbox, acquire lock to
+ * avoid race condition from interrupt handler. Operations on mailbox
+ * FIFO will trigger interrupt to PF/SM, in which interrupt handler
+ * will handle and generate an interrupt to our side. Then, FIFO in
+ * mailbox will be touched.
+ */
+ if (hw->mac.dglort_map == FM10K_DGLORTMAP_NONE) {
+ PMD_INIT_LOG(ERR, "dglort_map is not ready");
+ return -1;
+ }
+
+ fm10k_mbx_lock(hw);
+ /* Enable port first */
+ hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+ MAX_LPORT_NUM, 1);
+ /* Set unicast mode by default. App can change to other mode in other
+ * API func.
+ */
+ hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+ FM10K_XCAST_MODE_NONE);
+ fm10k_mbx_unlock(hw);
+
+ /* Make sure default VID is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (hw->mac.default_vid)
+ break;
+ /* Delay some time to acquire async port VLAN info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (hw->mac.default_vid == 0)
+ hw->mac.default_vid = 1;
+ }
+
+ /* Add default mac address */
+ fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+ MAIN_VSI_POOL_NUMBER);
+
+ return 0;
+}
+#endif
+
static int
eth_fm10k_dev_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
- int diag, i;
+ int diag;
+#ifndef ENABLE_FM10K_MANAGEMENT
+ int i;
+#endif
struct fm10k_macvlan_filter_info *macvlan;
PMD_INIT_FUNC_TRACE();
@@ -3114,7 +3244,9 @@ static void __attribute__((cold))
" Try to blacklist unused devices.");
return -EIO;
}
-
+#ifdef ENABLE_FM10K_MANAGEMENT
+ hw->sw_addr = (void *)pdev->mem_resource[4].addr;
+#endif
/* Store fm10k_adapter pointer */
hw->back = dev->data->dev_private;
@@ -3125,6 +3257,25 @@ static void __attribute__((cold))
return -EIO;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (hw->mac.type == fm10k_mac_pf) {
+ if (hw->hw_addr == NULL || hw->sw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ } else {
+ if (hw->hw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ }
+
+ /* Store fm10k_adapter pointer */
+ hw->back = dev->data->dev_private;
+#endif
+
/* Initialize parameters */
fm10k_params_init(dev);
@@ -3205,6 +3356,7 @@ static void __attribute__((cold))
hw->mac.ops.update_int_moderator(hw);
/* Make sure Switch Manager is ready before going forward. */
+#ifndef ENABLE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf) {
int switch_ready = 0;
@@ -3264,11 +3416,25 @@ static void __attribute__((cold))
MAIN_VSI_POOL_NUMBER);
return 0;
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ bool master = FM10K_READ_REG(hw,
+ FM10K_CTRL) & FM10K_CTRL_BAR4_ALLOWED;
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 1, master, eth_fm10k_dev_init_hook);
+ } else { /* It may not work for VF */
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 0, false, eth_fm10k_dev_init_hook);
+ }
+#endif
}
static int
eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
{
+#ifdef ENABLE_FM10K_MANAGEMENT
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
PMD_INIT_FUNC_TRACE();
/* only uninitialize in the primary process */
@@ -3278,6 +3444,10 @@ static void __attribute__((cold))
/* safe to close dev here */
fm10k_dev_close(dev);
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_port_stop(hw);
+#endif
+
return 0;
}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 6/7] net/fm10k: add mirror and filter ctrl
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
` (4 preceding siblings ...)
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 5/7] net/fm10k: add switch initialization Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 7/7] net/fm10k: add dpdk port mapping Xiaojun Liu
2020-01-21 2:53 ` [dpdk-dev] [PATCH v2 0/7] support switch management Wang, Xiao W
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
Modify fm10k/fm10k_ethdev.c.
Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
to support mirror operation.
Add fm10k_dev_filter_ctrl to
support flow operation.
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/fm10k_ethdev.c | 76 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 76 insertions(+)
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 4c81952..1c01684 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2288,6 +2288,77 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
return 0;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+static int
+fm10k_mirror_rule_set(struct rte_eth_dev *dev,
+ struct rte_eth_mirror_conf *mirror_conf,
+ uint8_t sw_id, uint8_t on)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG,
+ "Mirror set, switch %d to port %d attach vlan %d on %d",
+ sw_id, mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0], on);
+
+ if (on) {
+ if (fm10k_switch_mirror_set(hw,
+ mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0]) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ } else {
+ if (fm10k_switch_mirror_reset(hw) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t sw_id)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG, "Mirror reset, switch %d", sw_id);
+
+ fm10k_switch_mirror_reset(hw);
+
+ return 0;
+}
+
+static int
+fm10k_dev_filter_ctrl(struct rte_eth_dev *dev,
+ enum rte_filter_type filter_type,
+ enum rte_filter_op filter_op,
+ void *arg)
+{
+ int ret = 0;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ switch (filter_type) {
+ case RTE_ETH_FILTER_GENERIC:
+ if (filter_op != RTE_ETH_FILTER_GET)
+ return -EINVAL;
+ *(const void **)arg = fm10k_flow_ops_get();
+ break;
+ default:
+ PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
+ filter_type);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+#endif
+
static void
fm10k_dev_enable_intr_pf(struct rte_eth_dev *dev)
{
@@ -2949,6 +3020,11 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
.reta_query = fm10k_reta_query,
.rss_hash_update = fm10k_rss_hash_update,
.rss_hash_conf_get = fm10k_rss_hash_conf_get,
+#ifdef ENABLE_FM10K_MANAGEMENT
+ .mirror_rule_set = fm10k_mirror_rule_set,
+ .mirror_rule_reset = fm10k_mirror_rule_reset,
+ .filter_ctrl = fm10k_dev_filter_ctrl,
+#endif
};
static int ftag_check_handler(__rte_unused const char *key,
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 7/7] net/fm10k: add dpdk port mapping
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
` (5 preceding siblings ...)
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 6/7] net/fm10k: add mirror and filter ctrl Xiaojun Liu
@ 2019-12-11 9:52 ` Xiaojun Liu
2020-01-21 2:53 ` [dpdk-dev] [PATCH v2 0/7] support switch management Wang, Xiao W
7 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-11 9:52 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal, jacob.e.keller
Cc: dev, Xiaojun Liu
Modify fm10k/fm10k_ethdev.c.
Add dpdk port and pf mapping, so
the dpdk port can map to a specific pf
and 1 dpdk port can map to 2 pf to get
total 100G throughput.
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
To enable switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/fm10k_ethdev.c | 322 +++++++++++++++++++++++++++++++++++----
1 file changed, 294 insertions(+), 28 deletions(-)
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 1c01684..8af97a7 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -517,6 +517,15 @@ struct fm10k_xstats_name_off {
struct rte_eth_conf *dev_conf = &dev->data->dev_conf;
uint32_t mrqc, *key, i, reta, j;
uint64_t hf;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ uint16_t nb_rx_queues = dev->data->nb_rx_queues;
+ int mapped_num;
+ struct fm10k_hw *mapped_hws[2];
+
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num == 2)
+ nb_rx_queues /= 2;
+#endif
#define RSS_KEY_SIZE 40
static uint8_t rss_intel_key[RSS_KEY_SIZE] = {
@@ -646,27 +655,48 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_tx_init(struct rte_eth_dev *dev)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ uint32_t data;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_tx_queue *txq;
uint64_t base_addr;
uint32_t size;
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Disable TXINT to avoid possible interrupt */
for (i = 0; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_TXINT(i),
3 << FM10K_TXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_tx_queue_num_set(unmap_hw,
+ dev->data->nb_tx_queues);
+#endif
/* Setup TX queue */
for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id);
+#endif
txq = dev->data->tx_queues[i];
base_addr = txq->hw_ring_phys_addr;
size = txq->nb_desc * sizeof(struct fm10k_tx_desc);
/* disable queue to avoid issues while updating state */
- ret = tx_queue_disable(hw, i);
+ ret = tx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Enable use of FTAG bit in TX descriptor, PFVTCTL
@@ -674,7 +704,7 @@ struct fm10k_xstats_name_off {
*/
if (fm10k_check_ftag(dev->device->devargs)) {
if (hw->mac.type == fm10k_mac_pf) {
- FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_PFVTCTL(hw_queue_id),
FM10K_PFVTCTL_FTAG_DESC_ENABLE);
PMD_INIT_LOG(DEBUG, "FTAG mode is enabled");
} else {
@@ -684,15 +714,25 @@ struct fm10k_xstats_name_off {
}
/* set location and size for descriptor ring */
- FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_TDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_TDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_TDLEN(hw_queue_id), size);
/* assign default SGLORT for each TX queue by PF */
+#ifndef ENABLE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf)
- FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(i), hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw,
+ FM10K_TX_SGLORT(hw_queue_id),
+ hw->mac.dglort_map);
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ data = FM10K_SW_MAKE_REG_FIELD
+ (TX_SGLORT_SGLORT, hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(hw_queue_id), data);
+ }
+#endif
}
/* set up vector or scalar TX function as appropriate */
@@ -704,19 +744,27 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_rx_init(struct rte_eth_dev *dev)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct fm10k_macvlan_filter_info *macvlan;
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+ uint32_t logic_port = hw->mac.dglort_map;
+ uint16_t queue_stride = 0;
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_rx_queue *rxq;
uint64_t base_addr;
uint32_t size;
uint32_t rxdctl = FM10K_RXDCTL_WRITE_BACK_MIN_DELAY;
- uint32_t logic_port = hw->mac.dglort_map;
uint16_t buf_size;
- uint16_t queue_stride = 0;
+#ifndef ENABLE_FM10K_MANAGEMENT
/* enable RXINT for interrupt mode */
i = 0;
if (rte_intr_dp_is_en(intr_handle)) {
@@ -736,26 +784,36 @@ struct fm10k_xstats_name_off {
for (; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_RXINT(i),
3 << FM10K_RXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_rx_queue_num_set(unmap_hw, dev->data->nb_rx_queues);
+#endif
/* Setup RX queues */
for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_rx_queues, &hw, &hw_queue_id);
+#endif
rxq = dev->data->rx_queues[i];
base_addr = rxq->hw_ring_phys_addr;
size = rxq->nb_desc * sizeof(union fm10k_rx_desc);
/* disable queue to avoid issues while updating state */
- ret = rx_queue_disable(hw, i);
+ ret = rx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Setup the Base and Length of the Rx Descriptor Ring */
- FM10K_WRITE_REG(hw, FM10K_RDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_RDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_RDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_RDLEN(hw_queue_id), size);
/* Configure the Rx buffer size for one buff without split */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
@@ -769,7 +827,7 @@ struct fm10k_xstats_name_off {
*/
buf_size -= FM10K_RX_DATABUF_ALIGN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id),
(buf_size >> FM10K_SRRCTL_BSIZEPKT_SHIFT) |
FM10K_SRRCTL_LOOPBACK_SUPPRESS);
@@ -779,9 +837,9 @@ struct fm10k_xstats_name_off {
rxq->offloads & DEV_RX_OFFLOAD_SCATTER) {
uint32_t reg;
dev->data->scattered_rx = 1;
- reg = FM10K_READ_REG(hw, FM10K_SRRCTL(i));
+ reg = FM10K_READ_REG(hw, FM10K_SRRCTL(hw_queue_id));
reg |= FM10K_SRRCTL_BUFFER_CHAINING_EN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i), reg);
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id), reg);
}
/* Enable drop on empty, it's RO for VF */
@@ -801,6 +859,7 @@ struct fm10k_xstats_name_off {
/* update RX_SGLORT for loopback suppress*/
if (hw->mac.type != fm10k_mac_pf)
return 0;
+#ifndef ENABLE_FM10K_MANAGEMENT
macvlan = FM10K_DEV_PRIVATE_TO_MACVLAN(dev->data->dev_private);
if (macvlan->nb_queue_pools)
queue_stride = dev->data->nb_rx_queues / macvlan->nb_queue_pools;
@@ -809,6 +868,7 @@ struct fm10k_xstats_name_off {
logic_port++;
FM10K_WRITE_REG(hw, FM10K_RX_SGLORT(i), logic_port);
}
+#endif
return 0;
}
@@ -816,13 +876,31 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret;
+#endif
int err;
uint32_t reg;
struct fm10k_rx_queue *rxq;
+ uint16_t hw_queue_id = rx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ rx_queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id);
+ if (ret < 0)
+ return -EIO;
+ else if (ret != 1) /* reference port's queue don't need start */
+ return 0;
+#endif
+
rxq = dev->data->rx_queues[rx_queue_id];
err = rx_queue_reset(rxq);
if (err == -ENOMEM) {
@@ -841,23 +919,23 @@ struct fm10k_xstats_name_off {
* this comment and the following two register writes when the
* emulation platform is no longer being used.
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
/* Set PF ownership flag for PF devices */
- reg = FM10K_READ_REG(hw, FM10K_RXQCTL(rx_queue_id));
+ reg = FM10K_READ_REG(hw, FM10K_RXQCTL(hw_queue_id));
if (hw->mac.type == fm10k_mac_pf)
reg |= FM10K_RXQCTL_PF;
reg |= FM10K_RXQCTL_ENABLE;
/* enable RX queue */
- FM10K_WRITE_REG(hw, FM10K_RXQCTL(rx_queue_id), reg);
+ FM10K_WRITE_REG(hw, FM10K_RXQCTL(hw_queue_id), reg);
FM10K_WRITE_FLUSH(hw);
/* Setup the HW Rx Head and Tail Descriptor Pointers
* Note: this must be done AFTER the queue is enabled
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -883,22 +961,39 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret;
+#endif
/** @todo - this should be defined in the shared code */
#define FM10K_TXDCTL_WRITE_BACK_MIN_DELAY 0x00010000
uint32_t txdctl = FM10K_TXDCTL_WRITE_BACK_MIN_DELAY;
struct fm10k_tx_queue *q = dev->data->tx_queues[tx_queue_id];
+ uint16_t hw_queue_id = tx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ tx_queue_id, dev->data->nb_tx_queues, &hw, &hw_queue_id);
+ if (ret < 0)
+ return -EIO;
+ else if (ret != 1)
+ return 0;
+#endif
+
q->ops->reset(q);
/* reset head and tail pointers */
- FM10K_WRITE_REG(hw, FM10K_TDH(tx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_TDT(tx_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDT(hw_queue_id), 0);
/* enable TX queue */
- FM10K_WRITE_REG(hw, FM10K_TXDCTL(tx_queue_id),
+ FM10K_WRITE_REG(hw, FM10K_TXDCTL(hw_queue_id),
FM10K_TXDCTL_ENABLE | txdctl);
FM10K_WRITE_FLUSH(hw);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -1089,9 +1184,22 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
int i, diag;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ struct fm10k_hw *mapped_hws[2];
+ int j, mapped_num;
+ uint32_t data;
+#endif
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > 2)
+ return -EIO;
+#endif
+
+
+#ifndef ENABLE_FM10K_MANAGEMENT
/* stop, init, then start the hw */
diag = fm10k_stop_hw(hw);
if (diag != FM10K_SUCCESS) {
@@ -1110,6 +1218,62 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
return -EIO;
}
+#else
+ for (j = 0; j < mapped_num; j++) {
+ struct rte_pci_device *pdev =
+ RTE_ETH_DEV_TO_PCI((struct rte_eth_dev *)
+ (fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[j])));
+ struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+ /* stop, init, then start the hw */
+ diag = fm10k_stop_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware stop failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_init_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware init failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_start_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
+ return -EIO;
+ }
+
+ /* Disable TXINT to avoid possible interrupt */
+ for (i = 0; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_TXINT(i),
+ 3 << FM10K_TXINT_TIMER_SHIFT);
+
+ /* enable RXINT for interrupt mode */
+ i = 0;
+ if (rte_intr_dp_is_en(intr_handle)) {
+ for (; i < dev->data->nb_rx_queues; i++) {
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_RXINT(i), Q2V(pdev, i));
+ if (mapped_hws[j]->mac.type == fm10k_mac_pf)
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_ITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ else
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_VFITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ }
+ }
+
+ /* Disable other RXINT to avoid possible interrupt */
+ for (; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_RXINT(i),
+ 3 << FM10K_RXINT_TIMER_SHIFT);
+ }
+#endif
diag = fm10k_dev_tx_init(dev);
if (diag) {
@@ -1161,12 +1325,32 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
}
}
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Update default vlan when not in VMDQ mode */
if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+#endif
fm10k_link_update(dev, 0);
+#ifdef ENABLE_FM10K_MANAGEMENT
+ /* Admit all VLANs */
+ for (j = 0; j <= 64; j++) {
+ for (i = 0; i < FM10K_SW_VLAN_TABLE_ENTRIES; i++)
+ FM10K_WRITE_REG(hw,
+ FM10K_SW_VLAN_TABLE_ENTRY(j, i),
+ 0xffffffff);
+ }
+
+ /* Disable PEP 1loopback */
+ /* XXX Does this need to be done by the master
+ * PEP while the switch is in reset?
+ */
+ data = FM10K_READ_REG(hw, FM10K_CTRL_EXT);
+ data &= ~FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK;
+ FM10K_WRITE_REG(hw, FM10K_CTRL_EXT, data);
+#endif
+
return 0;
}
@@ -1327,17 +1511,41 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
fm10k_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
uint64_t ipackets, opackets, ibytes, obytes, imissed;
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw =
FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_hw *mapped_hws[2];
+ int mapped_num;
+ uint16_t hw_queue_id;
+#endif
struct fm10k_hw_stats *hw_stats =
FM10K_DEV_PRIVATE_TO_STATS(dev->data->dev_private);
int i;
PMD_INIT_FUNC_TRACE();
+#ifndef ENABLE_FM10K_MANAGEMENT
fm10k_update_hw_stats(hw, hw_stats);
+#else
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(unmap_hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > 2)
+ return -EIO;
+
+ for (i = 0; i < mapped_num; i++) {
+ struct rte_eth_dev *mydev =
+ fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[i]);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS(mydev->data->dev_private);
+ fm10k_update_hw_stats(mapped_hws[i], hw_stats);
+ }
+#endif
ipackets = opackets = ibytes = obytes = imissed = 0;
+
+#ifndef ENABLE_FM10K_MANAGEMENT
for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
(i < hw->mac.max_queues); ++i) {
stats->q_ipackets[i] = hw_stats->q[i].rx_packets.count;
@@ -1351,6 +1559,36 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
obytes += stats->q_obytes[i];
imissed += stats->q_errors[i];
}
+#else
+ if (mapped_num) {
+ for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
+ (i < unmap_hw->mac.max_queues); ++i) {
+ hw_queue_id = i;
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, unmap_hw->mac.max_queues,
+ &hw, &hw_queue_id);
+ if (mapped_hws[1]) {
+ struct rte_eth_dev *mydev;
+ mydev = fm10k_switch_dpdk_port_rte_dev_get(hw);
+ hw_stats =
+ FM10K_DEV_PRIVATE_TO_STATS
+ (mydev->data->dev_private);
+ }
+ stats->q_ipackets[i] =
+ hw_stats->q[hw_queue_id].rx_packets.count;
+ stats->q_opackets[i] =
+ hw_stats->q[hw_queue_id].tx_packets.count;
+ stats->q_ibytes[i] =
+ hw_stats->q[hw_queue_id].rx_bytes.count;
+ stats->q_obytes[i] =
+ hw_stats->q[hw_queue_id].tx_bytes.count;
+ ipackets += stats->q_ipackets[i];
+ opackets += stats->q_opackets[i];
+ ibytes += stats->q_ibytes[i];
+ obytes += stats->q_obytes[i];
+ }
+ }
+#endif
stats->ipackets = ipackets;
stats->opackets = opackets;
stats->ibytes = ibytes;
@@ -1821,15 +2059,29 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *conf, struct rte_mempool *mp)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
struct fm10k_dev_info *dev_info =
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
struct fm10k_rx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id) < 0)
+ return -EIO;
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads;
/* make sure the mempool element size can account for alignment. */
@@ -1875,7 +2127,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
q->port_id = dev->data->port_id;
q->queue_id = queue_id;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_RDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_RDT(hw_queue_id)];
q->offloads = offloads;
if (handle_rxconf(q, conf))
return -EINVAL;
@@ -2010,13 +2262,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *conf)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
struct fm10k_tx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id) < 0)
+ return -EIO;
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.txmode.offloads;
/* make sure a valid number of descriptors have been requested */
@@ -2058,7 +2324,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
q->offloads = offloads;
q->ops = &def_txq_ops;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_TDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_TDT(hw_queue_id)];
if (handle_txconf(q, conf))
return -EINVAL;
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
@ 2019-12-11 15:48 ` Jerin Jacob
2019-12-12 9:35 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
1 sibling, 1 reply; 50+ messages in thread
From: Jerin Jacob @ 2019-12-11 15:48 UTC (permalink / raw)
To: Xiaojun Liu
Cc: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal,
jacob.e.keller, dev
On Wed, Dec 11, 2019 at 3:22 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
>
> To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers).
> Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
> Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
> Add fm10k/switch/fm10k_i2c.c(support I2C access).
> Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
> Add fm10k/switch/fm10k_sbus.c(support SBUS access).
> and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
> add fm10k_i2c.c and fm10k_sbus.c).
Integrating NIC with the integrated or onboard switches over i2c or
sbus is a common problem.
Instead of polluting ethdev driver with i2c and sbus _bus_ code, Why
not add new i2c bus
and move this code to driver/bus/i2c/xxxx/
>
> To avoid configuration for both kernel driver
> and userspace SDK outside DPDK, we add switch
> management in FM10K DPDK PMD driver.
> To enable switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-11 15:48 ` Jerin Jacob
@ 2019-12-12 9:35 ` Xiaojun Liu
2019-12-12 17:12 ` Jerin Jacob
0 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-12 9:35 UTC (permalink / raw)
To: Jerin Jacob
Cc: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal,
jacob.e.keller, dev, Jeff Zheng, Eyal Cohen
This is not a generic i2c bus, but rather an indirect access to i2c devices through FM10k registers. It's only purpose is to support the fm10k chip. Also there's currently no i2c bus support framework in DPDK.
-----Original Message-----
From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
Sent: Wednesday, December 11, 2019 11:48 PM
To: Xiaojun Liu
Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
On Wed, Dec 11, 2019 at 3:22 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
>
> To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers).
> Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
> Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
> Add fm10k/switch/fm10k_i2c.c(support I2C access).
> Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
> Add fm10k/switch/fm10k_sbus.c(support SBUS access).
> and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
> add fm10k_i2c.c and fm10k_sbus.c).
Integrating NIC with the integrated or onboard switches over i2c or
sbus is a common problem.
Instead of polluting ethdev driver with i2c and sbus _bus_ code, Why
not add new i2c bus
and move this code to driver/bus/i2c/xxxx/
>
> To avoid configuration for both kernel driver
> and userspace SDK outside DPDK, we add switch
> management in FM10K DPDK PMD driver.
> To enable switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-12 9:35 ` Xiaojun Liu
@ 2019-12-12 17:12 ` Jerin Jacob
2019-12-13 2:44 ` Xiaojun Liu
0 siblings, 1 reply; 50+ messages in thread
From: Jerin Jacob @ 2019-12-12 17:12 UTC (permalink / raw)
To: Xiaojun Liu
Cc: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal,
jacob.e.keller, dev, Jeff Zheng, Eyal Cohen
On Thu, Dec 12, 2019 at 3:05 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
>
> This is not a generic i2c bus, but rather an indirect access to i2c devices through FM10k registers. It's only purpose is to support the fm10k chip. Also there's currently no i2c bus support framework in DPDK.
To understand it better:
# Who are the i2c master and i2c slave here?
# Is i2c slave connected on board or in-built to FM10K SoC?
# What is the purpose of the i2c API in the ethdev driver?
>
> -----Original Message-----
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Wednesday, December 11, 2019 11:48 PM
> To: Xiaojun Liu
> Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
>
> On Wed, Dec 11, 2019 at 3:22 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
> >
> > To support switch management, add the following files:
> > Add fm10k/switch/fm10k_debug.h(define log Macros).
> > Add fm10k/switch/fm10k_regs.h(define all the registers).
> > Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
> > Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
> > Add fm10k/switch/fm10k_i2c.c(support I2C access).
> > Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
> > Add fm10k/switch/fm10k_sbus.c(support SBUS access).
> > and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
> > add fm10k_i2c.c and fm10k_sbus.c).
>
> Integrating NIC with the integrated or onboard switches over i2c or
> sbus is a common problem.
> Instead of polluting ethdev driver with i2c and sbus _bus_ code, Why
> not add new i2c bus
> and move this code to driver/bus/i2c/xxxx/
>
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> > Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-12 17:12 ` Jerin Jacob
@ 2019-12-13 2:44 ` Xiaojun Liu
2019-12-16 4:54 ` Jerin Jacob
0 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2019-12-13 2:44 UTC (permalink / raw)
To: Jerin Jacob
Cc: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal,
jacob.e.keller, dev, Jeff Zheng, Eyal Cohen
1. FM10K is the i2c master, the PCA9545/PCA9505/PCA9538 are the slave.
2. All the i2c slave connect on board to FM10K SoC.
3. To control the PHY and LED
-----Original Message-----
From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
Sent: Friday, December 13, 2019 1:12 AM
To: Xiaojun Liu
Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org; Jeff Zheng; Eyal Cohen
Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
On Thu, Dec 12, 2019 at 3:05 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
>
> This is not a generic i2c bus, but rather an indirect access to i2c devices through FM10k registers. It's only purpose is to support the fm10k chip. Also there's currently no i2c bus support framework in DPDK.
To understand it better:
# Who are the i2c master and i2c slave here?
# Is i2c slave connected on board or in-built to FM10K SoC?
# What is the purpose of the i2c API in the ethdev driver?
>
> -----Original Message-----
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Wednesday, December 11, 2019 11:48 PM
> To: Xiaojun Liu
> Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
>
> On Wed, Dec 11, 2019 at 3:22 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
> >
> > To support switch management, add the following files:
> > Add fm10k/switch/fm10k_debug.h(define log Macros).
> > Add fm10k/switch/fm10k_regs.h(define all the registers).
> > Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
> > Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
> > Add fm10k/switch/fm10k_i2c.c(support I2C access).
> > Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
> > Add fm10k/switch/fm10k_sbus.c(support SBUS access).
> > and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
> > add fm10k_i2c.c and fm10k_sbus.c).
>
> Integrating NIC with the integrated or onboard switches over i2c or
> sbus is a common problem.
> Instead of polluting ethdev driver with i2c and sbus _bus_ code, Why
> not add new i2c bus
> and move this code to driver/bus/i2c/xxxx/
>
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> > Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
2019-12-13 2:44 ` Xiaojun Liu
@ 2019-12-16 4:54 ` Jerin Jacob
0 siblings, 0 replies; 50+ messages in thread
From: Jerin Jacob @ 2019-12-16 4:54 UTC (permalink / raw)
To: Xiaojun Liu
Cc: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jakub.fornal,
jacob.e.keller, dev, Jeff Zheng, Eyal Cohen
On Fri, Dec 13, 2019 at 8:14 AM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
>
> 1. FM10K is the i2c master, the PCA9545/PCA9505/PCA9538 are the slave.
> 2. All the i2c slave connect on board to FM10K SoC.
> 3. To control the PHY and LED
Yes. These are onboard devices, so it makes sense to segregate code
for i2c master(in this case FM10K) and slave and bus API
to reuse the logic. This to address the case where PCA9545 used by
other i2c master or new board come up with a new i2c device instead of
PCA9545 etc.
And yes, DPDK is missing all these infrastructure code pieces. So it
is good to add infrastructure first.
I am leaving the decision to you and/or community.
>
>
> -----Original Message-----
> From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> Sent: Friday, December 13, 2019 1:12 AM
> To: Xiaojun Liu
> Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org; Jeff Zheng; Eyal Cohen
> Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
>
> On Thu, Dec 12, 2019 at 3:05 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
> >
> > This is not a generic i2c bus, but rather an indirect access to i2c devices through FM10k registers. It's only purpose is to support the fm10k chip. Also there's currently no i2c bus support framework in DPDK.
>
> To understand it better:
>
> # Who are the i2c master and i2c slave here?
> # Is i2c slave connected on board or in-built to FM10K SoC?
> # What is the purpose of the i2c API in the ethdev driver?
>
>
>
> >
> > -----Original Message-----
> > From: Jerin Jacob [mailto:jerinjacobk@gmail.com]
> > Sent: Wednesday, December 11, 2019 11:48 PM
> > To: Xiaojun Liu
> > Cc: xiao.w.wang@intel.com; qi.z.zhang@intel.com; ngai-mint.kwan@intel.com; jakub.fornal@intel.co; jacob.e.keller@intel.com; dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition
> >
> > On Wed, Dec 11, 2019 at 3:22 PM Xiaojun Liu <xiaojun.liu@silicom.co.il> wrote:
> > >
> > > To support switch management, add the following files:
> > > Add fm10k/switch/fm10k_debug.h(define log Macros).
> > > Add fm10k/switch/fm10k_regs.h(define all the registers).
> > > Add fm10k/switch/fm10k_switch.h(define switch Macros and APIs).
> > > Add fm10k/switch/fm10k_i2c.h(define I2C interfaces).
> > > Add fm10k/switch/fm10k_i2c.c(support I2C access).
> > > Add fm10k/switch/fm10k_sbus.h(define SBUS interface).
> > > Add fm10k/switch/fm10k_sbus.c(support SBUS access).
> > > and modify fm10k/Makefile(add ENABLE_FM10K_MANAGEMENT support,
> > > add fm10k_i2c.c and fm10k_sbus.c).
> >
> > Integrating NIC with the integrated or onboard switches over i2c or
> > sbus is a common problem.
> > Instead of polluting ethdev driver with i2c and sbus _bus_ code, Why
> > not add new i2c bus
> > and move this code to driver/bus/i2c/xxxx/
> >
> > >
> > > To avoid configuration for both kernel driver
> > > and userspace SDK outside DPDK, we add switch
> > > management in FM10K DPDK PMD driver.
> > > To enable switch management, you need add
> > > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > > config/common_linux when building.
> > >
> > > Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
` (6 preceding siblings ...)
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 7/7] net/fm10k: add dpdk port mapping Xiaojun Liu
@ 2020-01-21 2:53 ` Wang, Xiao W
2020-01-21 6:15 ` Xiaojun Liu
7 siblings, 1 reply; 50+ messages in thread
From: Wang, Xiao W @ 2020-01-21 2:53 UTC (permalink / raw)
To: Xiaojun Liu
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, jakub.fornal, Keller, Jacob E
Hi Xiaojun,
Could you please help to improve the commit logs of all the 7 patches? They look very similar, and info like below is not very helpful for reviewer, since we already know which file you are adding.
"To support switch management, add the following files:
Add fm10k/switch/fm10k_debug.h(define log Macros).
Add fm10k/switch/fm10k_regs.h(define all the registers)."
Please talk more about the design and implementation details in the commit log. Refer to history patches if you need a sample.
Also please help to address the compile error reported by automation in link http://patches.dpdk.org/patch/63742/:
"ci/Intel-compilation fail Compilation issues"
I would look deeper into your change, and you can address above comments simultaneously.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Wednesday, December 11, 2019 5:52 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> jakub.fornal@intel.co; Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v2 0/7] support switch management
>
> To avoid configuration for both kernel driver
> and userspace SDK outside DPDK, we add switch
> management in FM10K DPDK PMD driver.
> To enable switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
>
> Xiaojun Liu (7):
> net/fm10k: add i2c sbus registers definition
> net/fm10k: add some modules of port
> net/fm10k: add config ffu statistics support
> net/fm10k: add flow and switch management
> net/fm10k: add switch initialization
> net/fm10k: add mirror and filter ctrl
> net/fm10k: add dpdk port mapping
>
> drivers/net/fm10k/Makefile | 22 +
> drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> drivers/net/fm10k/switch/fm10k_regs.h | 2202 ++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> +++++++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> drivers/net/fm10k/switch/fm10k_switch.c | 2562
> +++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> 26 files changed, 17188 insertions(+), 34 deletions(-)
> create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
2020-01-21 2:53 ` [dpdk-dev] [PATCH v2 0/7] support switch management Wang, Xiao W
@ 2020-01-21 6:15 ` Xiaojun Liu
2020-02-11 10:31 ` Wang, Xiao W
0 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-01-21 6:15 UTC (permalink / raw)
To: Wang, Xiao W
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, jakub.fornal, Keller, Jacob E,
Jeff Zheng, Eyal Cohen
Hi Xiao,
Thank you! I will update the commit log and prepare a document to describe the design and implementation.
Best regards,
Xiaojun
-----Original Message-----
From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
Sent: Tuesday, January 21, 2020 10:53 AM
To: Xiaojun Liu
Cc: dev@dpdk.org; Zhang, Qi Z; Kwan, Ngai-mint; jakub.fornal@intel.co; Keller, Jacob E
Subject: RE: [PATCH v2 0/7] support switch management
Hi Xiaojun,
Could you please help to improve the commit logs of all the 7 patches? They look very similar, and info like below is not very helpful for reviewer, since we already know which file you are adding.
"To support switch management, add the following files:
Add fm10k/switch/fm10k_debug.h(define log Macros).
Add fm10k/switch/fm10k_regs.h(define all the registers)."
Please talk more about the design and implementation details in the commit log. Refer to history patches if you need a sample.
Also please help to address the compile error reported by automation in link http://patches.dpdk.org/patch/63742/:
"ci/Intel-compilation fail Compilation issues"
I would look deeper into your change, and you can address above comments simultaneously.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Wednesday, December 11, 2019 5:52 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> jakub.fornal@intel.co; Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v2 0/7] support switch management
>
> To avoid configuration for both kernel driver
> and userspace SDK outside DPDK, we add switch
> management in FM10K DPDK PMD driver.
> To enable switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
>
> Xiaojun Liu (7):
> net/fm10k: add i2c sbus registers definition
> net/fm10k: add some modules of port
> net/fm10k: add config ffu statistics support
> net/fm10k: add flow and switch management
> net/fm10k: add switch initialization
> net/fm10k: add mirror and filter ctrl
> net/fm10k: add dpdk port mapping
>
> drivers/net/fm10k/Makefile | 22 +
> drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> drivers/net/fm10k/switch/fm10k_regs.h | 2202 ++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> +++++++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> drivers/net/fm10k/switch/fm10k_switch.c | 2562
> +++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> 26 files changed, 17188 insertions(+), 34 deletions(-)
> create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
>
> --
> 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
2020-01-21 6:15 ` Xiaojun Liu
@ 2020-02-11 10:31 ` Wang, Xiao W
2020-02-14 2:46 ` Xiaojun Liu
[not found] ` <ORIGINAL-RELEASE-1581835643311863404-DB7PR04MB5196A5418792DFB1F96DB8B7BD150@DB7PR04MB5196.eurprd04.prod.outlook.com>
0 siblings, 2 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-02-11 10:31 UTC (permalink / raw)
To: Xiaojun Liu
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, Fornal, Jakub, Keller,
Jacob E, Jeff Zheng, Eyal Cohen
Since this is a big code change, so just some general comments/suggestions for your next version patch set:
- please clean up comments like "XXX" in the code.
- It's better to define MACRO for all register addrs and bit shift, please try to avoid magic number (e.g. in the serdes part).
- There're threads created for handling events and timers, we'd better use pthread_join() to recycle them in fm10k_sm_detach().
- set "tapstop" and "shiftwidth" to 8 in vimrc, then recheck the macros and keep them aligned.
- There're some "printf" for debugging, please use dpdk logging API instead.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Tuesday, January 21, 2020 2:15 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint
> <ngai-mint.kwan@intel.com>; jakub.fornal@intel.co; Keller, Jacob E
> <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal
> Cohen <eyalc@silicom.co.il>
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiao,
>
> Thank you! I will update the commit log and prepare a document to describe
> the design and implementation.
>
> Best regards,
> Xiaojun
>
> -----Original Message-----
> From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
> Sent: Tuesday, January 21, 2020 10:53 AM
> To: Xiaojun Liu
> Cc: dev@dpdk.org; Zhang, Qi Z; Kwan, Ngai-mint; jakub.fornal@intel.co;
> Keller, Jacob E
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiaojun,
>
> Could you please help to improve the commit logs of all the 7 patches? They
> look very similar, and info like below is not very helpful for reviewer, since
> we already know which file you are adding.
> "To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers)."
>
> Please talk more about the design and implementation details in the commit
> log. Refer to history patches if you need a sample.
>
> Also please help to address the compile error reported by automation in link
> http://patches.dpdk.org/patch/63742/:
> "ci/Intel-compilation fail Compilation issues"
>
> I would look deeper into your change, and you can address above comments
> simultaneously.
>
> Best Regards,
> Xiao
>
> > -----Original Message-----
> > From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Sent: Wednesday, December 11, 2019 5:52 PM
> > To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> > <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> > jakub.fornal@intel.co; Keller, Jacob E <jacob.e.keller@intel.com>
> > Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Subject: [PATCH v2 0/7] support switch management
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> >
> > Xiaojun Liu (7):
> > net/fm10k: add i2c sbus registers definition
> > net/fm10k: add some modules of port
> > net/fm10k: add config ffu statistics support
> > net/fm10k: add flow and switch management
> > net/fm10k: add switch initialization
> > net/fm10k: add mirror and filter ctrl
> > net/fm10k: add dpdk port mapping
> >
> > drivers/net/fm10k/Makefile | 22 +
> > drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> > drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> > drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> > drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> > drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> > drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> > drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> > drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> > drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> > drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> > drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> > drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> > drivers/net/fm10k/switch/fm10k_regs.h | 2202
> ++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> > drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> > drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> > drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> > drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> > drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> > +++++++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> > drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> > drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> > drivers/net/fm10k/switch/fm10k_switch.c | 2562
> > +++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> > 26 files changed, 17188 insertions(+), 34 deletions(-)
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
> >
> > --
> > 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
2020-02-11 10:31 ` Wang, Xiao W
@ 2020-02-14 2:46 ` Xiaojun Liu
[not found] ` <ORIGINAL-RELEASE-1581835643311863404-DB7PR04MB5196A5418792DFB1F96DB8B7BD150@DB7PR04MB5196.eurprd04.prod.outlook.com>
1 sibling, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-14 2:46 UTC (permalink / raw)
To: Wang, Xiao W
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, Fornal, Jakub, Keller,
Jacob E, Jeff Zheng, Eyal Cohen
hi Xiao,
Thanks for your comments! I will fix them ASAP.
Best regards,
Xiaojun
________________________________
From: Wang, Xiao W <xiao.w.wang@intel.com>
Sent: Tuesday, February 11, 2020 6:31 PM
To: Xiaojun Liu <xiaojun.liu@silicom.co.il>
Cc: dev@dpdk.org <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Fornal, Jakub <jakub.fornal@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal Cohen <eyalc@silicom.co.il>
Subject: RE: [PATCH v2 0/7] support switch management
Since this is a big code change, so just some general comments/suggestions for your next version patch set:
- please clean up comments like "XXX" in the code.
- It's better to define MACRO for all register addrs and bit shift, please try to avoid magic number (e.g. in the serdes part).
- There're threads created for handling events and timers, we'd better use pthread_join() to recycle them in fm10k_sm_detach().
- set "tapstop" and "shiftwidth" to 8 in vimrc, then recheck the macros and keep them aligned.
- There're some "printf" for debugging, please use dpdk logging API instead.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Tuesday, January 21, 2020 2:15 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint
> <ngai-mint.kwan@intel.com>; jakub.fornal@intel.co; Keller, Jacob E
> <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal
> Cohen <eyalc@silicom.co.il>
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiao,
>
> Thank you! I will update the commit log and prepare a document to describe
> the design and implementation.
>
> Best regards,
> Xiaojun
>
> -----Original Message-----
> From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
> Sent: Tuesday, January 21, 2020 10:53 AM
> To: Xiaojun Liu
> Cc: dev@dpdk.org; Zhang, Qi Z; Kwan, Ngai-mint; jakub.fornal@intel.co;
> Keller, Jacob E
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiaojun,
>
> Could you please help to improve the commit logs of all the 7 patches? They
> look very similar, and info like below is not very helpful for reviewer, since
> we already know which file you are adding.
> "To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers)."
>
> Please talk more about the design and implementation details in the commit
> log. Refer to history patches if you need a sample.
>
> Also please help to address the compile error reported by automation in link
> http://patches.dpdk.org/patch/63742/:
> "ci/Intel-compilation fail Compilation issues"
>
> I would look deeper into your change, and you can address above comments
> simultaneously.
>
> Best Regards,
> Xiao
>
> > -----Original Message-----
> > From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Sent: Wednesday, December 11, 2019 5:52 PM
> > To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> > <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> > jakub.fornal@intel.co; Keller, Jacob E <jacob.e.keller@intel.com>
> > Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Subject: [PATCH v2 0/7] support switch management
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> >
> > Xiaojun Liu (7):
> > net/fm10k: add i2c sbus registers definition
> > net/fm10k: add some modules of port
> > net/fm10k: add config ffu statistics support
> > net/fm10k: add flow and switch management
> > net/fm10k: add switch initialization
> > net/fm10k: add mirror and filter ctrl
> > net/fm10k: add dpdk port mapping
> >
> > drivers/net/fm10k/Makefile | 22 +
> > drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> > drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> > drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> > drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> > drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> > drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> > drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> > drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> > drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> > drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> > drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> > drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> > drivers/net/fm10k/switch/fm10k_regs.h | 2202
> ++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> > drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> > drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> > drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> > drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> > drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> > +++++++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> > drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> > drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> > drivers/net/fm10k/switch/fm10k_switch.c | 2562
> > +++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> > 26 files changed, 17188 insertions(+), 34 deletions(-)
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
> >
> > --
> > 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
[not found] ` <ORIGINAL-RELEASE-1581835643311863404-DB7PR04MB5196A5418792DFB1F96DB8B7BD150@DB7PR04MB5196.eurprd04.prod.outlook.com>
@ 2020-02-19 5:58 ` Xiaojun Liu
2020-02-19 10:56 ` Wang, Xiao W
0 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-19 5:58 UTC (permalink / raw)
To: Wang, Xiao W
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, Fornal, Jakub, Keller,
Jacob E, Jeff Zheng, Eyal Cohen
Hi Xiao,
I have already finished the fixing.
Thank you for your helping so much. I still have some questions.
1. Shall I generate the patch based on old patches ?
2. I used checkpatches.sh+check-git-log.sh+test-build.sh to check the patch, is that enough ?
3. May I send the patch to you first ? So you can help checking it before I submit.
Best regards,
Xiaojun
________________________________
From: dev <dev-bounces@dpdk.org> on behalf of Xiaojun Liu <xiaojun.liu@silicom.co.il>
Sent: Friday, February 14, 2020 10:46 AM
To: Wang, Xiao W <xiao.w.wang@intel.com>
Cc: dev@dpdk.org <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Fornal, Jakub <jakub.fornal@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal Cohen <eyalc@silicom.co.il>
Subject: Re: [dpdk-dev] [PATCH v2 0/7] support switch management
hi Xiao,
Thanks for your comments! I will fix them ASAP.
Best regards,
Xiaojun
________________________________
From: Wang, Xiao W <xiao.w.wang@intel.com>
Sent: Tuesday, February 11, 2020 6:31 PM
To: Xiaojun Liu <xiaojun.liu@silicom.co.il>
Cc: dev@dpdk.org <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Fornal, Jakub <jakub.fornal@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal Cohen <eyalc@silicom.co.il>
Subject: RE: [PATCH v2 0/7] support switch management
Since this is a big code change, so just some general comments/suggestions for your next version patch set:
- please clean up comments like "XXX" in the code.
- It's better to define MACRO for all register addrs and bit shift, please try to avoid magic number (e.g. in the serdes part).
- There're threads created for handling events and timers, we'd better use pthread_join() to recycle them in fm10k_sm_detach().
- set "tapstop" and "shiftwidth" to 8 in vimrc, then recheck the macros and keep them aligned.
- There're some "printf" for debugging, please use dpdk logging API instead.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Tuesday, January 21, 2020 2:15 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint
> <ngai-mint.kwan@intel.com>; jakub.fornal@intel.co; Keller, Jacob E
> <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal
> Cohen <eyalc@silicom.co.il>
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiao,
>
> Thank you! I will update the commit log and prepare a document to describe
> the design and implementation.
>
> Best regards,
> Xiaojun
>
> -----Original Message-----
> From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
> Sent: Tuesday, January 21, 2020 10:53 AM
> To: Xiaojun Liu
> Cc: dev@dpdk.org; Zhang, Qi Z; Kwan, Ngai-mint; jakub.fornal@intel.co;
> Keller, Jacob E
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiaojun,
>
> Could you please help to improve the commit logs of all the 7 patches? They
> look very similar, and info like below is not very helpful for reviewer, since
> we already know which file you are adding.
> "To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers)."
>
> Please talk more about the design and implementation details in the commit
> log. Refer to history patches if you need a sample.
>
> Also please help to address the compile error reported by automation in link
> http://patches.dpdk.org/patch/63742/:
> "ci/Intel-compilation fail Compilation issues"
>
> I would look deeper into your change, and you can address above comments
> simultaneously.
>
> Best Regards,
> Xiao
>
> > -----Original Message-----
> > From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Sent: Wednesday, December 11, 2019 5:52 PM
> > To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> > <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> > jakub.fornal@intel.co; Keller, Jacob E <jacob.e.keller@intel.com>
> > Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> > Subject: [PATCH v2 0/7] support switch management
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> >
> > Xiaojun Liu (7):
> > net/fm10k: add i2c sbus registers definition
> > net/fm10k: add some modules of port
> > net/fm10k: add config ffu statistics support
> > net/fm10k: add flow and switch management
> > net/fm10k: add switch initialization
> > net/fm10k: add mirror and filter ctrl
> > net/fm10k: add dpdk port mapping
> >
> > drivers/net/fm10k/Makefile | 22 +
> > drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> > drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> > drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> > drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> > drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> > drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> > drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> > drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> > drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> > drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> > drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> > drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> > drivers/net/fm10k/switch/fm10k_regs.h | 2202
> ++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> > drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> > drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> > drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> > drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> > drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> > +++++++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> > drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> > drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> > drivers/net/fm10k/switch/fm10k_switch.c | 2562
> > +++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> > 26 files changed, 17188 insertions(+), 34 deletions(-)
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
> >
> > --
> > 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/7] support switch management
2020-02-19 5:58 ` Xiaojun Liu
@ 2020-02-19 10:56 ` Wang, Xiao W
0 siblings, 0 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-02-19 10:56 UTC (permalink / raw)
To: Xiaojun Liu
Cc: dev, Zhang, Qi Z, Kwan, Ngai-mint, Fornal, Jakub, Keller,
Jacob E, Jeff Zheng, Eyal Cohen
Hi Xiaojun,
1. No, don't base on the old patch. Just send new version.
2. Yes.
3. No need, just send to the same names as this version. Please remember to add "-in-reply-to <Message-ID>" when sending new version.
The Message-ID can be found in the email header of the previous patch or in http://patches.dpdk.org/project/dpdk/list/?state=*&archive=both.
Also, please document what change you made from previous version. You can refer to others' patches in the mailing list.
Best Regards,
Xiao
From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
Sent: Wednesday, February 19, 2020 1:58 PM
To: Wang, Xiao W <xiao.w.wang@intel.com>
Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Fornal, Jakub <jakub.fornal@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>; Jeff Zheng <jeff.zheng@silicom.co.il>; Eyal Cohen <eyalc@silicom.co.il>
Subject: Re: [PATCH v2 0/7] support switch management
Hi Xiao,
I have already finished the fixing.
Thank you for your helping so much. I still have some questions.
1. Shall I generate the patch based on old patches ?
2. I used checkpatches.sh+check-git-log.sh+test-build.sh to check the patch, is that enough ?
3. May I send the patch to you first ? So you can help checking it before I submit.
Best regards,
Xiaojun
________________________________
From: dev <dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>> on behalf of Xiaojun Liu <xiaojun.liu@silicom.co.il<mailto:xiaojun.liu@silicom.co.il>>
Sent: Friday, February 14, 2020 10:46 AM
To: Wang, Xiao W <xiao.w.wang@intel.com<mailto:xiao.w.wang@intel.com>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org<mailto:dev@dpdk.org>>; Zhang, Qi Z <qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com<mailto:ngai-mint.kwan@intel.com>>; Fornal, Jakub <jakub.fornal@intel.com<mailto:jakub.fornal@intel.com>>; Keller, Jacob E <jacob.e.keller@intel.com<mailto:jacob.e.keller@intel.com>>; Jeff Zheng <jeff.zheng@silicom.co.il<mailto:jeff.zheng@silicom.co.il>>; Eyal Cohen <eyalc@silicom.co.il<mailto:eyalc@silicom.co.il>>
Subject: Re: [dpdk-dev] [PATCH v2 0/7] support switch management
hi Xiao,
Thanks for your comments! I will fix them ASAP.
Best regards,
Xiaojun
________________________________
From: Wang, Xiao W <xiao.w.wang@intel.com<mailto:xiao.w.wang@intel.com>>
Sent: Tuesday, February 11, 2020 6:31 PM
To: Xiaojun Liu <xiaojun.liu@silicom.co.il<mailto:xiaojun.liu@silicom.co.il>>
Cc: dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org<mailto:dev@dpdk.org>>; Zhang, Qi Z <qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com<mailto:ngai-mint.kwan@intel.com>>; Fornal, Jakub <jakub.fornal@intel.com<mailto:jakub.fornal@intel.com>>; Keller, Jacob E <jacob.e.keller@intel.com<mailto:jacob.e.keller@intel.com>>; Jeff Zheng <jeff.zheng@silicom.co.il<mailto:jeff.zheng@silicom.co.il>>; Eyal Cohen <eyalc@silicom.co.il<mailto:eyalc@silicom.co.il>>
Subject: RE: [PATCH v2 0/7] support switch management
Since this is a big code change, so just some general comments/suggestions for your next version patch set:
- please clean up comments like "XXX" in the code.
- It's better to define MACRO for all register addrs and bit shift, please try to avoid magic number (e.g. in the serdes part).
- There're threads created for handling events and timers, we'd better use pthread_join() to recycle them in fm10k_sm_detach().
- set "tapstop" and "shiftwidth" to 8 in vimrc, then recheck the macros and keep them aligned.
- There're some "printf" for debugging, please use dpdk logging API instead.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il<mailto:xiaojun.liu@silicom.co.il>>
> Sent: Tuesday, January 21, 2020 2:15 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com<mailto:xiao.w.wang@intel.com>>
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>>; Kwan, Ngai-mint
> <ngai-mint.kwan@intel.com<mailto:ngai-mint.kwan@intel.com>>; jakub.fornal@intel.co<mailto:jakub.fornal@intel.co>; Keller, Jacob E
> <jacob.e.keller@intel.com<mailto:jacob.e.keller@intel.com>>; Jeff Zheng <jeff.zheng@silicom.co.il<mailto:jeff.zheng@silicom.co.il>>; Eyal
> Cohen <eyalc@silicom.co.il<mailto:eyalc@silicom.co.il>>
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiao,
>
> Thank you! I will update the commit log and prepare a document to describe
> the design and implementation.
>
> Best regards,
> Xiaojun
>
> -----Original Message-----
> From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
> Sent: Tuesday, January 21, 2020 10:53 AM
> To: Xiaojun Liu
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Zhang, Qi Z; Kwan, Ngai-mint; jakub.fornal@intel.co<mailto:jakub.fornal@intel.co>;
> Keller, Jacob E
> Subject: RE: [PATCH v2 0/7] support switch management
>
> Hi Xiaojun,
>
> Could you please help to improve the commit logs of all the 7 patches? They
> look very similar, and info like below is not very helpful for reviewer, since
> we already know which file you are adding.
> "To support switch management, add the following files:
> Add fm10k/switch/fm10k_debug.h(define log Macros).
> Add fm10k/switch/fm10k_regs.h(define all the registers)."
>
> Please talk more about the design and implementation details in the commit
> log. Refer to history patches if you need a sample.
>
> Also please help to address the compile error reported by automation in link
> http://patches.dpdk.org/patch/63742/:
> "ci/Intel-compilation fail Compilation issues"
>
> I would look deeper into your change, and you can address above comments
> simultaneously.
>
> Best Regards,
> Xiao
>
> > -----Original Message-----
> > From: Xiaojun Liu <xiaojun.liu@silicom.co.il<mailto:xiaojun.liu@silicom.co.il>>
> > Sent: Wednesday, December 11, 2019 5:52 PM
> > To: Wang, Xiao W <xiao.w.wang@intel.com<mailto:xiao.w.wang@intel.com>>; Zhang, Qi Z
> > <qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com<mailto:ngai-mint.kwan@intel.com>>;
> > jakub.fornal@intel.co<mailto:jakub.fornal@intel.co>; Keller, Jacob E <jacob.e.keller@intel.com<mailto:jacob.e.keller@intel.com>>
> > Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Xiaojun Liu <xiaojun.liu@silicom.co.il<mailto:xiaojun.liu@silicom.co.il>>
> > Subject: [PATCH v2 0/7] support switch management
> >
> > To avoid configuration for both kernel driver
> > and userspace SDK outside DPDK, we add switch
> > management in FM10K DPDK PMD driver.
> > To enable switch management, you need add
> > CONFIG_RTE_FM10K_MANAGEMENT=y in
> > config/common_linux when building.
> >
> >
> > Xiaojun Liu (7):
> > net/fm10k: add i2c sbus registers definition
> > net/fm10k: add some modules of port
> > net/fm10k: add config ffu statistics support
> > net/fm10k: add flow and switch management
> > net/fm10k: add switch initialization
> > net/fm10k: add mirror and filter ctrl
> > net/fm10k: add dpdk port mapping
> >
> > drivers/net/fm10k/Makefile | 22 +
> > drivers/net/fm10k/fm10k_ethdev.c | 580 +++++-
> > drivers/net/fm10k/switch/fm10k_config.c | 855 ++++++++
> > drivers/net/fm10k/switch/fm10k_config.h | 171 ++
> > drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> > drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
> > drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
> > drivers/net/fm10k/switch/fm10k_ffu.c | 1209 +++++++++++
> > drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
> > drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
> > drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> > drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
> > drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> > drivers/net/fm10k/switch/fm10k_regs.h | 2202
> ++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
> > drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> > drivers/net/fm10k/switch/fm10k_serdes.c | 1886 +++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
> > drivers/net/fm10k/switch/fm10k_sm.c | 182 ++
> > drivers/net/fm10k/switch/fm10k_sm.h | 78 +
> > drivers/net/fm10k/switch/fm10k_spico_code.c | 2966
> > +++++++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
> > drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
> > drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
> > drivers/net/fm10k/switch/fm10k_switch.c | 2562
> > +++++++++++++++++++++++
> > drivers/net/fm10k/switch/fm10k_switch.h | 336 +++
> > 26 files changed, 17188 insertions(+), 34 deletions(-)
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
> > create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
> >
> > --
> > 1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 0/5] support switch management
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
2019-12-11 15:48 ` Jerin Jacob
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for " Xiaojun Liu
` (4 more replies)
1 sibling, 5 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
To avoid configuration for both kernel driver
and userspace SDK outside DPDK, we add switch
management in FM10K DPDK PMD driver.
Split dev init to 2 parts.
First only register the port in switch
management; second init hook will be
called after all the pf are registered
and switch initialization. It will finish
dev init. Also add switch interrupt support.
Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
to support mirror operation. Add fm10k_dev_filter_ctrl
to support flow operation.
Add dpdk port and pf mapping, so
the dpdk port can map to a specific pf
and 1 dpdk port can map to 2 pf to get
total 100G throughput.
Add flow interface to support offload flow into HW.
It supports parse vlan and parse mpls, all these
data will be transffered to ffu data.
Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
It create 3 threads. One for interrupt handler, one for
LED controller, one for statistics.
Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
It also supports flowset for a group flow definition.
The config file can configure debug log, port speed,
epl port mapping dpdk port, flowset. All these configuration
will be used by switch management.
Statistics includes epl port, ffu rule, dpdk port, and error.
All these statistics data are read from HW.
Modify switch header file to support getting logical port
and glort and device info.
Add epl serdes include loading spico,
controling pcsl, dma, dfe, ical. Add spico code.
Add state machine for epl lane and port, it creates
a pthread to handle the state changing event.
Add external port management, which will use
state machine to handle the event from lane and port.
The lane state will change between DOWN, WAIT_PLL_LOCK,
WAIT_SIGNAL_OK, WAIT_DFE_ICAL, WAIT_DFE_PCAL, UP.
The port state will change between DOWN, WAIT_LANE_UP, UP.
Add I2C to control the inside LED and PHY.
All the operations of I2C are using fm10k I2C register.
Add SBUS to communicate with spico(micro code in serdes)
by using fm10k SBUS register. This is like I2C operations.
Add registers defination, which include all the registers
will be used in the driver. Add switch management log API.
Add switch management structures. Modify Makefile to add
new files building
Xiaojun Liu (5):
net/fm10k: add basic functions for switch management
net/fm10k: add epl serdes and port control functions
net/fm10k: add ffu and statistics and config file functions
net/fm10k: add flow interface and switch management
net/fm10k: add switch management support
drivers/net/fm10k/Makefile | 25 +
drivers/net/fm10k/fm10k_ethdev.c | 587 +++++-
drivers/net/fm10k/switch/fm10k_config.c | 863 ++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 ++
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_ffu.c | 1253 +++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 81 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
drivers/net/fm10k/switch/fm10k_switch.c | 2376 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_switch.h | 475 +++++
26 files changed, 17364 insertions(+), 41 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for switch management
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
` (3 subsequent siblings)
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add I2C to control the inside LED and PHY.
All the operations of I2C are using fm10k I2C register.
Add SBUS to communicate with spico(micro code in serdes)
by using fm10k SBUS register. This is like I2C operations.
Add registers defination, which include all the registers
will be used in the driver. Add switch management log API.
Add switch management structures. Modify Makefile to add
new files building
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 16 +
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 ++++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
8 files changed, 3368 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 0271bcb..a628e2d 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -11,6 +11,9 @@ LIB = librte_pmd_fm10k.a
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
CFLAGS += -DALLOW_EXPERIMENTAL_API
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+CFLAGS += -DENABLE_FM10K_MANAGEMENT
+endif
EXPORT_MAP := rte_pmd_fm10k_version.map
@@ -49,6 +52,9 @@ endif
LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash
LDLIBS += -lrte_bus_pci
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+LDLIBS += -lpthread
+endif
#
# Add extra flags for base driver source files to disable warnings in them
@@ -58,6 +64,10 @@ $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER))
VPATH += $(SRCDIR)/base
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+VPATH += $(SRCDIR)/switch
+endif
+
#
# all source are stored in SRCS-y
# base driver is based on the package of cid-fm10k.2017.01.24.tar.gz
@@ -71,6 +81,12 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_common.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_mbx.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
+
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+endif
+
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/net/fm10k/switch/fm10k_debug.h b/drivers/net/fm10k/switch/fm10k_debug.h
new file mode 100644
index 0000000..f7b5c06
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_debug.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_DEBUG_H_
+#define _FM10K_DEBUG_H_
+
+
+#define FM10K_SW_ERR(...) PMD_INIT_LOG(ERR, __VA_ARGS__)
+#define FM10K_SW_INFO(...) PMD_INIT_LOG(INFO, __VA_ARGS__)
+#define FM10K_SW_TRACE(...) PMD_INIT_LOG(DEBUG, __VA_ARGS__)
+
+#define FM10K_SW_ASSERT(...) do {} while (0)
+
+#define FM10K_SW_STATS_TRACE_ENABLE 1
+#define FM10K_SW_FFU_CONF_TRACE_ENABLE 0
+#define FM10K_SW_MIRROR_TRACE_ENABLE 0
+
+#endif /* _FM10K_DEBUG_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.c b/drivers/net/fm10k/switch/fm10k_i2c.c
new file mode 100644
index 0000000..28b0c34
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.c
@@ -0,0 +1,310 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+
+static void fm10k_i2c_init(struct fm10k_i2c *);
+
+struct fm10k_i2c *
+fm10k_i2c_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_i2c *i2c;
+
+ FM10K_SW_TRACE("i2c: attaching");
+
+ i2c = (struct fm10k_i2c *)rte_zmalloc("fm10k_i2c",
+ sizeof(struct fm10k_i2c), 0);
+ if (i2c == NULL) {
+ FM10K_SW_INFO("i2c: failed to allocate context");
+ goto fail;
+ }
+
+ i2c->sw = sw;
+ pthread_mutex_init(&i2c->req_lock, NULL);
+ pthread_mutex_init(&i2c->bus_lock, NULL);
+ sem_init(&i2c->req_cv, 0, 0);
+
+ fm10k_i2c_init(i2c);
+
+ FM10K_SW_TRACE("i2c: attach successful");
+ return i2c;
+fail:
+ if (i2c)
+ fm10k_i2c_detach(i2c);
+ return NULL;
+}
+
+void
+fm10k_i2c_detach(struct fm10k_i2c *i2c)
+{
+ FM10K_SW_TRACE("i2c: detaching");
+
+ rte_free(i2c);
+}
+
+static void
+fm10k_i2c_init(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_device_info *cfg = sw->info;
+ uint32_t freq = FM10K_SW_I2C_CFG_DIVIDER_400_KHZ;
+ uint32_t data;
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ freq = FM10K_SW_I2C_CFG_DIVIDER_100_KHZ;
+
+ /* clear any pending interrupt */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ /* 400 KHz, master mode, unmask interrupt */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CFG);
+ data &= ~FM10K_SW_I2C_CFG_SLAVE_ENABLE;
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ FM10K_SW_REPLACE_REG_FIELD(data, I2C_CFG_DIVIDER, freq, data);
+ data &= ~FM10K_SW_I2C_CFG_INTERRUPT_MASK;
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CFG, data);
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ /* reset I2C */
+ fm10k_gpio_output_set(sw, 5, 1);
+}
+
+unsigned int
+fm10k_i2c_intr(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_i2c_req *req;
+ int i;
+ uint32_t data[3];
+ uint32_t ctrl;
+
+ req = i2c->cur_req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CTRL);
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ req->status = FM10K_SW_REG_FIELD(ctrl, I2C_CTRL_COMMAND_COMPLETED);
+
+ if ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->status == FM10K_SW_I2C_COMPLETION_NORMAL) {
+ for (i = 0; i < FM10K_SW_HOWMANY(req->read_len, 4, 4); i++)
+ data[i] = fm10k_read_switch_reg
+ (sw, FM10K_SW_I2C_DATA(i));
+
+ for (i = 0; i < req->read_len; i++)
+ req->msg[i] =
+ (data[i / 4] >> (24 - (i % 4) * 8)) & 0xff;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ sem_post(&i2c->req_cv);
+
+ return 1;
+}
+
+int
+fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ int i;
+ uint32_t ctrl;
+ uint32_t data[3];
+
+ if (((req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->write_len > FM10K_SW_I2C_MSG_MAX) ||
+ ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ ((req->read_len == 0 ||
+ req->read_len > FM10K_SW_I2C_MSG_MAX))))
+ return (-1);
+
+ FM10K_SW_TRACE("i2c: initiating command %u", req->cmd);
+
+ ctrl =
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_ADDR, req->addr << 1) |
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_COMMAND, req->cmd);
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) {
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_W, req->write_len);
+
+ data[0] = 0;
+ data[1] = 0;
+ data[2] = 0;
+
+ for (i = 0; i < req->write_len; i++)
+ data[i / 4] |= req->msg[i] << (24 - (i % 4) * 8);
+
+ for (i = 0; i < FM10K_SW_HOWMANY(req->write_len, 4, 4); i++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_I2C_DATA(i), data[i]);
+ }
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD)
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_R, req->read_len);
+
+ req->status = FM10K_SW_I2C_COMPLETION_RUNNING;
+ i2c->cur_req = req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* zero command field */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, 0);
+ /* initiate command */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, ctrl);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ while (req->status == FM10K_SW_I2C_COMPLETION_RUNNING)
+ sem_wait(&i2c->req_cv);
+
+ return 0;
+}
+
+int
+fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_RD;
+ req.read_len = 1;
+ req.msg[0] = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR_RD;
+ req.write_len = 1;
+ req.read_len = 1;
+ req.msg[0] = reg;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 1;
+ req.msg[0] = data;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 2;
+ req.msg[0] = data0;
+ req.msg[1] = data1;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.h b/drivers/net/fm10k/switch/fm10k_i2c.h
new file mode 100644
index 0000000..f835afe
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_I2C_H_
+#define _FM10K_SW_I2C_H_
+
+#include <semaphore.h>
+#include <pthread.h>
+#include "rte_spinlock.h"
+#include "fm10k_debug.h"
+
+#define FM10K_SW_I2C_MSG_MAX 12
+
+struct fm10k_i2c_req {
+ uint8_t addr; /* 7-bit address */
+ uint8_t cmd; /* FM10K_SW_I2C_COMMAND_* */
+ uint8_t write_len;
+ uint8_t read_len;
+ uint8_t status; /* FM10K_SW_I2C_COMPLETION_ */
+ uint8_t msg[FM10K_SW_I2C_MSG_MAX];
+};
+
+struct fm10k_i2c {
+ struct fm10k_switch *sw;
+ pthread_mutex_t bus_lock;
+ pthread_mutex_t req_lock;
+ sem_t req_cv;
+ struct fm10k_i2c_req *cur_req;
+};
+
+#define FM10K_SW_I2C_LOCK(i2c_) \
+ pthread_mutex_lock(&(i2c_)->bus_lock)
+#define FM10K_SW_I2C_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&(i2c_)->bus_lock)
+
+#define FM10K_SW_I2C_REQ_LOCK(i2c_) \
+ pthread_mutex_lock(&((i2c_)->req_lock))
+#define FM10K_SW_I2C_REQ_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&((i2c_)->req_lock))
+
+struct fm10k_i2c *fm10k_i2c_attach(struct fm10k_switch *sw);
+void fm10k_i2c_detach(struct fm10k_i2c *i2c);
+unsigned int fm10k_i2c_intr(struct fm10k_i2c *i2c);
+int fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req);
+int fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result);
+int fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result);
+int fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data);
+int fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1);
+int fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr);
+
+#endif /* _FM10K_SW_I2C_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_regs.h b/drivers/net/fm10k/switch/fm10k_regs.h
new file mode 100644
index 0000000..2663773
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_regs.h
@@ -0,0 +1,2302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_REGS_H_
+#define _FM10K_REGS_H_
+
+#include <stdint.h>
+
+
+/* Convert a 32-bit word offset into a byte offset */
+#define FM10K_SW_REG_OFF(wo_) (wo_)
+#define FM10K_SW_MASK32(max_bit_, min_bit_) \
+ ((0xffffffffU << (min_bit_)) & \
+ (0xffffffffU >> (31 - (max_bit_))))
+#define FM10K_SW_REG_FIELD(r_, name_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) >> FM10K_SW_##name_##_lsb)
+#define FM10K_SW_REG_FIELD_IDX(r_, name_, i_, i1_, i2_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i_), \
+ FM10K_SW_##name_##_lsb(i2_))) >> \
+ FM10K_SW_##name_##_lsb(i3_))
+#define FM10K_SW_MAKE_REG_FIELD(name_, v_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb))
+#define FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_, i1_, i2_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb(i_)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) | \
+ FM10K_SW_MAKE_REG_FIELD(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_)))
+#define FM10K_SW_MASK64(max_bit_, min_bit_) \
+ ((0xffffffffffffffffULL << (min_bit_)) & \
+ (0xffffffffffffffffULL >> (63 - (max_bit_))))
+#define FM10K_SW_REG_FIELD64(r_, name_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) >> \
+ FM10K_SW_##name_##_lsb64)
+#define FM10K_SW_REG_FIELD_IDX64(r_, name_, i_, i1_, i2_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i_), \
+ FM10K_SW_##name_##_lsb64(i1_))) >> \
+ FM10K_SW_##name_##_lsb64(i2_))
+#define FM10K_SW_MAKE_REG_FIELD64(name_, v_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64))
+#define FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_, i1_, i2_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64(i_)) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD64(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) | \
+ FM10K_SW_MAKE_REG_FIELD64(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX64(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_)))
+
+/* These operate on arrays of 32-bit words */
+#define FM10K_SW_SET_ARRAY_BIT(a_, name_, b_, a1_) \
+ ((a_)[FM10K_SW_##name_##_bit / 32] = \
+ ((a1_)[FM10K_SW_##name_##_bit / 32] & \
+ ~(1 << (FM10K_SW_##name_##_bit % 32))) | \
+ (((b_) & 0x1) << (FM10K_SW_##name_##_bit % 32)))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_MAKE_ARRAY_FIELD(name_, v_) \
+ (((uint32_t)(v_) << (FM10K_SW_##name_##_lsb % 32)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_REPLACE_ARRAY_FIELD(a_, name_, v_, a1_) \
+ ((a_)[FM10K_SW_##name_##_lsb / 32] = \
+ (((a1_)[FM10K_SW_##name_##_lsb / 32] & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32)) | \
+ FM10K_SW_MAKE_ARRAY_FIELD(name_, v_)))
+
+
+/*
+ * BAR0 registers
+ */
+/* Interrupt throttle timer selection values */
+#define FM10K_SW_INT_TIMER_0 0
+#define FM10K_SW_INT_TIMER_1 1
+#define FM10K_SW_INT_TIMER_IMMEDIATE 2
+#define FM10K_SW_INT_TIMER_DISABLED 3
+
+#define FM10K_SW_CTRL FM10K_SW_REG_OFF(0x0)
+#define FM10K_SW_CTRL_BAR4_ALLOWED (1 << 2)
+#define FM10K_SW_CTRL_EXT FM10K_SW_REG_OFF(0x1)
+#define FM10K_SW_CTRL_EXT_NS_DIS (1 << 0)
+#define FM10K_SW_CTRL_EXT_RO_DIS (1 << 1)
+#define FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK (1 << 2)
+#define FM10K_SW_EXVET FM10K_SW_REG_OFF(0x2)
+#define FM10K_SW_GCR FM10K_SW_REG_OFF(0x3)
+#define FM10K_SW_FACTPS FM10K_SW_REG_OFF(0x4)
+#define FM10K_SW_GCR_EXT FM10K_SW_REG_OFF(0x5)
+#define FM10K_SW_EICR FM10K_SW_REG_OFF(0x6)
+#define FM10K_SW_EICR_PCA_FAULT_SHIFT 0
+#define FM10K_SW_EICR_PCA_FAULT \
+ (1 << FM10K_SW_EICR_PCA_FAULT_SHIFT)
+#define FM10K_SW_EICR_THI_FAULT_SHIFT 2
+#define FM10K_SW_EICR_THI_FAULT \
+ (1 << FM10K_SW_EICR_THI_FAULT_SHIFT)
+#define FM10K_SW_EICR_FUM_FAULT_SHIFT 5
+#define FM10K_SW_EICR_FUM_FAULT \
+ (1 << FM10K_SW_EICR_FUM_FAULT_SHIFT)
+#define FM10K_SW_EICR_MAILBOX_SHIFT 6
+#define FM10K_SW_EICR_MAILBOX \
+ (1 << FM10K_SW_EICR_MAILBOX_SHIFT)
+#define FM10K_SW_EICR_SWITCH_READY_SHIFT 7
+#define FM10K_SW_EICR_SWITCH_READY \
+ (1 << FM10K_SW_EICR_SWITCH_READY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_NREADY_SHIFT 8
+#define FM10K_SW_EICR_SWITCH_NREADY \
+ (1 << FM10K_SW_EICR_SWITCH_NREADY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_INT_SHIFT 9
+#define FM10K_SW_EICR_SWITCH_INT \
+ (1 << FM10K_SW_EICR_SWITCH_INT_SHIFT)
+#define FM10K_SW_EICR_SRAM_ERROR_SHIFT 10
+#define FM10K_SW_EICR_SRAM_ERROR \
+ (1 << FM10K_SW_EICR_SRAM_ERROR_SHIFT)
+#define FM10K_SW_EICR_VFLR_SHIFT 11
+#define FM10K_SW_EICR_VFLR \
+ (1 << FM10K_SW_EICR_VFLR_SHIFT)
+#define FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT 12
+#define FM10K_SW_EICR_MAX_HOLD_TIME \
+ (1 << FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT)
+#define FM10K_SW_EIMR FM10K_SW_REG_OFF(0x7)
+#define FM10K_SW_EIMR_DISABLE_ALL 0x55555555
+#define FM10K_SW_EIMR_NO_CHANGE 0x0
+#define FM10K_SW_EIMR_DISABLE 0x1
+#define FM10K_SW_EIMR_ENABLE 0x2
+#define FM10K_SW_EIMR_FIELD(i_, v_) \
+ (FM10K_SW_EIMR_##v_ << (FM10K_SW_EICR_##i_##_SHIFT * 2))
+#define FM10K_SW_EIMR_ENABLED(e_, i_) \
+ ((((e_) >> (FM10K_SW_EICR_##i_##_SHIFT * 2)) & 0x3) == \
+ FM10K_SW_EIMR_ENABLE)
+#define FM10K_SW_PCA_FAULT FM10K_SW_REG_OFF(0x8)
+#define FM10K_SW_THI_FAULT FM10K_SW_REG_OFF(0x10)
+#define FM10K_SW_FUM_FAULT FM10K_SW_REG_OFF(0x1C)
+#define FM10K_SW_MAXHOLDQ(n_) FM10K_SW_REG_OFF(0x20 + (n_))
+#define FM10K_SW_MAXHOLDQ_ENTRIES 8
+#define FM10K_SW_SM_AREA FM10K_SW_REG_OFF(0x28)
+#define FM10K_SW_DGLORTMAP(n_) FM10K_SW_REG_OFF(0x30 + (n_))
+#define FM10K_SW_DGLORTMAP_ENTRIES 8
+#define FM10K_SW_DGLORTMAP_MATCH_ANY 0x00000000
+#define FM10K_SW_DGLORTMAP_MATCH_NONE 0x0000ffff
+#define FM10K_SW_DGLORTMAP_VALUE_lsb 0
+#define FM10K_SW_DGLORTMAP_VALUE_msb 15
+#define FM10K_SW_DGLORTMAP_MASK_lsb 16
+#define FM10K_SW_DGLORTMAP_MASK_msb 31
+#define FM10K_SW_DGLORTDEC(n_) FM10K_SW_REG_OFF(0x38 + (n_))
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_lsb 0
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_msb 3
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_lsb 4
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_msb 6
+#define FM10K_SW_DGLORTDEC_VSI_BASE_lsb 7
+#define FM10K_SW_DGLORTDEC_VSI_BASE_msb 13
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_lsb 14
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_msb 15
+#define FM10K_SW_DGLORTDEC_Q_BASE_lsb 16
+#define FM10K_SW_DGLORTDEC_Q_BASE_msb 23
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_MAX 7
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_lsb 24
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_msb 26
+#define FM10K_SW_DGLORTDEC_INNTER_RSS (1 << 27)
+#define FM10K_SW_TUNNEL_CFG FM10K_SW_REG_OFF(0x40)
+#define FM10K_SW_SWPRI_MAP(pri_) FM10K_SW_REG_OFF(0x50 + (pri_))
+#define FM10K_SW_RSSRK(f_, n_) \
+ FM10K_SW_REG_OFF(0x800 + 0x10 * (f_) + (n_))
+#define FM10K_SW_RSSRK_ENTRIES 10
+#define FM10K_SW_RETA(f_, n_) \
+ FM10K_SW_REG_OFF(0x1000 + 0x20 * (f_) + (n_))
+#define FM10K_SW_RETA_ENTRIES 32
+#define FM10K_SW_RETA_ENTRY(e0_, e1_, e2_, e3_) \
+ (((e0_) & 0xff) | (((e1_) & 0xff) << 8) | \
+ (((e2_) & 0xff) << 16) | (((e3_) & 0xff) << 24))
+#define FM10K_SW_TC_CREDIT(g_) FM10K_SW_REG_OFF(0x2000 + (g_))
+#define FM10K_SW_TC_MAXCREDIT(g_) FM10K_SW_REG_OFF(0x2040 + (g_))
+#define FM10K_SW_TC_RATE(g_) FM10K_SW_REG_OFF(0x2080 + (g_))
+#define FM10K_SW_TC_RATE_STATUS FM10K_SW_REG_OFF(0x20C0)
+#define FM10K_SW_PAUSE FM10K_SW_REG_OFF(0x20C2)
+#define FM10K_SW_DMA_CTRL FM10K_SW_REG_OFF(0x20C3)
+#define FM10K_SW_DMA_CTRL_TX_ENABLE (1 << 0)
+#define FM10K_SW_DMA_CTRL_TX_HOST_PENDING (1 << 1)
+#define FM10K_SW_DMA_CTRL_TX_DATA (1 << 2)
+#define FM10K_SW_DMA_CTRL_TX_ACTIVE (1 << 3)
+#define FM10K_SW_DMA_CTRL_RX_ENABLE (1 << 4)
+#define FM10K_SW_DMA_CTRL_RX_HOST_PENDING (1 << 5)
+#define FM10K_SW_DMA_CTRL_RX_DATA (1 << 6)
+#define FM10K_SW_DMA_CTRL_RX_ACTIVE (1 << 7)
+#define FM10K_SW_DMA_CTRL_RX_DESC_SIZE_32 (1 << 8)
+#define FM10K_SW_DMA_CTRL_MIN_MSS_lsb 9
+#define FM10K_SW_DMA_CTRL_MIN_MSS_msb 22
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_lsb 23
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_msb 27
+#define FM10K_SW_DMA_CTRL_DATA_PATH_RESET (1 << 29)
+#define FM10K_SW_DMA_CTRL_MAX_QS_lsb 30
+#define FM10K_SW_DMA_CTRL_MAX_QS_msb 31
+#define FM10K_SW_DMA_CTRL_MAX_QS_256 0
+#define FM10K_SW_DMA_CTRL_MAX_QS_128 1
+#define FM10K_SW_DMA_CTRL_MAX_QS_64 2
+#define FM10K_SW_DMA_CTRL2 FM10K_SW_REG_OFF(0x20C4)
+#define FM10K_SW_DTXTCPFLGL FM10K_SW_REG_OFF(0x20C5)
+#define FM10K_SW_DTXTCPFLGH FM10K_SW_REG_OFF(0x20C6)
+#define FM10K_SW_TPH_CTRL FM10K_SW_REG_OFF(0x20C7)
+#define FM10K_SW_MRQC(f_) \
+ FM10K_SW_REG_OFF(0x2100 + (f_))
+#define FM10K_SW_MRQC_TCPIPV4 (1 << 0)
+#define FM10K_SW_MRQC_IPV4 (1 << 1)
+#define FM10K_SW_MRQC_IPV6 (1 << 4)
+#define FM10K_SW_MRQC_TCPIPV6 (1 << 5)
+#define FM10K_SW_MRQC_UDPIPV4 (1 << 6)
+#define FM10K_SW_MRQC_UDPIPV6 (1 << 7)
+#define FM10K_SW_TQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x2800 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_RQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x3000 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_STATS_TIMEOUT FM10K_SW_REG_OFF(0x3800)
+#define FM10K_SW_STATS_UR FM10K_SW_REG_OFF(0x3801)
+#define FM10K_SW_STATS_CA FM10K_SW_REG_OFF(0x3802)
+#define FM10K_SW_STATS_UM FM10K_SW_REG_OFF(0x3803)
+#define FM10K_SW_STATS_XEC FM10K_SW_REG_OFF(0x3804)
+#define FM10K_SW_STATS_VLAN_DROP FM10K_SW_REG_OFF(0x3805)
+#define FM10K_SW_STATS_LOOPBACK_DROP FM10K_SW_REG_OFF(0x3806)
+#define FM10K_SW_STATS_NODESC_DROP FM10K_SW_REG_OFF(0x3807)
+#define FM10K_SW_RRTIME_CFG FM10K_SW_REG_OFF(0x3808)
+#define FM10K_SW_RRTIME_LIMIT(n_) \
+ FM10K_SW_REG_OFF(0x380C + 0x40 * (n_))
+#define FM10K_SW_RRTIME_COUNT(n_) \
+ FM10K_SW_REG_OFF(0x3810 + 0x40 * (n_))
+#define FM10K_SW_SYSTIME FM10K_SW_REG_OFF(0x3814)
+#define FM10K_SW_SYSTIME0 FM10K_SW_REG_OFF(0x3816)
+#define FM10K_SW_SYSTIME_CFG FM10K_SW_REG_OFF(0x3818)
+#define FM10K_SW_PFVFBME FM10K_SW_REG_OFF(0x381A)
+#define FM10K_SW_PHYADDR FM10K_SW_REG_OFF(0x381C)
+#define FM10K_SW_RDBAL(q_) \
+ FM10K_SW_REG_OFF(0x4000 + 0x40 * (q_))
+#define FM10K_SW_RDBAH(q_) \
+ FM10K_SW_REG_OFF(0x4001 + 0x40 * (q_))
+#define FM10K_SW_RDLEN(q_) \
+ FM10K_SW_REG_OFF(0x4002 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x4003 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_RXCTRL_HEADER_TPHEN (1 << 6)
+#define FM10K_SW_TPH_RXCTRL_PAYLOAD_TPHEN (1 << 7)
+#define FM10K_SW_TPH_RXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_RXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_RXCTRL_DATA_WRITE_RO_EN (1 << 13)
+#define FM10K_SW_TPH_RXCTRL_REP_HEADER_RO_EN (1 << 15)
+#define FM10K_SW_RDH(q_) \
+ FM10K_SW_REG_OFF(0x4004 + 0x40 * (q_))
+#define FM10K_SW_RDT(q_) \
+ FM10K_SW_REG_OFF(0x4005 + 0x40 * (q_))
+#define FM10K_SW_RDT_RDT_lsb 0
+#define FM10K_SW_RDT_RDT_msb 15
+#define FM10K_SW_RXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x4006 + 0x40 * (q_))
+#define FM10K_SW_RXQCTL_ENABLE (1 << 0)
+#define FM10K_SW_RXQCTL_VF_lsb 2
+#define FM10K_SW_RXQCTL_VF_msb 7
+#define FM10K_SW_RXQCTL_OWNED_BY_VF (1 << 8)
+#define FM10K_SW_RXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x4007 + 0x40 * (q_))
+#define FM10K_SW_RXDCTL_MAX_TIME_lsb 0
+#define FM10K_SW_RXDCTL_MAX_TIME_msb 7
+#define FM10K_SW_RXDCTL_WRITE_BACK_IMM (1 << 8)
+#define FM10K_SW_RXDCTL_DROP_ON_EMPTY (1 << 9)
+#define FM10K_SW_RXDCTL_WRITE_RSS_HASH (1 << 10)
+#define FM10K_SW_RXINT(q_) \
+ FM10K_SW_REG_OFF(0x4008 + 0x40 * (q_))
+#define FM10K_SW_RXINT_INT_lsb 0
+#define FM10K_SW_RXINT_INT_msb 7
+#define FM10K_SW_RXINT_INT_TIMER_lsb 8
+#define FM10K_SW_RXINT_INT_TIMER_msb 9
+#define FM10K_SW_SRRCTL(q_) \
+ FM10K_SW_REG_OFF(0x4009 + 0x40 * (q_))
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_lsb 0
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_msb 7
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_lsb 8
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_msb 13
+#define FM10K_SW_SRRCTL_DESC_TYPE_lsb 14
+#define FM10K_SW_SRRCTL_DESC_TYPE_msb 15
+#define FM10K_SW_SRRCTL_DESC_TYPE_NO_SPLIT 0
+#define FM10K_SW_SRRCTL_DESC_TYPE_HDR_SPLIT 1
+#define FM10K_SW_SRRCTL_DESC_TYPE_SIZE_SPLIT 2
+#define FM10K_SW_SRRCTL_PSR_TYPE_lsb 16
+#define FM10K_SW_SRRCTL_PSR_TYPE_msb 29
+#define FM10K_SW_SRRCTL_LOOPBACK_SUPPRESS (1 << 30)
+#define FM10K_SW_SRRCTL_BUFFER_CHAINING_EN (1 << 31)
+#define FM10K_SW_QPRC(q_) \
+ FM10K_SW_REG_OFF(0x400A + 0x40 * (q_))
+#define FM10K_SW_QPRDC(q_) \
+ FM10K_SW_REG_OFF(0x400B + 0x40 * (q_))
+#define FM10K_SW_QBRC_L(q_) \
+ FM10K_SW_REG_OFF(0x400C + 0x40 * (q_))
+#define FM10K_SW_QBRC_H(q_) \
+ FM10K_SW_REG_OFF(0x400D + 0x40 * (q_))
+#define FM10K_SW_RX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x400E + 0x40 * (q_))
+#define FM10K_SW_TDBAL(q_) \
+ FM10K_SW_REG_OFF(0x8000 + 0x40 * (q_))
+#define FM10K_SW_TDBAH(q_) \
+ FM10K_SW_REG_OFF(0x8001 + 0x40 * (q_))
+#define FM10K_SW_TDLEN(q_) \
+ FM10K_SW_REG_OFF(0x8002 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x8003 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_TXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_TXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_TXCTRL_DATA_READ_RO_EN (1 << 13)
+#define FM10K_SW_TDH(q_) \
+ FM10K_SW_REG_OFF(0x8004 + 0x40 * (q_))
+#define FM10K_SW_TDT(q_) \
+ FM10K_SW_REG_OFF(0x8005 + 0x40 * (q_))
+#define FM10K_SW_TDT_TDT_lsb 0
+#define FM10K_SW_TDT_TDT_msb 15
+#define FM10K_SW_TXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x8006 + 0x40 * (q_))
+#define FM10K_SW_TXDCTL_PTHRESH_lsb 0
+#define FM10K_SW_TXDCTL_PTHRESH_msb 6
+#define FM10K_SW_TXDCTL_HTHRESH_lsb 7
+#define FM10K_SW_TXDCTL_HTHRESH_msb 13
+#define FM10K_SW_TXDCTL_ENABLE (1 << 14)
+#define FM10K_SW_TXDCTL_MAX_TIME_lsb 16
+#define FM10K_SW_TXDCTL_MAX_TIME_msb 27
+#define FM10K_SW_TXDCTL_PUSH_DESC (1 << 28)
+#define FM10K_SW_TXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x8007 + 0x40 * (q_))
+#define FM10K_SW_TXQCTL_VF_lsb 0
+#define FM10K_SW_TXQCTL_VF_msb 5
+#define FM10K_SW_TXQCTL_OWNED_BY_VF (1 << 6)
+#define FM10K_SW_TXQCTL_PC_lsb 7
+#define FM10K_SW_TXQCTL_PC_msb 9
+#define FM10K_SW_TXQCTL_TC_lsb 10
+#define FM10K_SW_TXQCTL_TC_msb 15
+#define FM10K_SW_TXQCTL_VID_lsb 16
+#define FM10K_SW_TXQCTL_VID_msb 27
+#define FM10K_SW_TXQCTL_UNLIMITED_BW (1 << 28)
+#define FM10K_SW_TXQCTL_PUSH_MODE_DIS (1 << 29)
+#define FM10K_SW_TXINT(q_) \
+ FM10K_SW_REG_OFF(0x8008 + 0x40 * (q_))
+#define FM10K_SW_TXINT_INT_lsb 0
+#define FM10K_SW_TXINT_INT_msb 7
+#define FM10K_SW_TXINT_INT_TIMER_lsb 8
+#define FM10K_SW_TXINT_INT_TIMER_msb 9
+#define FM10K_SW_QPTC(q_) \
+ FM10K_SW_REG_OFF(0x8009 + 0x40 * (q_))
+#define FM10K_SW_QBTC_L(q_) \
+ FM10K_SW_REG_OFF(0x800A + 0x40 * (q_))
+#define FM10K_SW_QBTC_H(q_) \
+ FM10K_SW_REG_OFF(0x800B + 0x40 * (q_))
+#define FM10K_SW_TQDLOC(q_) \
+ FM10K_SW_REG_OFF(0x800C + 0x40 * (q_))
+#define FM10K_SW_TQDLOC_BASE_lsb 0
+#define FM10K_SW_TQDLOC_BASE_msb 15
+#define FM10K_SW_TQDLOC_SIZE_lsb 16
+#define FM10K_SW_TQDLOC_SIZE_msb 19
+#define FM10K_SW_TQDLOC_SIZE_32 5
+#define FM10K_SW_TQDLOC_SIZE_64 6
+#define FM10K_SW_TQDLOC_SIZE_128 7
+#define FM10K_SW_TX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x800D + 0x40 * (q_))
+#define FM10K_SW_TX_SGLORT_SGLORT_lsb 0
+#define FM10K_SW_TX_SGLORT_SGLORT_msb 15
+#define FM10K_SW_PFVTCTL(q_) \
+ FM10K_SW_REG_OFF(0x800E + 0x40 * (q_))
+#define FM10K_SW_TX_DESC(q_, d_, w_) \
+ FM10K_SW_REG_OFF(0x40000 + \
+ 0x400 * (q_) + 0x4 * (d_) + (w_))
+#define FM10K_SW_PBACL(n_) \
+ FM10K_SW_REG_OFF(0x10000 + (n_))
+#define FM10K_SW_INT_MAP(n_) \
+ FM10K_SW_REG_OFF(0x10080 + (n_))
+#define FM10K_SW_INT_MAP_ENTRIES 8
+#define FM10K_SW_INT_MAP_INDEX_MAILBOX 0
+#define FM10K_SW_INT_MAP_INDEX_FAULT 1
+#define FM10K_SW_INT_MAP_INDEX_SWITCH_UP_DOWN 2
+#define FM10K_SW_INT_MAP_INDEX_SWITCH 3
+#define FM10K_SW_INT_MAP_INDEX_SRAM 4
+#define FM10K_SW_INT_MAP_INDEX_VFLR 5
+#define FM10K_SW_INT_MAP_INDEX_MAX_HOLD_TIME 6
+#define FM10K_SW_INT_MAP_INT_lsb 0
+#define FM10K_SW_INT_MAP_INT_msb 7
+#define FM10K_SW_INT_MAP_INT_TIMER_lsb 8
+#define FM10K_SW_INT_MAP_INT_TIMER_msb 9
+#define FM10K_SW_MSIX_VECTOR(v_) \
+ FM10K_SW_REG_OFF(0x11000 + 0x4 * (v_))
+#define FM10K_SW_INT_CTRL \
+ FM10K_SW_REG_OFF(0x12000)
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_lsb 0
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_msb 9
+#define FM10K_SW_INT_CTRL_ENABLE_MODERATOR (1 << 10)
+#define FM10K_SW_ITR(v_) \
+ FM10K_SW_REG_OFF(0x12400 + (v_))
+
+/*
+ * Interrupt throttle timer intervals in microseconds. These provide the
+ * direct values for programming the ITR interval field when using a Gen3
+ * PCLK, otherwise they need to be scaled appropriately.
+ */
+#define FM10K_SW_ITR_INTERVAL_20K 50
+#define FM10K_SW_ITR_INTERVAL_40K 25
+#define FM10K_SW_ITR_INTERVAL_0_lsb 0
+#define FM10K_SW_ITR_INTERVAL_0_msb 11
+#define FM10K_SW_ITR_INTERVAL_1_lsb 12
+#define FM10K_SW_ITR_INTERVAL_1_msb 23
+#define FM10K_SW_ITR_TIMER_0_EXPIRED (1 << 24)
+#define FM10K_SW_ITR_TIMER_1_EXPIRED (1 << 25)
+#define FM10K_SW_ITR_PENDING_0 (1 << 26)
+#define FM10K_SW_ITR_PENDING_1 (1 << 27)
+#define FM10K_SW_ITR_PENDING_2 (1 << 28)
+#define FM10K_SW_ITR_AUTO_MASK (1 << 29)
+#define FM10K_SW_ITR_MASK_lsb 30
+#define FM10K_SW_ITR_MASK_msb 31
+#define FM10K_SW_ITR_MASK_R_ENABLED 0
+#define FM10K_SW_ITR_MASK_R_BLOCKED 1
+#define FM10K_SW_ITR_MASK_W_KEEP 0
+#define FM10K_SW_ITR_MASK_W_BLOCK 1
+#define FM10K_SW_ITR_MASK_W_ENABLE 2
+#define FM10K_SW_ITR2(v_) \
+ FM10K_SW_REG_OFF(0x12800 + 0x2 * (v_))
+#define FM10K_SW_IP FM10K_SW_REG_OFF(0x13000)
+#define FM10K_SW_IP_HOT_RESET (1 << 0)
+#define FM10K_SW_IP_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IP_MAILBOX (1 << 2)
+#define FM10K_SW_IP_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IP_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IP_PFLR (1 << 5)
+#define FM10K_SW_IP_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IP_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IP_NOT_IN_RESET (1 << 8)
+#define FM10K_SW_IP_TIMEOUT (1 << 9)
+#define FM10K_SW_IP_VFLR (1 << 10)
+#define FM10K_SW_IM FM10K_SW_REG_OFF(0x13001)
+#define FM10K_SW_IM_ALL FM10K_SW_MASK32(10, 0)
+#define FM10K_SW_IM_HOT_RESET (1 << 0)
+#define FM10K_SW_IM_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IM_MAILBOX (1 << 2)
+#define FM10K_SW_IM_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IM_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IM_PFLR (1 << 5)
+#define FM10K_SW_IM_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IM_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IM_TIMEOUT (1 << 9)
+#define FM10K_SW_IM_VFLR (1 << 10)
+#define FM10K_SW_IB FM10K_SW_REG_OFF(0x13002)
+#define FM10K_SW_SRAM_IP FM10K_SW_REG_OFF(0x13003)
+#define FM10K_SW_SRAM_IM FM10K_SW_REG_OFF(0x13004)
+#define FM10K_SW_VLAN_TABLE(f_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_))
+#define FM10K_SW_VLAN_TABLE_ENTRIES 128
+#define FM10K_SW_VLAN_TABLE_ENTRY(f_, n_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + (n_))
+#define FM10K_SW_VLAN_TABLE_VLAN_ENTRY(f_, vl_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + ((vl_) >> 5))
+#define FM10K_SW_VLAN_TABLE_VLAN_BIT(vl_) (1 << ((vl_) & 0x1f))
+#define FM10K_SW_MBMEM(n_) \
+ FM10K_SW_REG_OFF(0x18000 + (n_))
+#define FM10K_SW_MBX(vf_) \
+ FM10K_SW_REG_OFF(0x18800 + (vf_))
+#define FM10K_SW_MBICR(vf_) \
+ FM10K_SW_REG_OFF(0x18840 + ((vf_) >> 5))
+#define FM10K_SW_GMBX FM10K_SW_REG_OFF(0x18842)
+#define FM10K_SW_GMBX_GLOBAL_REQ (1 << 1)
+#define FM10K_SW_GMBX_GLOBAL_ACK (1 << 2)
+#define FM10K_SW_GMBX_PF_REQ_INTERRUPT (1 << 3)
+#define FM10K_SW_GMBX_PF_ACK_INTERRUPT (1 << 4)
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_lsb 5
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_msb 6
+#define FM10K_SW_GMBX_INTERRUPT_NO_CHANGE 0
+#define FM10K_SW_GMBX_INTERRUPT_ENABLE 1
+#define FM10K_SW_GMBX_INTERRUPT_DISABLE 2
+#define FM10K_SW_GMBX_PF_REQ (1 << 7)
+#define FM10K_SW_GMBX_PF_ACK (1 << 8)
+#define FM10K_SW_GMBX_GLOBAL_REQ_INTERRUPT (1 << 9)
+#define FM10K_SW_GMBX_GLOBAL_ACK_INTERRUPT (1 << 10)
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_lsb 11
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_msb 12
+#define FM10K_SW_PFVFLRE FM10K_SW_REG_OFF(0x18844)
+#define FM10K_SW_PFVFLREC FM10K_SW_REG_OFF(0x18846)
+#define FM10K_SW_TEST_CFG0 FM10K_SW_REG_OFF(0x18849)
+#define FM10K_SW_INT_SRAM_CTRL FM10K_SW_REG_OFF(0x18850)
+#define FM10K_SW_FUM_SRAM_CTRL FM10K_SW_REG_OFF(0x18852)
+#define FM10K_SW_PCA_SRAM_CTRL FM10K_SW_REG_OFF(0x18854)
+#define FM10K_SW_PP_SRAM_CTRL FM10K_SW_REG_OFF(0x18858)
+#define FM10K_SW_PCW_SRAM_CTRL FM10K_SW_REG_OFF(0x1885C)
+#define FM10K_SW_RHI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18872)
+#define FM10K_SW_RHI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18860)
+#define FM10K_SW_THI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18864)
+#define FM10K_SW_THI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18868)
+#define FM10K_SW_TIMEOUT_CFG FM10K_SW_REG_OFF(0x1886B)
+#define FM10K_SW_LVMMC FM10K_SW_REG_OFF(0x18880)
+#define FM10K_SW_LVMMI FM10K_SW_REG_OFF(0x18881)
+#define FM10K_SW_HOST_MISC FM10K_SW_REG_OFF(0x19000)
+#define FM10K_SW_HOST_LANE_CTRL FM10K_SW_REG_OFF(0x19001)
+#define FM10K_SW_SERDES_CTRL(l_) \
+ FM10K_SW_REG_OFF(0x19010 + 0x2 * (l_))
+
+/*
+ * BAR4 registers
+ */
+
+/*
+ * Access to non-master PEP registers via BAR4
+ */
+#define FM10K_SW_PCIE_GLOBAL(p_, r_) \
+ (FM10K_SW_REG_OFF(((p_) + 1) << 20) | FM10K_SW_##r_)
+
+/*
+ * SBUS register fields for use with both the PCIE and EPL SBUS interfaces.
+ */
+#define FM10K_SW_SBUS_CFG_SBUS_CONTROLLER_RESET (1 << 0)
+#define FM10K_SW_SBUS_CFG_ROM_ENABLE (1 << 1)
+#define FM10K_SW_SBUS_CFG_ROM_BUSY (1 << 2)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_PASS (1 << 3)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_FAIL (1 << 4)
+
+#define FM10K_SW_SBUS_COMMAND_REGISTER_lsb 0
+#define FM10K_SW_SBUS_COMMAND_REGISTER_msb 7
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_lsb 8
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_msb 15
+#define FM10K_SW_SBUS_COMMAND_OP_lsb 16
+#define FM10K_SW_SBUS_COMMAND_OP_msb 23
+#define FM10K_SW_SBUS_OP_RESET 0x20
+#define FM10K_SW_SBUS_OP_WRITE 0x21
+#define FM10K_SW_SBUS_OP_READ 0x22
+#define FM10K_SW_SBUS_COMMAND_EXECUTE (1 << 24)
+#define FM10K_SW_SBUS_COMMAND_BUSY (1 << 25)
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_lsb 26
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_msb 28
+#define FM10K_SW_SBUS_RESULT_RESET 0x00
+#define FM10K_SW_SBUS_RESULT_WRITE 0x01
+#define FM10K_SW_SBUS_RESULT_READ 0x04
+
+#define FM10K_SW_SBUS_ADDR_EPL_RMON2 1
+#define FM10K_SW_SBUS_ADDR_EPL_LANE(e_, l_) (4 * (e_) + (l_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_SERDES(s_) ((s_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_RMON3 38
+#define FM10K_SW_SBUS_ADDR_EPL_PMRO 39
+
+/* Common to both EPL and PCIE
+ */
+#define FM10K_SW_SBUS_ADDR_SPICO 253
+#define FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER 254
+#define FM10K_SW_SBUS_ADDR_BROADCAST 0xFF
+
+
+#define FM10K_SW_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x000000)
+#define FM10K_SW_MGMT_REG(wo_) \
+ (FM10K_SW_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_FATAL_CODE FM10K_SW_MGMT_REG(0x0)
+#define FM10K_SW_LAST_FATAL_CODE FM10K_SW_MGMT_REG(0x1)
+#define FM10K_SW_FATAL_COUNT FM10K_SW_MGMT_REG(0x2)
+#define FM10K_SW_SOFT_RESET FM10K_SW_MGMT_REG(0x3)
+#define FM10K_SW_SOFT_RESET_COLD_RESET (1 << 0)
+#define FM10K_SW_SOFT_RESET_EPL_RESET (1 << 1)
+#define FM10K_SW_SOFT_RESET_SWITCH_RESET (1 << 2)
+#define FM10K_SW_SOFT_RESET_SWITCH_READY (1 << 3)
+#define FM10K_SW_SOFT_RESET_PCIE_RESET(p_) (1 << ((p_) + 4))
+#define FM10K_SW_SOFT_RESET_PCIE_ACTIVE(p_) (1 << ((p_) + 13))
+#define FM10K_SW_DEVICE_CFG FM10K_SW_MGMT_REG(0x4)
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS 4
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(p_) (1 << (p_))
+#define FM10K_SW_DEVICE_CFG_PCIE_100G_DIS (1 << 4)
+#define FM10K_SW_DEVICE_CFG_FEATURE_lsb 5
+#define FM10K_SW_DEVICE_CFG_FEATURE_msb 6
+#define FM10K_SW_DEVICE_CFG_PCIE_FULL 0
+#define FM10K_SW_DEVICE_CFG_PCIE_HALF 1
+#define FM10K_SW_DEVICE_CFG_PCIE_BASIC 2
+#define FM10K_SW_DEVICE_CFG_PCIE_EN(p_) (1 << (7 + (p_)))
+#define FM10K_SW_RESET_CFG FM10K_SW_MGMT_REG(0x5)
+#define FM10K_SW_WATCHDOG_CFG FM10K_SW_MGMT_REG(0x6)
+#define FM10K_SW_MGMT_SCRATCH(n_) FM10K_SW_MGMT_REG(0x8 + (n_))
+#define FM10K_SW_VITAL_PRODUCT_DATA FM10K_SW_MGMT_REG(0x304)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x400)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_lsb64 0
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_msb64 8
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_lsb64 9
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_msb64 17
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_lsb64 18
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_msb64 26
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_lsb64 27
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_msb64 28
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CORE (1ULL << 29)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SOFTWARE (1ULL << 30)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_GPIO (1ULL << 31)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C (1ULL << 32)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_MDIO (1ULL << 33)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CRM (1ULL << 34)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_TAIL (1ULL << 35)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_HEAD (1ULL << 36)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PINS (1ULL << 39)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FIBM (1ULL << 40)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_BSM (1ULL << 41)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_INT \
+ FM10K_SW_MGMT_REG(0x402)
+#define FM10K_SW_INTERRUPT_MASK_PCIE(p_) \
+ FM10K_SW_MGMT_REG(0x420 + 0x2 * (p_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_lsb64 0
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_msb64 8
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_lsb64 9
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_msb64 17
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_lsb64 18
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_msb64 26
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_lsb64 27
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_msb64 28
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CORE (1ULL << 29)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SOFTWARE (1ULL << 30)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_GPIO (1ULL << 31)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_I2C (1ULL << 32)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_MDIO (1ULL << 33)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CRM (1ULL << 34)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_TAIL (1ULL << 35)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_HEAD (1ULL << 36)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PINS (1ULL << 39)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FIBM (1ULL << 40)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_BSM (1ULL << 41)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_ALL FM10K_SW_MASK64(42, 0)
+#define FM10K_SW_INTERRUPT_MASK_FIBM FM10K_SW_MGMT_REG(0x440)
+#define FM10K_SW_INTERRUPT_MASK_BSM FM10K_SW_MGMT_REG(0x442)
+#define FM10K_SW_CORE_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x444)
+#define FM10K_SW_CORE_INTERRUPT_MASK FM10K_SW_MGMT_REG(0x445)
+#define FM10K_SW_SRAM_ERR_IP FM10K_SW_MGMT_REG(0x446)
+#define FM10K_SW_SRAM_ERR_IM FM10K_SW_MGMT_REG(0x448)
+#define FM10K_SW_PINS_STAT FM10K_SW_MGMT_REG(0x44A)
+#define FM10K_SW_PINS_IP FM10K_SW_MGMT_REG(0x44B)
+#define FM10K_SW_PINS_IM FM10K_SW_MGMT_REG(0x44C)
+#define FM10K_SW_SW_IP FM10K_SW_MGMT_REG(0x44D)
+#define FM10K_SW_SW_IM FM10K_SW_MGMT_REG(0x44E)
+#define FM10K_SW_SW_TEST_AND_SET FM10K_SW_MGMT_REG(0x44F)
+#define FM10K_SW_LSM_CLKOBS_CTRL FM10K_SW_MGMT_REG(0x450)
+#define FM10K_SW_CHIP_VERSION FM10K_SW_MGMT_REG(0x452)
+#define FM10K_SW_BSM_SCRATCH(n_) FM10K_SW_MGMT_REG(0x800 + (n_))
+#define FM10K_SW_BSM_CTRL FM10K_SW_MGMT_REG(0xC00)
+#define FM10K_SW_BSM_ARGS FM10K_SW_MGMT_REG(0xC01)
+#define FM10K_SW_BSM_ADDR_OFFSET(n_) FM10K_SW_MGMT_REG(0xC04 + (n_))
+#define FM10K_SW_BSM_COUNTER(n_) FM10K_SW_MGMT_REG(0xC08 + (n_))
+#define FM10K_SW_BSM_SRAM_CTRL FM10K_SW_MGMT_REG(0xC0A)
+#define FM10K_SW_BSM_IP FM10K_SW_MGMT_REG(0xC0B)
+#define FM10K_SW_BSM_IM FM10K_SW_MGMT_REG(0xC0C)
+#define FM10K_SW_PIN_STRAP_STAT FM10K_SW_MGMT_REG(0xC0D)
+#define FM10K_SW_FUSE_DATA_0 FM10K_SW_MGMT_REG(0xC0E)
+#define FM10K_SW_FUSE_SKU_lsb 11
+#define FM10K_SW_FUSE_SKU_msb 15
+#define FM10K_SW_FUSE_SKU_FM10840 0
+#define FM10K_SW_FUSE_SKU_FM10420 1
+#define FM10K_SW_FUSE_SKU_FM10064 2
+#define FM10K_SW_FUSE_DATA_1 FM10K_SW_MGMT_REG(0xC0F)
+#define FM10K_SW_BIST_CTRL FM10K_SW_MGMT_REG(0xC10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_MODE_EPL (1ULL << 41)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC (1ULL << 42)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL (1ULL << 43)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_BSM (1ULL << 44)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_CRM (1ULL << 45)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FIBM (1ULL << 46)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_SBM (1ULL << 47)
+#define FM10K_SW_REI_CTRL FM10K_SW_MGMT_REG(0xC12)
+#define FM10K_SW_REI_STAT FM10K_SW_MGMT_REG(0xC13)
+#define FM10K_SW_GPIO_CFG FM10K_SW_MGMT_REG(0xC15)
+#define FM10K_SW_GPIO_DATA FM10K_SW_MGMT_REG(0xC16)
+#define FM10K_SW_GPIO_IP FM10K_SW_MGMT_REG(0xC17)
+#define FM10K_SW_GPIO_IM FM10K_SW_MGMT_REG(0xC18)
+#define FM10K_SW_I2C_CFG FM10K_SW_MGMT_REG(0xC19)
+#define FM10K_SW_I2C_CFG_SLAVE_ENABLE (1 << 0)
+#define FM10K_SW_I2C_CFG_ADDR_lsb 1
+#define FM10K_SW_I2C_CFG_ADDR_msb 7
+#define FM10K_SW_I2C_CFG_DIVIDER_lsb 8
+#define FM10K_SW_I2C_CFG_DIVIDER_msb 19
+#define FM10K_SW_I2C_CFG_DIVIDER_100_KHZ 52
+#define FM10K_SW_I2C_CFG_DIVIDER_400_KHZ 10
+#define FM10K_SW_I2C_CFG_INTERRUPT_MASK (1 << 20)
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_lsb 21
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_msb 27
+#define FM10K_SW_I2C_DATA(n_) FM10K_SW_MGMT_REG(0xC1C + (n_))
+#define FM10K_SW_I2C_CTRL FM10K_SW_MGMT_REG(0xC20)
+#define FM10K_SW_I2C_CTRL_ADDR_lsb 0
+#define FM10K_SW_I2C_CTRL_ADDR_msb 7
+#define FM10K_SW_I2C_CTRL_COMMAND_lsb 8
+#define FM10K_SW_I2C_CTRL_COMMAND_msb 9
+#define FM10K_SW_I2C_COMMAND_NULL 0
+#define FM10K_SW_I2C_COMMAND_WR 1
+#define FM10K_SW_I2C_COMMAND_WR_RD 2
+#define FM10K_SW_I2C_COMMAND_RD 3
+#define FM10K_SW_I2C_CTRL_LENGTH_W_lsb 10
+#define FM10K_SW_I2C_CTRL_LENGTH_W_msb 13
+#define FM10K_SW_I2C_CTRL_LENGTH_R_lsb 14
+#define FM10K_SW_I2C_CTRL_LENGTH_R_msb 17
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_lsb 18
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_msb 21
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_lsb 22
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_msb 25
+#define FM10K_SW_I2C_COMPLETION_RUNNING 0
+#define FM10K_SW_I2C_COMPLETION_NORMAL 1
+#define FM10K_SW_I2C_COMPLETION_PREMATURE 2
+#define FM10K_SW_I2C_COMPLETION_NO_DEVICE 3
+#define FM10K_SW_I2C_COMPLETION_TIMEOUT 4
+#define FM10K_SW_I2C_COMPLETION_LOST_ARB 5
+#define FM10K_SW_I2C_COMPLETION_BUS_WAIT 6
+#define FM10K_SW_I2C_COMPLETION_INVALID 7
+#define FM10K_SW_I2C_CTRL_INTERRUPT_PENDING (1 << 26)
+#define FM10K_SW_MDIO_CFG FM10K_SW_MGMT_REG(0xC22)
+#define FM10K_SW_MDIO_DATA FM10K_SW_MGMT_REG(0xC23)
+#define FM10K_SW_MDIO_CTRL FM10K_SW_MGMT_REG(0xC24)
+#define FM10K_SW_SPI_TX_DATA FM10K_SW_MGMT_REG(0xC26)
+#define FM10K_SW_SPI_RX_DATA FM10K_SW_MGMT_REG(0xC27)
+#define FM10K_SW_SPI_HEADER FM10K_SW_MGMT_REG(0xC28)
+#define FM10K_SW_SPI_CTRL FM10K_SW_MGMT_REG(0xC29)
+#define FM10K_SW_LED_CFG FM10K_SW_MGMT_REG(0xC2B)
+#define FM10K_SW_SCAN_DATA_IN FM10K_SW_MGMT_REG(0xC2D)
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_lsb 0
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_msb 24
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
+#define FM10K_SW_SCAN_DATA_IN_INJECT (1 << 28)
+#define FM10K_SW_SCAN_DATA_IN_DRAIN (1 << 29)
+#define FM10K_SW_SCAN_DATA_IN_PASSTHRU (1 << 30)
+#define FM10K_SW_SCAN_DATA_IN_SINGLE (1 << 31)
+#define FM10K_SW_CRM_DATA(m_, n_) \
+ FM10K_SW_MGMT_REG(0x1000 + 0x2 * (m_) + (n_))
+
+#define FM10K_SW_CRM_CTRL FM10K_SW_MGMT_REG(0x2000)
+#define FM10K_SW_CRM_STATUS FM10K_SW_MGMT_REG(0x2001)
+#define FM10K_SW_CRM_TIME FM10K_SW_MGMT_REG(0x2002)
+#define FM10K_SW_CRM_SRAM_CTRL FM10K_SW_MGMT_REG(0x2004)
+#define FM10K_SW_CRM_IP FM10K_SW_MGMT_REG(0x2008)
+#define FM10K_SW_CRM_IM FM10K_SW_MGMT_REG(0x200C)
+#define FM10K_SW_CRM_COMMAND(n_) FM10K_SW_MGMT_REG(0x2080 + 0x2 * (n_))
+#define FM10K_SW_CRM_REGISTER(n_) FM10K_SW_MGMT_REG(0x2100 + 0x2 * (n_))
+#define FM10K_SW_CRM_PERIOD(n_) FM10K_SW_MGMT_REG(0x2180 + 0x2 * (n_))
+#define FM10K_SW_CRM_PARAM(n_) FM10K_SW_MGMT_REG(0x2200 + (n_))
+#define FM10K_SW_PLL_PCIE_CTRL FM10K_SW_MGMT_REG(0x2241)
+#define FM10K_SW_PLL_PCIE_STAT FM10K_SW_MGMT_REG(0x2242)
+#define FM10K_SW_SBUS_PCIE_CFG FM10K_SW_MGMT_REG(0x2243)
+#define FM10K_SW_SBUS_PCIE_COMMAND FM10K_SW_MGMT_REG(0x2244)
+#define FM10K_SW_SBUS_PCIE_REQUEST FM10K_SW_MGMT_REG(0x2245)
+#define FM10K_SW_SBUS_PCIE_RESPONSE FM10K_SW_MGMT_REG(0x2246)
+#define FM10K_SW_SBUS_PCIE_SPICO_IN FM10K_SW_MGMT_REG(0x2247)
+#define FM10K_SW_SBUS_PCIE_SPICO_OUT FM10K_SW_MGMT_REG(0x2248)
+#define FM10K_SW_SBUS_PCIE_IP FM10K_SW_MGMT_REG(0x2249)
+#define FM10K_SW_SBUS_PCIE_IM FM10K_SW_MGMT_REG(0x224A)
+#define FM10K_SW_MGMT_SYSTIME_CFG FM10K_SW_MGMT_REG(0x224C)
+#define FM10K_SW_MGMT_SYSTIME FM10K_SW_MGMT_REG(0x224E)
+#define FM10K_SW_MGMT_SYSTIME0 FM10K_SW_MGMT_REG(0x2250)
+#define FM10K_SW_SYSTIME_PULSE_(n_) FM10K_SW_MGMT_REG(0x2252 + (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_LO(n_) FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_HI(n_) \
+ FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_) + 0x1)
+#define FM10K_SW_PCIE_XPLL_CTRL FM10K_SW_MGMT_REG(0x3000)
+#define FM10K_SW_PCIE_CLK_CTRL FM10K_SW_MGMT_REG(0x3001)
+#define FM10K_SW_PCIE_CLK_CTRL2 FM10K_SW_MGMT_REG(0x3002)
+#define FM10K_SW_PCIE_CLKMON_RATIO_CFG FM10K_SW_MGMT_REG(0x3003)
+#define FM10K_SW_PCIE_CLKMON_TOLERANCE_CFG FM10K_SW_MGMT_REG(0x3004)
+#define FM10K_SW_PCIE_CLKMON_DEADLINES_CFG FM10K_SW_MGMT_REG(0x3005)
+#define FM10K_SW_PCIE_CLK_STAT FM10K_SW_MGMT_REG(0x3006)
+#define FM10K_SW_PCIE_CLK_IP FM10K_SW_MGMT_REG(0x3007)
+#define FM10K_SW_PCIE_CLK_IM FM10K_SW_MGMT_REG(0x3008)
+#define FM10K_SW_PCIE_WARM_RESET_DELAY FM10K_SW_MGMT_REG(0x3009)
+#define FM10K_SW_EPL_BASE FM10K_SW_REG_OFF(0x0E0000)
+#define FM10K_SW_EPL_PORT_REG(p_, r_) \
+ (FM10K_SW_EPL_BASE + FM10K_SW_REG_OFF(0x400 * (p_) + (r_)))
+#define FM10K_SW_EPL_LANE_REG(p_, l_, r_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x80 * (l_) + (r_))
+
+/* EPL enumerated types
+ */
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX 0
+#define FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1 1
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_L4 2
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_L4_XX 3
+#define FM10K_SW_EPL_QPL_MODE_XX_L4_XX_XX 4
+#define FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX 5
+#define FM10K_SW_EPL_PCS_SEL_DISABLE 0
+#define FM10K_SW_EPL_PCS_SEL_AN_73 1
+#define FM10K_SW_EPL_PCS_SEL_SGMII_10 2
+#define FM10K_SW_EPL_PCS_SEL_SGMII_100 3
+#define FM10K_SW_EPL_PCS_SEL_SGMII_1000 4
+#define FM10K_SW_EPL_PCS_SEL_1000BASEX 5
+#define FM10K_SW_EPL_PCS_SEL_10GBASER 6
+#define FM10K_SW_EPL_PCS_SEL_40GBASER 7
+#define FM10K_SW_EPL_PCS_SEL_100GBASER 8
+
+#define FM10K_SW_LANE_OVERRIDE_NORMAL 0
+#define FM10K_SW_LANE_OVERRIDE_FORCE_GOOD 1
+#define FM10K_SW_LANE_OVERRIDE_FORCE_BAD 2
+
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSTHRU 0
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSSTHRU_CHECK 1
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_GOOD 2
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_BAD 3
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL 4
+
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_DRAIN 0
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL 1
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_NORMAL 2
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_HOLD 3
+
+#define FM10K_SW_TX_MAC_FAULT_MODE_NORMAL 0
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_IDLE 1
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LOCAL_FAULT 2
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_REMOTE_FAULT 3
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LINK_INTERRUPTION 4
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_OK 5
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_ERROR 6
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_USER_VAL 7
+
+#define FM10K_SW_EPL_IP(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x300)
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_lsb 0
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_msb 3
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT(n_) (1 << ((n_) + 0))
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_lsb 4
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_msb 7
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT(n_) (1 << ((n_) + 4))
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_lsb 8
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_msb 11
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT(n_) (1 << ((n_) + 8))
+#define FM10K_SW_EPL_IP_ERROR_INTERRUPT (1 << 12)
+
+#define FM10K_SW_EPL_ERROR_IP(p_) FM10K_SW_EPL_PORT_REG((p_), 0x301)
+#define FM10K_SW_EPL_ERROR_IM(p_) FM10K_SW_EPL_PORT_REG((p_), 0x302)
+#define FM10K_SW_EPL_ERROR_IM_ALL FM10K_SW_MASK32(8, 0)
+#define FM10K_SW_EPL_BIST_STATUS(p_) FM10K_SW_EPL_PORT_REG((p_), 0x303)
+#define FM10K_SW_EPL_CFG_A(p_) FM10K_SW_EPL_PORT_REG((p_), 0x304)
+#define FM10K_SW_EPL_CFG_A_SPEED_UP (1 << 0)
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_lsb 1
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_msb 6
+#define FM10K_SW_EPL_CFG_A_ACTIVE(p_) (1 << ((p_) + 7))
+#define FM10K_SW_EPL_CFG_A_ACTIVE_QUAD (0xf << 7)
+
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_lsb 11
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_msb 16
+
+#define FM10K_SW_EPL_CFG_B(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x305)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_lsb(p_) ((p_) * 4)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_msb(p_) (((p_) * 4) + 3)
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_lsb 16
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_msb 18
+#define FM10K_SW_EPL_LED_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x306)
+
+#define FM10K_SW_EPL_LED_STATUS_PORT_RESET(n_) (1 << ((n_) * 6 + 0))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP(n_) (1 << ((n_) * 6 + 1))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LOCAL_FAULT(n_) (1 << ((n_) * 6 + 2))
+#define FM10K_SW_EPL_LED_STATUS_PORT_REMOTE_FAULT(n_) (1 << ((n_) * 6 + 3))
+#define FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING(n_) (1 << ((n_) * 6 + 4))
+#define FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING(n_) (1 << ((n_) * 6 + 5))
+
+#define FM10K_SW_EPL_FIFO_ERROR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x307)
+#define FM10K_SW_EPL_TX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x308)
+#define FM10K_SW_EPL_TX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x309)
+#define FM10K_SW_EPL_TX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30A)
+#define FM10K_SW_EPL_TX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30B)
+#define FM10K_SW_EPL_TX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30C)
+#define FM10K_SW_EPL_TX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30D)
+#define FM10K_SW_EPL_RX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30E)
+#define FM10K_SW_EPL_RX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30F)
+#define FM10K_SW_EPL_RX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x310)
+#define FM10K_SW_EPL_RX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x311)
+#define FM10K_SW_EPL_RX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x312)
+#define FM10K_SW_EPL_RX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x313)
+#define FM10K_SW_PCS_ML_BASER_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x314)
+#define FM10K_SW_PCS_ML_BASER_RX_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x315)
+#define FM10K_SW_PCS_100GBASER_BIP_0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x317)
+#define FM10K_SW_PCS_100GBASER_BIP_1(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x318)
+#define FM10K_SW_PCS_100GBASER_BIP_2(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x319)
+#define FM10K_SW_PCS_100GBASER_BIP_3(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31A)
+#define FM10K_SW_PCS_100GBASER_BIP_4(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31B)
+#define FM10K_SW_PCS_100GBASER_BIP_5(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31C)
+#define FM10K_SW_PCS_100GBASER_BIP_6(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31D)
+#define FM10K_SW_PCS_100GBASER_BIP_7(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31E)
+#define FM10K_SW_PCS_100GBASER_BIP_8(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31F)
+#define FM10K_SW_PCS_100GBASER_BIP_9(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x320)
+#define FM10K_SW_PCS_100GBASER_BIP_10(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x321)
+#define FM10K_SW_PCS_100GBASER_BIP_11(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x322)
+#define FM10K_SW_PCS_100GBASER_BIP_12(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x323)
+#define FM10K_SW_PCS_100GBASER_BIP_13(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x324)
+#define FM10K_SW_PCS_100GBASER_BIP_14(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x325)
+#define FM10K_SW_PCS_100GBASER_BIP_15(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x326)
+#define FM10K_SW_PCS_100GBASER_BIP_16(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x327)
+#define FM10K_SW_PCS_100GBASER_BIP_17(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x328)
+#define FM10K_SW_PCS_100GBASER_BIP_18(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x329)
+#define FM10K_SW_PCS_100GBASER_BIP_19(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32A)
+#define FM10K_SW_PCS_100GBASER_BLOCK_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32B)
+#define FM10K_SW_PCS_100GBASER_AMPS_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32C)
+#define FM10K_SW_RS_FEC_UNCORRECTED(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x33D)
+#define FM10K_SW_RS_FEC_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32E)
+#define FM10K_SW_RS_FEC_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x330)
+#define FM10K_SW_EPL_SYSTIME(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x331)
+#define FM10K_SW_EPL_SYSTIME0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x332)
+#define FM10K_SW_EPL_SYSTIME_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x333)
+#define FM10K_SW_PORT_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x0)
+#define FM10K_SW_AN_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1)
+
+#define FM10K_SW_AN_IM_ALL FM10K_SW_MASK32(18, 0)
+#define FM10K_SW_LINK_IM(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2)
+#define FM10K_SW_LINK_IM_ALL FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_AN_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3)
+#define FM10K_SW_LINK_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4)
+#define FM10K_SW_MP_CFG(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_73_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_LINK_RULES(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xC)
+
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_lsb 0
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_msb 3
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_lsb 4
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_msb 8
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_lsb 9
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_msb 12
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_lsb 13
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_msb 17
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_lsb 18
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_msb 21
+#define FM10K_SW_MAC_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x10)
+#define FM10K_SW_MAC_CFG_ARRAY_SIZE 8
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_lsb 0
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_msb 5
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_lsb 6
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_msb 9
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_lsb 10
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_msb 17
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_lsb 18
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_msb 25
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_lsb 26
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_msb 31
+
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_lsb 32
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_msb 47
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE_bit 48
+
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_lsb 49
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_msb 51
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_lsb 52
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_msb 55
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_lsb 56
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_msb 63
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_lsb 64
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_msb 65
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_lsb 66
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_msb 71
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_lsb 72
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_msb 83
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_lsb 84
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_msb 95
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_lsb 96
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_msb 107
+#define FM10K_SW_MAC_CFG_TX_LP_IDLE_REQUEST_bit 108
+#define FM10K_SW_MAC_CFG_TX_LPI_AUTOMATIC_bit 109
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_lsb 110
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_msb 117
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_lsb 118
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_msb 119
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_lsb 120
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_msb 127
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_lsb 128
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_msb 129
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_lsb 130
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_msb 132
+#define FM10K_SW_MAC_CFG_TX_OBEY_LINT_bit 133
+#define FM10K_SW_MAC_CFG_TX_IDLE_ENABLE_DIC_bit 134
+#define FM10K_SW_MAC_CFG_CJPAT_ENABLE_bit 135
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_lsb 136
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_msb 143
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_lsb 144
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_msb 159
+#define FM10K_SW_MAC_CFG_START_CHAR_D_lsb 160
+#define FM10K_SW_MAC_CFG_START_CHAR_D_msb 167
+#define FM10K_SW_MAC_CFG_IEEE_1588_ENABLE_bit 168
+#define FM10K_SW_MAC_CFG_FCS_START_bit 169
+#define FM10K_SW_MAC_CFG_PREAMBLE_MODE_bit 170
+#define FM10K_SW_MAC_CFG_COUNTER_WRAP_bit 171
+#define FM10K_SW_MAC_CFG_LINK_FAULT_DISABLE_bit 172
+#define FM10K_SW_MAC_CFG_RX_DRAIN_bit 173
+#define FM10K_SW_MAC_CFG_RX_FCS_FORCE_BAD_bit 174
+#define FM10K_SW_MAC_CFG_RX_IGNORE_CODE_ERRORS_bit 175
+#define FM10K_SW_MAC_CFG_RX_IGNORE_UNDERSIZE_ERRORS_bit 176
+#define FM10K_SW_MAC_CFG_RX_IGNORE_OVERSIZE_ERRORS_bit 177
+#define FM10K_SW_MAC_CFG_RX_IGNORE_FCS_ERRORS_bit 178
+#define FM10K_SW_MAC_CFG_RX_IGNORE_PREAMBLE_ERRORS_bit 179
+#define FM10K_SW_MAC_CFG_RX_IGNORE_IFG_ERRORS_bit 180
+#define FM10K_SW_MAC_CFG_ERR_WRITE_lsb 181
+#define FM10K_SW_MAC_CFG_ERR_WRITE_msb 182
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_lsb 183
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_msb 186
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_lsb 187
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_msb 191
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_lsb 192
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_msb 196
+
+#define FM10K_SW_TX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1A)
+#define FM10K_SW_RX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1C)
+#define FM10K_SW_MAC_1588_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1E)
+#define FM10K_SW_WAKE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x20)
+#define FM10K_SW_MAC_OVERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x21)
+#define FM10K_SW_MAC_JABBER_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x22)
+#define FM10K_SW_MAC_UNDERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x23)
+#define FM10K_SW_MAC_RUNT_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x24)
+#define FM10K_SW_MAC_OVERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x25)
+#define FM10K_SW_MAC_UNDERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x26)
+#define FM10K_SW_MAC_CODE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x27)
+#define FM10K_SW_EPL_TX_FRAME_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x28)
+#define FM10K_SW_MAC_LINK_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x29)
+#define FM10K_SW_PCS_1000BASEX_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2A)
+#define FM10K_SW_PCS_1000BASEX_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2B)
+#define FM10K_SW_PCS_1000BASEX_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2C)
+#define FM10K_SW_PCS_10GBASER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2D)
+#define FM10K_SW_PCS_10GBASER_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2E)
+#define FM10K_SW_PCS_10GBASER_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2F)
+#define FM10K_SW_AN_37_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x30)
+#define FM10K_SW_AN_37_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_37_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_SGMII_AN_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_SGMII_AN_TX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_TX_CONFIG_LOOPBACK(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_RX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x32)
+#define FM10K_SW_AN_73_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x33)
+#define FM10K_SW_AN_73_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_73_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x36)
+#define FM10K_SW_AN_73_TX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x38)
+#define FM10K_SW_AN_73_RX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3A)
+#define FM10K_SW_PCSL_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3C)
+
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_lsb 0
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_msb 7
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE (1 << 8)
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL (1 << 9)
+#define FM10K_SW_PCSL_CFG_RX_GB_NARROW (1 << 10)
+#define FM10K_SW_PCSL_CFG_TX_GB_NARROW (1 << 11)
+
+#define FM10K_SW_MP_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_1000BASEX_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_10GBASER_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_MP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_40GBASER_RX_BIP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_10GBASER_RX_BER_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_DISPARITY_ERROR_8B10B(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_LANE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x45)
+#define FM10K_SW_LANE_SERDES_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x46)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x47)
+
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_msb 1
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT (1 << 5)
+
+#define FM10K_SW_LANE_ACTIVITY_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x48)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x49)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_msb 1
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT (1 << 5)
+#define FM10K_SW_LANE_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4A)
+#define FM10K_SW_LANE_SERDES_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4B)
+
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_lsb 0
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_msb 7
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_lsb 8
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_msb 23
+#define FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK (1 << 12)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_RDY (1 << 24)
+#define FM10K_SW_LANE_SERDES_STATUS_TX_RDY (1 << 25)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_IDLE_DETECT (1 << 26)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_ACTIVITY (1 << 27)
+#define FM10K_SW_SERDES_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4C)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IM_CORE_STATUS_4
+#define FM10K_SW_SERDES_IM_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IM_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IM_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IM_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IM_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IM_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IM_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IM_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IM_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IM_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IM_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_SERDES_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4D)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IP_CORE_STATUS_4
+#define FM10K_SW_SERDES_IP_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IP_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IP_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IP_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IP_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IP_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IP_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IP_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IP_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IP_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IP_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_LANE_ANALOG_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4E)
+#define FM10K_SW_LANE_ANALOG_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4F)
+#define FM10K_SW_LANE_SAI_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x50)
+#define FM10K_SW_LANE_SAI_CFG_CODE_lsb64 0
+#define FM10K_SW_LANE_SAI_CFG_CODE_msb64 15
+#define FM10K_SW_LANE_SAI_CFG_DATA_lsb64 16
+#define FM10K_SW_LANE_SAI_CFG_DATA_msb64 31
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_lsb64 32
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_msb64 47
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_lsb64 48
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_msb64 49
+#define FM10K_SW_LANE_SAI_CFG_REQUEST (1ULL << 50)
+#define FM10K_SW_LANE_SAI_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x52)
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_lsb 0
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_msb 15
+#define FM10K_SW_LANE_SAI_STATUS_COMPLETE (1 << 16)
+#define FM10K_SW_LANE_SAI_STATUS_ACCESS_REQUEST (1 << 17)
+#define FM10K_SW_LANE_SAI_STATUS_IN_PROGRESS (1 << 18)
+#define FM10K_SW_LANE_SAI_STATUS_BUSY (1 << 19)
+#define FM10K_SW_LANE_SAI_STATUS_REQUEST_ERROR (1 << 20)
+
+#define FM10K_SW_PORTS_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x0E8000)
+#define FM10K_SW_PORTS_MGMT_REG(wo_) \
+ (FM10K_SW_PORTS_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_PLL_EPL_CTRL FM10K_SW_PORTS_MGMT_REG(0x0)
+#define FM10K_SW_PLL_EPL_STAT FM10K_SW_PORTS_MGMT_REG(0x1)
+#define FM10K_SW_PLL_FABRIC_CTRL FM10K_SW_PORTS_MGMT_REG(0x2)
+
+#define FM10K_SW_PLL_FABRIC_REFDIV_lsb 3
+#define FM10K_SW_PLL_FABRIC_REFDIV_msb 8
+#define FM10K_SW_PLL_FABRIC_FBDIV4_lsb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV4_msb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV255_lsb 10
+#define FM10K_SW_PLL_FABRIC_FBDIV255_msb 17
+#define FM10K_SW_PLL_FABRIC_OUTDIV_lsb 18
+#define FM10K_SW_PLL_FABRIC_OUTDIV_msb 23
+#define FM10K_SW_PLL_FABRIC_STAT FM10K_SW_PORTS_MGMT_REG(0x3)
+#define FM10K_SW_PLL_FABRIC_LOCK FM10K_SW_PORTS_MGMT_REG(0x4)
+#define FM10K_SW_PLL_FABRIC_FREQSEL_lsb 4
+#define FM10K_SW_PLL_FABRIC_FREQSEL_msb 7
+#define FM10K_SW_PLL_FABRIC_FREQSEL_CTRL 0
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F600 1
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F500 2
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F400 3
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F300 4
+#define FM10K_SW_SBUS_EPL_CFG FM10K_SW_PORTS_MGMT_REG(0x5)
+#define FM10K_SW_SBUS_EPL_COMMAND FM10K_SW_PORTS_MGMT_REG(0x6)
+#define FM10K_SW_SBUS_EPL_REQUEST FM10K_SW_PORTS_MGMT_REG(0x7)
+#define FM10K_SW_SBUS_EPL_RESPONSE FM10K_SW_PORTS_MGMT_REG(0x8)
+#define FM10K_SW_SBUS_EPL_SPICO_IN FM10K_SW_PORTS_MGMT_REG(0x9)
+#define FM10K_SW_SBUS_EPL_SPICO_OUT FM10K_SW_PORTS_MGMT_REG(0xA)
+#define FM10K_SW_SBUS_EPL_IP FM10K_SW_PORTS_MGMT_REG(0xB)
+#define FM10K_SW_SBUS_EPL_IM FM10K_SW_PORTS_MGMT_REG(0xC)
+
+#define FM10K_SW_ETHCLK_CFG(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0xE + (n_))
+#define FM10K_SW_ETHCLK_RATIO(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0x10 + (n_))
+#define FM10K_SW_PM_CLKOBS_CTRL FM10K_SW_PORTS_MGMT_REG(0x12)
+
+#define FM10K_SW_PCIE_CFG_BASE FM10K_SW_REG_OFF(0x120000)
+#define FM10K_SW_PCIE_CFG_REG(wo_) \
+ (FM10K_SW_PCIE_CFG_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * These register offsets can also be passed to fm10k_read_config(), which
+ * will mask off the upper bits
+ */
+#define FM10K_SW_PCIE_CFG_ID FM10K_SW_PCIE_CFG_REG(0x0)
+#define FM10K_SW_PCIE_CFG_CMD FM10K_SW_PCIE_CFG_REG(0x1)
+#define FM10K_SW_PCIE_CFG_1 FM10K_SW_PCIE_CFG_REG(0x2)
+#define FM10K_SW_PCIE_CFG_2 FM10K_SW_PCIE_CFG_REG(0x3)
+#define FM10K_SW_PCIE_CFG_BAR0 FM10K_SW_PCIE_CFG_REG(0x4)
+#define FM10K_SW_PCIE_CFG_BAR1 FM10K_SW_PCIE_CFG_REG(0x5)
+#define FM10K_SW_PCIE_CFG_BAR2 FM10K_SW_PCIE_CFG_REG(0x6)
+#define FM10K_SW_PCIE_CFG_BAR3 FM10K_SW_PCIE_CFG_REG(0x7)
+#define FM10K_SW_PCIE_CFG_BAR4 FM10K_SW_PCIE_CFG_REG(0x8)
+#define FM10K_SW_PCIE_CFG_BAR5 FM10K_SW_PCIE_CFG_REG(0x9)
+#define FM10K_SW_PCIE_CFG_CARDBUS FM10K_SW_PCIE_CFG_REG(0xA)
+#define FM10K_SW_PCIE_CFG_SUBID FM10K_SW_PCIE_CFG_REG(0xB)
+#define FM10K_SW_PCIE_CFG_EXP_ROM FM10K_SW_PCIE_CFG_REG(0xC)
+#define FM10K_SW_PCIE_CFG_CAP_PTR FM10K_SW_PCIE_CFG_REG(0xD)
+#define FM10K_SW_PCIE_CFG_RSVD FM10K_SW_PCIE_CFG_REG(0xE)
+#define FM10K_SW_PCIE_CFG_INT FM10K_SW_PCIE_CFG_REG(0xF)
+#define FM10K_SW_PCIE_CFG_PM_CAP FM10K_SW_PCIE_CFG_REG(0x10)
+#define FM10K_SW_PCIE_CFG_PM_CTRL FM10K_SW_PCIE_CFG_REG(0x11)
+#define FM10K_SW_PCIE_CFG_PCIE_CAP FM10K_SW_PCIE_CFG_REG(0x1C)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP FM10K_SW_PCIE_CFG_REG(0x1D)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL FM10K_SW_PCIE_CFG_REG(0x1E)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CAP FM10K_SW_PCIE_CFG_REG(0x1F)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL FM10K_SW_PCIE_CFG_REG(0x20)
+
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_lsb 16
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_msb 19
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_2P5 1
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_5 2
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_8 3
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_lsb 20
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_msb 24
+
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP2 FM10K_SW_PCIE_CFG_REG(0x25)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL2 FM10K_SW_PCIE_CFG_REG(0x26)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL2 FM10K_SW_PCIE_CFG_REG(0x28)
+#define FM10K_SW_PCIE_CFG_MSIX_CAP FM10K_SW_PCIE_CFG_REG(0x2C)
+#define FM10K_SW_PCIE_CFG_MSIX_TABLE_OFFSET FM10K_SW_PCIE_CFG_REG(0x2D)
+#define FM10K_SW_PCIE_CFG_MSIX_PBA FM10K_SW_PCIE_CFG_REG(0x2E)
+#define FM10K_SW_PCIE_CFG_VPD_CAP FM10K_SW_PCIE_CFG_REG(0x34)
+#define FM10K_SW_PCIE_CFG_VPD_DATA FM10K_SW_PCIE_CFG_REG(0x35)
+#define FM10K_SW_PCIE_CFG_AER_HDR FM10K_SW_PCIE_CFG_REG(0x40)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_STATUS FM10K_SW_PCIE_CFG_REG(0x41)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_MASK FM10K_SW_PCIE_CFG_REG(0x42)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_SEVERITY FM10K_SW_PCIE_CFG_REG(0x43)
+#define FM10K_SW_PCIE_CFG_AER_COERR_STATUS FM10K_SW_PCIE_CFG_REG(0x44)
+#define FM10K_SW_PCIE_CFG_AER_COERR_MASK FM10K_SW_PCIE_CFG_REG(0x45)
+#define FM10K_SW_PCIE_CFG_AER_CTRL FM10K_SW_PCIE_CFG_REG(0x46)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG0 FM10K_SW_PCIE_CFG_REG(0x47)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG1 FM10K_SW_PCIE_CFG_REG(0x48)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG2 FM10K_SW_PCIE_CFG_REG(0x49)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG3 FM10K_SW_PCIE_CFG_REG(0x4A)
+#define FM10K_SW_PCIE_CFG_SPD_HDR FM10K_SW_PCIE_CFG_REG(0x52)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_L FM10K_SW_PCIE_CFG_REG(0x53)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_H FM10K_SW_PCIE_CFG_REG(0x54)
+#define FM10K_SW_PCIE_CFG_ARI_HDR FM10K_SW_PCIE_CFG_REG(0x56)
+#define FM10K_SW_PCIE_CFG_ARI_CTRL FM10K_SW_PCIE_CFG_REG(0x57)
+#define FM10K_SW_PCIE_CFG_SPCIE_HDR FM10K_SW_PCIE_CFG_REG(0x5A)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_CTRL3 FM10K_SW_PCIE_CFG_REG(0x5B)
+#define FM10K_SW_PCIE_CFG_SPCIE_ERR_STS FM10K_SW_PCIE_CFG_REG(0x5C)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ01 FM10K_SW_PCIE_CFG_REG(0x5D)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ23 FM10K_SW_PCIE_CFG_REG(0x5E)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ45 FM10K_SW_PCIE_CFG_REG(0x5F)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ67 FM10K_SW_PCIE_CFG_REG(0x60)
+#define FM10K_SW_PCIE_CFG_SRIOV_HDR FM10K_SW_PCIE_CFG_REG(0x62)
+#define FM10K_SW_PCIE_CFG_SRIOV_CAP FM10K_SW_PCIE_CFG_REG(0x63)
+#define FM10K_SW_PCIE_CFG_SRIOV_CTRL FM10K_SW_PCIE_CFG_REG(0x64)
+#define FM10K_SW_PCIE_CFG_SRIOV_CFG FM10K_SW_PCIE_CFG_REG(0x65)
+#define FM10K_SW_PCIE_CFG_SRIOV_NUM FM10K_SW_PCIE_CFG_REG(0x66)
+#define FM10K_SW_PCIE_CFG_SRIOV_MAP FM10K_SW_PCIE_CFG_REG(0x67)
+#define FM10K_SW_PCIE_CFG_SRIOV_DEVID FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_SUP FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_CFG FM10K_SW_PCIE_CFG_REG(0x6A)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR0 FM10K_SW_PCIE_CFG_REG(0x6B)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR1 FM10K_SW_PCIE_CFG_REG(0x6C)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR2 FM10K_SW_PCIE_CFG_REG(0x6D)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR3 FM10K_SW_PCIE_CFG_REG(0x6E)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR4 FM10K_SW_PCIE_CFG_REG(0x6F)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR5 FM10K_SW_PCIE_CFG_REG(0x70)
+#define FM10K_SW_PCIE_CFG_SRIOV_MIG FM10K_SW_PCIE_CFG_REG(0x71)
+#define FM10K_SW_PCIE_CFG_TPH_HDR FM10K_SW_PCIE_CFG_REG(0x72)
+#define FM10K_SW_PCIE_CFG_TPH_CAP FM10K_SW_PCIE_CFG_REG(0x73)
+#define FM10K_SW_PCIE_CFG_TPH_CTRL FM10K_SW_PCIE_CFG_REG(0x74)
+#define FM10K_SW_PCIE_CFG_ACS_HDR FM10K_SW_PCIE_CFG_REG(0x76)
+#define FM10K_SW_PCIE_CFG_ACS_CAP FM10K_SW_PCIE_CFG_REG(0x77)
+#define FM10K_SW_PCIE_PORTLOGIC FM10K_SW_PCIE_CFG_REG(0x1C0)
+#define FM10K_SW_PCIE_PORTLOGIC_LINK_STATE FM10K_SW_PCIE_CFG_REG(0x1CA)
+
+#define FM10K_SW_FFU_BASE FM10K_SW_REG_OFF(0xC00000)
+#define FM10K_SW_FFU_REG(wo_) \
+ (FM10K_SW_FFU_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_FFU_NUM_SLICES 32
+#define FM10K_SW_FFU_NUM_SCENARIOS 32
+
+/* FFU enumerated types */
+#define FM10K_SW_FFU_MUX_SEL_MAP_DIP_MAP_SIP 0
+#define FM10K_SW_FFU_MUX_SEL_MAP_DMAC_MAP_SMAC 1
+#define FM10K_SW_FFU_MUX_SEL_MAP_PROT_MAP_LENGTH 2
+#define FM10K_SW_FFU_MUX_SEL_MAP_SRC_MAP_TYPE 3
+#define FM10K_SW_FFU_MUX_SEL_USER 4
+#define FM10K_SW_FFU_MUX_SEL_FTYPE_SWPRI 5
+#define FM10K_SW_FFU_MUX_SEL_IPMISC 6
+#define FM10K_SW_FFU_MUX_SEL_TOS 7
+#define FM10K_SW_FFU_MUX_SEL_PROT 8
+#define FM10K_SW_FFU_MUX_SEL_TTL 9
+#define FM10K_SW_FFU_MUX_SEL_SRC_PORT 10
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID_11_8 11
+#define FM10K_SW_FFU_MUX_SEL_VID_7_0 12
+#define FM10K_SW_FFU_MUX_SEL_RXTAG 13
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_15_0 14
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_31_16 15
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_47_32 16
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_15_0 17
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_31_16 18
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_47_32 19
+#define FM10K_SW_FFU_MUX_SEL_DGLORT 20
+#define FM10K_SW_FFU_MUX_SEL_SGLORT 21
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID 22
+#define FM10K_SW_FFU_MUX_SEL_VPRI2_VID2 23
+#define FM10K_SW_FFU_MUX_SEL_L2_TYPE 24
+#define FM10K_SW_FFU_MUX_SEL_L4_DST 25
+#define FM10K_SW_FFU_MUX_SEL_L4_SRC 26
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_DST 27
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_SRC 28
+#define FM10K_SW_FFU_MUX_SEL_L4A 29
+#define FM10K_SW_FFU_MUX_SEL_L4B 30
+#define FM10K_SW_FFU_MUX_SEL_L4C 31
+#define FM10K_SW_FFU_MUX_SEL_L4D 32
+#define FM10K_SW_FFU_MUX_SEL_MAP_VPRI1_VID1 33
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_31_0 34
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_63_32 35
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_95_64 36
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_127_96 37
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_31_0 38
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_63_32 39
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64 40
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96 41
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_msb 19
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_TYPE_EXP (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_FLOOD_SET (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_msb 7
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_lsb 8
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_msb 20
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_msb 11
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_msb 17
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_PRI (1 << 19)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_msb 5
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_DSCP (1 << 17)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_PRI (1 << 19)
+
+/* FFU Registers */
+#define FM10K_SW_FFU_SLICE_TCAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x2000 * (sl_) + 0x4 * (n_))
+#define FM10K_SW_FFU_SLICE_TCAM_ENTRIES 1024
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_lsb64 0
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_msb64 31
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_lsb64 32
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_msb64 39
+#define FM10K_SW_FFU_SLICE_SRAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x1000 + 0x2000 * (sl_) + 0x2 * (n_))
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_lsb64 21
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_msb64 22
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_ARP 0
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT 1
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_BIT_SET 2
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET 3
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_lsb64 23
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_msb64 34
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_lsb64 35
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_msb64 36
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_lsb64 37
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_msb64 39
+#define FM10K_SW_FFU_SLICE_VALID(sl_) \
+ FM10K_SW_FFU_REG(0x1800 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_VALID_SCENARIO(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_FFU_SLICE_CASCADE_ACTION(sl_) \
+ FM10K_SW_FFU_REG(0x1804 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG(sl_, scen_) \
+ FM10K_SW_FFU_REG(0x1840 + 0x2 * (scen_) + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_lsb64 0
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_msb64 5
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_lsb64 6
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_msb64 11
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_lsb64 12
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_msb64 17
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_lsb64 18
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_msb64 23
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_lsb64 24
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_msb64 29
+#define FM10K_SW_FFU_SLICE_CFG_START_COMPARE (1ULL << 30)
+#define FM10K_SW_FFU_SLICE_CFG_START_ACTION (1ULL << 31)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_LOW (1ULL << 32)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_HIGH (1ULL << 33)
+#define FM10K_SW_FFU_SLICE_CFG_CASE_lsb64 34
+#define FM10K_SW_FFU_SLICE_CFG_CASE_msb64 37
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_lsb64 38
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_msb64 39
+
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_NOT_MAPPED 0
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_LOW_NIBBLE 1
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_HIGH_NIBBLE 2
+
+#define FM10K_SW_FFU_MASTER_VALID \
+ FM10K_SW_FFU_REG(0x40000)
+#define FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_SLICES_VALID FM10K_SW_MASK64(31, 0)
+#define FM10K_SW_FFU_MASTER_VALID_CHUNK_VALID(c_) (1ULL << ((c_) + 32))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_CHUNKS_VALID FM10K_SW_MASK64(63, 32)
+
+#define FM10K_SW_L2LOOKUP_BASE \
+ FM10K_SW_REG_OFF(0xC80000)
+#define FM10K_SW_L2LOOKUP_REG(wo_) \
+ (FM10K_SW_L2LOOKUP_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_MA_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x10000 * (t_) + 0x4 * (n_))
+#define FM10K_SW_INGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x20000 + 0x4 * (v_))
+#define FM10K_SW_INGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative
+ * to the start of the upper 64-bit word
+ */
+#define FM10K_SW_INGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_msb64 (89 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_REFLECT (1ULL << (90 - 64))
+#define FM10K_SW_INGRESS_VID_TABLE_TRAP_IGMP (1ULL << (91 - 64))
+#define FM10K_SW_EGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x24000 + 0x4 * (v_))
+#define FM10K_SW_EGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative to
+ * the start of the upper 64-bit word
+ */
+#define FM10K_SW_EGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_msb64 (86 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_lsb64 (87 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_msb64 (92 - 64)
+
+#define FM10K_SW_MA_USED_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28000 + 0x200 * (t_) + (n_))
+#define FM10K_SW_INGRESS_MST_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28400 + 0x200 * (t_) + 0x2 * (n_))
+
+#define FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE 24
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) \
+ (((l_) % FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) * 2)
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_msb64(l_) \
+ (FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) + 1)
+
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_DISABLE 0
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LISTENING 1
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LEARNING 2
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD 3
+
+#define FM10K_SW_EGRESS_MST_TABLE(t_) \
+ FM10K_SW_L2LOOKUP_REG(0x28800 + 0x2 * (t_))
+#define FM10K_SW_EGRESS_MST_TABLE_FORWARDING(l_) (1ULL << (l_))
+
+#define FM10K_SW_MA_TABLE_CFG_1 \
+ FM10K_SW_L2LOOKUP_REG(0x28A00)
+#define FM10K_SW_MA_TABLE_CFG_2 \
+ FM10K_SW_L2LOOKUP_REG(0x28A02)
+#define FM10K_SW_MTU_TABLE(n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28A08 + (n_))
+#define FM10K_SW_IEEE_RESERVED_MAC_ACTION \
+ FM10K_SW_L2LOOKUP_REG(0x28A10)
+#define FM10K_SW_IEEE_RESERVED_MAC_TRAP_PRIORITY \
+ FM10K_SW_L2LOOKUP_REG(0x28A14)
+#define FM10K_SW_IEEE_RESERVED_MAC_CFG \
+ FM10K_SW_L2LOOKUP_REG(0x28A16)
+
+
+#define FM10K_SW_GLORT_BASE FM10K_SW_REG_OFF(0xCE0000)
+#define FM10K_SW_GLORT_REG(wo_) \
+ (FM10K_SW_GLORT_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_GLORT_CAM_ENTRIES 256
+#define FM10K_SW_GLORT_DEST_TABLE(n_) \
+ FM10K_SW_GLORT_REG(0x0 + 0x2 * (n_))
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_lsb64 0
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_msb64 47
+
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_lsb64 48
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_msb64 59
+
+#define FM10K_SW_GLORT_CAM(n_) \
+ FM10K_SW_GLORT_REG(0x2000 + (n_))
+#define FM10K_SW_GLORT_CAM_MATCH_ANY 0x00000000
+#define FM10K_SW_GLORT_CAM_MATCH_NONE FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_GLORT_CAM_KEY_lsb 0
+#define FM10K_SW_GLORT_CAM_KEY_msb 15
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_lsb 16
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_msb 31
+#define FM10K_SW_GLORT_RAM(n_) \
+ FM10K_SW_GLORT_REG(0x2200 + 0x2 * (n_))
+#define FM10K_SW_GLORT_RAM_STRICT_lsb64 0
+#define FM10K_SW_GLORT_RAM_STRICT_msb64 1
+#define FM10K_SW_GLORT_RAM_STRICT_FTYPE 0
+#define FM10K_SW_GLORT_RAM_STRICT_RSVD 1
+#define FM10K_SW_GLORT_RAM_STRICT_HASHED 2
+#define FM10K_SW_GLORT_RAM_STRICT_STRICT 3
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_lsb64 2
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_msb64 13
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_lsb64 14
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_msb64 21
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_lsb64 22
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_msb64 29
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_lsb64 30
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_msb64 33
+#define FM10K_SW_GLORT_RAM_HASH_ROTATION (1ULL << 34)
+
+
+#define FM10K_SW_PARSER_BASE FM10K_SW_REG_OFF(0xCF0000)
+#define FM10K_SW_PARSER_REG(wo_) \
+ (FM10K_SW_PARSER_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_PARSER_PORT_CFG_1(p_) \
+ FM10K_SW_PARSER_REG(0x0 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_1_FTAG (1ULL << 0)
+
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(v_) (1ULL << (1 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(v_) (1ULL << (5 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_FIRST (1ULL << 9)
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_lsb 10
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_msb 21
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_lsb 22
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_lsb 26
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_msb 37
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_lsb 38
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_msb 41
+#define FM10K_SW_PARSER_PORT_CFG_1_USE_DEFAULT_VLAN (1ULL << 42)
+#define FM10K_SW_PARSER_PORT_CFG_2(p_) \
+ FM10K_SW_PARSER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_lsb 0
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_msb 3
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_lsb 4
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_msb 7
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_MPLS (1ULL << 8)
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_lsb 9
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_msb 11
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3 (1ULL << 12)
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4 (1ULL << 13)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV4_OPTIONS (1ULL << 14)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_HOP_BY_HOP (1ULL << 15)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_ROUTING (1ULL << 16)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_FRAG (1ULL << 17)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_DEST (1ULL << 18)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_AUTH (1ULL << 19)
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_lsb 20
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_TAGGED (1ULL << 26)
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_UNTAGGED (1ULL << 27)
+#define FM10K_SW_PARSER_PORT_CFG_2_USE_DEFAULT_DSCP (1ULL << 28)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_VLAN (1ULL << 29)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_DSCP (1ULL << 30)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_ISL (1ULL << 31)
+
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_PREFER_DSCP (1ULL << 32)
+
+#define FM10K_SW_PARSER_PORT_CFG_3(p_) \
+ FM10K_SW_PARSER_REG(0x100 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_ISL(p_) \
+ FM10K_SW_PARSER_REG(0x180 + (p_))
+
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_lsb 0
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_msb 15
+#define FM10K_SW_PORT_CFG_ISL_USR_lsb 16
+#define FM10K_SW_PORT_CFG_ISL_USR_msb 23
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_lsb 24
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_msb 27
+
+#define FM10K_SW_PARSER_VLAN_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C0 + (n_))
+#define FM10K_SW_PARSER_CUSTOM_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C4 + (n_))
+#define FM10K_SW_PARSER_MPLS_TAG \
+ FM10K_SW_PARSER_REG(0x1C8)
+#define FM10K_SW_PARSER_DI_CFG(n_) \
+ FM10K_SW_PARSER_REG(0x1D0 + 0x2 * (n_))
+#define FM10K_SW_RX_VPRI_MAP(p_) \
+ FM10K_SW_PARSER_REG(0x200 + 0x2 * (p_))
+#define FM10K_SW_DSCP_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x280 + (pri_))
+#define FM10K_SW_VPRI_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x2C0 + (pri_))
+
+#define FM10K_SW_HANDLER_BASE FM10K_SW_REG_OFF(0xD50000)
+#define FM10K_SW_HANDLER_REG(wo_) \
+ (FM10K_SW_HANDLER_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_SYS_CFG_1 \
+ FM10K_SW_HANDLER_REG(0x0)
+#define FM10K_SW_SYS_CFG_1_DROP_PAUSE (1 << 0)
+#define FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS (1 << 1)
+#define FM10K_SW_SYS_CFG_1_ENABLE_TRAP_PLUS_LOG (1 << 2)
+#define FM10K_SW_SYS_CFG_1_DROP_INVALID_SMAC (1 << 3)
+#define FM10K_SW_SYS_CFG_1_DROP_MAC_CTRL_ETHERTYPE (1 << 4)
+
+#define FM10K_SW_CPU_MAC FM10K_SW_HANDLER_REG(0x2)
+#define FM10K_SW_SYS_CFG_ROUTER FM10K_SW_HANDLER_REG(0x4)
+#define FM10K_SW_L34_HASH_CFG FM10K_SW_HANDLER_REG(0x5)
+#define FM10K_SW_L34_HASH_CFG_SYMMETRIC (1 << 0)
+#define FM10K_SW_L34_HASH_CFG_USE_SIP (1 << 1)
+#define FM10K_SW_L34_HASH_CFG_USE_DIP (1 << 2)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT (1 << 3)
+#define FM10K_SW_L34_HASH_CFG_USE_TCP (1 << 4)
+#define FM10K_SW_L34_HASH_CFG_USE_UDP (1 << 5)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT1 (1 << 6)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT2 (1 << 7)
+#define FM10K_SW_L34_HASH_CFG_USE_L4SRC (1 << 8)
+#define FM10K_SW_L34_HASH_CFG_USE_L4DST (1 << 9)
+
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_lsb 10
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_msb 11
+#define FM10K_SW_L34_HASH_CFG_PROT1_lsb 16
+#define FM10K_SW_L34_HASH_CFG_PROT1_msb 23
+#define FM10K_SW_L34_HASH_CFG_PROT2_lsb 24
+#define FM10K_SW_L34_HASH_CFG_PROT2_msb 31
+
+#define FM10K_SW_L34_FLOW_HASH_CFG_1 FM10K_SW_HANDLER_REG(0x6)
+#define FM10K_SW_L34_FLOW_HASH_CFG_2 FM10K_SW_HANDLER_REG(0x7)
+#define FM10K_SW_L234_HASH_CFG FM10K_SW_HANDLER_REG(0x8)
+#define FM10K_SW_L234_HASH_CFG_USE_L2_IF_IP (1 << 0)
+#define FM10K_SW_L234_HASH_CFG_USE_L34 (1 << 1)
+#define FM10K_SW_L234_HASH_CFG_SYMMETRIC (1 << 2)
+#define FM10K_SW_L234_HASH_CFG_USE_DMAC (1 << 3)
+#define FM10K_SW_L234_HASH_CFG_USE_SMAC (1 << 4)
+#define FM10K_SW_L234_HASH_CFG_USE_TYPE (1 << 5)
+#define FM10K_SW_L234_HASH_CFG_USE_VPRI (1 << 6)
+#define FM10K_SW_L234_HASH_CFG_USE_VID (1 << 8)
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_lsb 8
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_msb 9
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_lsb 10
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_msb 11
+#define FM10K_SW_CPU_TRAP_MASK_FH FM10K_SW_HANDLER_REG(0xA)
+#define FM10K_SW_TRAP_GLORT FM10K_SW_HANDLER_REG(0xC)
+#define FM10K_SW_RX_MIRROR_CFG FM10K_SW_HANDLER_REG(0xD)
+#define FM10K_SW_LOG_MIRROR_PROFILE FM10K_SW_HANDLER_REG(0xE)
+
+#define FM10K_SW_FH_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_HANDLER_REG(0x40 + (n_))
+#define FM10K_SW_PORT_CFG_2(p_) \
+ FM10K_SW_HANDLER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_3(p_) \
+ FM10K_SW_HANDLER_REG(0x100 + (p_))
+#define FM10K_SW_FH_LOOPBACK_SUPPRESS(p_) \
+ FM10K_SW_HANDLER_REG(0x140 + (p_))
+#define FM10K_SW_FH_HEAD_IP FM10K_SW_HANDLER_REG(0x180)
+#define FM10K_SW_FH_HEAD_IM FM10K_SW_HANDLER_REG(0x182)
+#define FM10K_SW_PARSER_EARLY_SRAM_CTRL FM10K_SW_HANDLER_REG(0x184)
+#define FM10K_SW_PARSER_LATE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x185)
+#define FM10K_SW_MAPPER_SRAM_CTRL FM10K_SW_HANDLER_REG(0x186)
+#define FM10K_SW_FFU_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1A0 + 0x4 * (n_))
+#define FM10K_SW_ARP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C0)
+#define FM10K_SW_VLAN_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C1)
+#define FM10K_SW_MA_TABLE_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1C4 + 0x2 * (n_))
+#define FM10K_SW_FID_GLORT_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C8)
+#define FM10K_SW_GLORT_RAM_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CA)
+#define FM10K_SW_GLORT_TABLE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CB)
+#define FM10K_SW_FH_HEAD_OUTPUT_FIFO_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CC)
+
+#define FM10K_SW_LAG_BASE FM10K_SW_REG_OFF(0xD90000)
+#define FM10K_SW_LAG_REG(wo_) \
+ (FM10K_SW_LAG_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_LAG_CFG(l_) FM10K_SW_LAG_REG(l_)
+#define FM10K_SW_LAG_CFG_LAG_SIZE_lsb 0
+#define FM10K_SW_LAG_CFG_LAG_SIZE_msb 3
+#define FM10K_SW_LAG_CFG_INDEX_lsb 4
+#define FM10K_SW_LAG_CFG_INDEX_msb 7
+#define FM10K_SW_LAG_CFG_HASH_ROTATION (1 << 8)
+#define FM10K_SW_LAG_CFG_IN_LAG (1 << 9)
+
+#define FM10K_SW_CANONICAL_GLORT_CAM(n_) \
+ FM10K_SW_LAG_REG((n_) + 0x40)
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_lsb 0
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_msb 15
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_lsb 16
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_msb 19
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_lsb 20
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_msb 22
+
+#define FM10K_SW_RX_STATS_BASE \
+ FM10K_SW_REG_OFF(0xE00000)
+#define FM10K_SW_RX_STATS_REG(wo_) \
+ (FM10K_SW_RX_STATS_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_RX_STATS_BANK(b_, i_) \
+ FM10K_SW_RX_STATS_REG(0x1000 * (b_) + 0x4 * (i_))
+#define FM10K_SW_RX_STATS_BANK_1_INDEX 0
+#define FM10K_SW_RX_STATS_BANK_2_INDEX 1
+#define FM10K_SW_RX_STATS_BANK_3_INDEX 2
+#define FM10K_SW_RX_STATS_BANK_4_INDEX 3
+#define FM10K_SW_RX_STATS_BANK_5_INDEX 4
+#define FM10K_SW_RX_STATS_BANK_6_INDEX 5
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_UCAST 0
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_MCAST 1
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_BCAST 2
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_UCAST 3
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_MCAST 4
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_BCAST 5
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_UCAST 6
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_MCAST 7
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_BCAST 8
+#define FM10K_SW_RX_STATS_BANK_1_IEEE802_3_PAUSE 9
+#define FM10K_SW_RX_STATS_BANK_1_CLASS_BASED_PAUSE 10
+#define FM10K_SW_RX_STATS_BANK_1_FRAMING_ERR 11
+#define FM10K_SW_RX_STATS_BANK_1_FCS_ERR 12
+#define FM10K_SW_RX_STATS_BANK_2_LEN_LT_64 0
+#define FM10K_SW_RX_STATS_BANK_2_LEN_EQ_64 1
+#define FM10K_SW_RX_STATS_BANK_2_LEN_65_127 2
+#define FM10K_SW_RX_STATS_BANK_2_LEN_128_255 3
+#define FM10K_SW_RX_STATS_BANK_2_LEN_256_511 4
+#define FM10K_SW_RX_STATS_BANK_2_LEN_512_1023 5
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1024_1522 6
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1523_2047 7
+#define FM10K_SW_RX_STATS_BANK_2_LEN_2048_4095 8
+#define FM10K_SW_RX_STATS_BANK_2_LEN_4096_8191 9
+#define FM10K_SW_RX_STATS_BANK_2_LEN_8192_10239 10
+#define FM10K_SW_RX_STATS_BANK_2_LEN_GE_10240 11
+#define FM10K_SW_RX_STATS_BANK_3_PRI(p_) (p_)
+#define FM10K_SW_RX_STATS_BANK_4_FID_FORWARDED 0
+#define FM10K_SW_RX_STATS_BANK_4_FLOOD_FORWARDED 1
+#define FM10K_SW_RX_STATS_BANK_4_SPECIALLY_HANDLED 2
+#define FM10K_SW_RX_STATS_BANK_4_PARSER_ERROR_DROP 3
+#define FM10K_SW_RX_STATS_BANK_4_ECC_ERROR_DROP 4
+#define FM10K_SW_RX_STATS_BANK_4_TRAPPED 5
+#define FM10K_SW_RX_STATS_BANK_4_PAUSE_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_4_STP_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_4_SECURITY_VIOLATIONS 8
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_TAG_DROPS 9
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_INGRESS_DROPS 10
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_EGRESS_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_4_GLORT_MISS_DROPS 12
+#define FM10K_SW_RX_STATS_BANK_4_FFU_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_4_TRIGGER_DROPS 14
+#define FM10K_SW_RX_STATS_BANK_5_POLICER_DROPS 0
+#define FM10K_SW_RX_STATS_BANK_5_TTL_DROPS 1
+#define FM10K_SW_RX_STATS_BANK_5_CM_PRIV_DROPS 2
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP0_DROPS 3
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP1_DROPS 4
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG0_DROPS 5
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG1_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG0_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG1_DROPS 8
+#define FM10K_SW_RX_STATS_BANK_5_TRIGGER_REDIRECTS 10
+#define FM10K_SW_RX_STATS_BANK_5_FLOOD_CONTROL_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_5_GLORT_FORWARDED 12
+#define FM10K_SW_RX_STATS_BANK_5_LOOPBACK_SUPP_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_5_OTHER_DROPS 14
+#define FM10K_SW_RX_PORT_STAT(b_, s_, p_, f_) \
+ (FM10K_SW_RX_STATS_BANK(FM10K_SW_RX_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_RX_STATS_##b_##_##s_) + \
+ ((f_) ? 0 : FM10K_SW_REG_OFF(2)))
+
+#define FM10K_SW_RX_STATS_CFG(p_) \
+ FM10K_SW_RX_STATS_REG(0x10000 + (p_))
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_lsb 0
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_msb 7
+#define FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS 0x00003f00
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_1 (1 << 8)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_2 (1 << 9)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_3 (1 << 10)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_4 (1 << 11)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_5 (1 << 12)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_6 (1 << 13)
+#define FM10K_SW_RX_STATS_CFG_SWITCH_PRI (1 << 14)
+
+#define FM10K_SW_HANDLER_TAIL_BASE \
+ FM10K_SW_REG_OFF(0xE30000)
+#define FM10K_SW_HANDLER_TAIL_REG(wo_) \
+ (FM10K_SW_HANDLER_TAIL_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_SAF_MATRIX(l_) \
+ FM10K_SW_HANDLER_TAIL_REG(0x2 * (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF(l_) (1ULL << (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS FM10K_SW_MASK64(47, 0)
+#define FM10K_SW_SAF_MATRIX_IGNORE_ERROR (1ULL << 50)
+
+#define FM10K_SW_FRAME_TIME_OUT FM10K_SW_HANDLER_TAIL_REG(0x80)
+#define FM10K_SW_SAF_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x81)
+#define FM10K_SW_EGRESS_PAUSE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x82)
+#define FM10K_SW_RX_STATS_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x84)
+#define FM10K_SW_POLICER_USAGE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x88)
+#define FM10K_SW_TCN_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x8C)
+#define FM10K_SW_FH_TAIL_IP FM10K_SW_HANDLER_TAIL_REG(0x8D)
+#define FM10K_SW_FH_TAIL_IM FM10K_SW_HANDLER_TAIL_REG(0x8E)
+#define FM10K_SW_TAIL_PERMIT_MGMT FM10K_SW_HANDLER_TAIL_REG(0x8F)
+#define FM10K_SW_TAIL_FORCE_IDLE FM10K_SW_HANDLER_TAIL_REG(0x90)
+
+
+#define FM10K_SW_CM_USAGE_BASE FM10K_SW_REG_OFF(0xE60000)
+#define FM10K_SW_CM_USAGE_REG(wo_) \
+ (FM10K_SW_CM_USAGE_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_CM_SWEEPER_SWITCH_PRI_TO_TC FM10K_SW_CM_USAGE_REG(0x0)
+#define FM10K_SW_CM_SWEEPER_TC_TO_SMP FM10K_SW_CM_USAGE_REG(0x2)
+#define FM10K_SW_CM_TX_TC_PRIVATE_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x200 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_TX_TC_HOG_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x400 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_PAUSE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x600 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_PRIVATE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x680 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_HOG_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x700 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_PAUSE_RESEND_INTERVAL(l_) \
+ FM10K_SW_CM_USAGE_REG(0x780 + (l_))
+#define FM10K_SW_CM_PAUSE_BASE_FREQ \
+ FM10K_SW_CM_USAGE_REG(0x7C0)
+#define FM10K_SW_CM_PAUSE_CFG(l_) \
+ FM10K_SW_CM_USAGE_REG(0x800 + (l_))
+#define FM10K_SW_CM_SHARED_WM(p_) \
+ FM10K_SW_CM_USAGE_REG(0x840 + (p_))
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_WM(s_) \
+ FM10K_SW_CM_USAGE_REG(0x850 + (s_))
+#define FM10K_SW_CM_GLOBAL_WM FM10K_SW_CM_USAGE_REG(0x852)
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_lsb 0
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_msb 14
+#define FM10K_SW_CM_GLOBAL_CFG FM10K_SW_CM_USAGE_REG(0x853)
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_lsb 0
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_msb 7
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_ON (1 << 8)
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_OFF (1 << 9)
+#define FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN (1 << 10)
+
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN (1 << 11)
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN (1 << 12)
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_lsb 13
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_msb 18
+
+#define FM10K_SW_CM_TC_PC_MAP(l_) FM10K_SW_CM_USAGE_REG(0x880 + (l_))
+#define FM10K_SW_CM_PC_SMP_MAP(l_) FM10K_SW_CM_USAGE_REG(0x8C0 + (l_))
+#define FM10K_SW_CM_SOFTDROP_WM(p_) FM10K_SW_CM_USAGE_REG(0x900 + (p_))
+
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_CFG(s_) \
+ FM10K_SW_CM_USAGE_REG(0x910 + 0x2 * (s_))
+#define FM10K_SW_TX_RATE_LIM_CFG(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xA00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_TX_RATE_LIM_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xC00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_BSG_MAP(l_) \
+ FM10K_SW_CM_USAGE_REG(0xE00 + (l_))
+#define FM10K_SW_CM_TX_TC_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x1000 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_USAGE(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x1200 + 0x2 * (l_) + (s_))
+#define FM10K_SW_MCAST_EPOCH_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1280 + (s_))
+#define FM10K_SW_CM_SHARED_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1282 + (s_))
+#define FM10K_SW_CM_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1284 + (s_))
+#define FM10K_SW_CM_GLOBAL_USAGE FM10K_SW_CM_USAGE_REG(0x1286)
+
+#define FM10K_SW_CM_PAUSE_GEN_STATE(l_) \
+ FM10K_SW_CM_USAGE_REG(0x12C0 + (l_))
+#define FM10K_SW_CM_PAUSE_RCV_TIMER FM10K_SW_CM_USAGE_REG(0x1300)
+#define FM10K_SW_CM_PAUSE_RCV_PORT_TIMER(l_) \
+ FM10K_SW_CM_USAGE_REG(0x1340 + (l_))
+
+/* FM10K_SW_CM_EGRESS_PAUSE_COUNT is also
+ * known as FM10K_SW_CM_PAUSE_RCV_STATE
+ */
+#define FM10K_SW_CM_EGRESS_PAUSE_COUNT(l_, dw_) \
+ FM10K_SW_CM_USAGE_REG(0x1400 + 0x4 * (l_) + 0x2 * (dw_))
+
+#define FM10K_SW_MOD_BASE FM10K_SW_REG_OFF(0xE80000)
+#define FM10K_SW_MOD_REG(wo_) \
+ (FM10K_SW_MOD_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_MOD_MAX_MGMT_WAIT_CYCLE FM10K_SW_MOD_REG(0xF8)
+#define FM10K_SW_MOD_IP FM10K_SW_MOD_REG(0x106)
+#define FM10K_SW_MOD_IM FM10K_SW_MOD_REG(0x108)
+#define FM10K_SW_MOD_SRAM_BIST_OUT FM10K_SW_MOD_REG(0x10A)
+#define FM10K_SW_MOD_SRAM_ERROR_WRITE FM10K_SW_MOD_REG(0x10C)
+#define FM10K_SW_MOD_PAUSE_SMAC FM10K_SW_MOD_REG(0x10E)
+#define FM10K_SW_MOD_ROUTER_SMAC(n_) \
+ FM10K_SW_MOD_REG(0x120 + 0x2 * (n_))
+#define FM10K_SW_MOD_MCAST_VLAN_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x10000 + 0x2 * (n_))
+#define FM10K_SW_MOD_VLAN_TAG_VID1_MAP(v_) \
+ FM10K_SW_MOD_REG(0x20000 + 0x2 * (v_))
+#define FM10K_SW_MOD_VID2_MAP(v_) \
+ FM10K_SW_MOD_REG(0x22000 + 0x2 * (v_))
+#define FM10K_SW_MOD_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x24000 + 0x2 * (n_))
+#define FM10K_SW_MOD_PER_PORT_CFG_1(p_) \
+ FM10K_SW_MOD_REG(0x24080 + 0x2 * (p_))
+#define FM10K_SW_MOD_PER_PORT_CFG_2(p_) \
+ FM10K_SW_MOD_REG(0x24100 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_lsb64 0
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_msb64 5
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP1_UPDATE (1ULL << 6)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP2_UPDATE (1ULL << 7)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI1_UPDATE (1ULL << 8)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI2_UPDATE (1ULL << 9)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_lsb64 10
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_msb64 11
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_lsb64 12
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_msb64 13
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DMAC_ROUTING (1ULL << 14)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_SMAC_ROUTING (1ULL << 15)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_TTL_DECREMENT (1ULL << 16)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DSCP_MODIFICATION (1ULL << 17)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_FTAG (1ULL << 18)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VID2_FIRST (1ULL << 19)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_lsb64 20
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_msb64 22
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_lsb64 23
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_msb64 30
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_TYPE (1ULL << 31)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_lsb64 32
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_msb64 47
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIN_FRAME_SIZE (1ULL << 48)
+
+#define FM10K_SW_MOD_VPRI1_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24180 + 0x2 * (p_))
+#define FM10K_SW_MOD_VPRI2_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24200 + 0x2 * (p_))
+#define FM10K_SW_MOD_VLAN_ETYPE(n_) \
+ FM10K_SW_MOD_REG(0x24280 + 0x2 * (n_))
+#define FM10K_SW_MOD_STATS_CFG(p_) \
+ FM10K_SW_MOD_REG(0x24300 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 (1 << 0)
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8 (1 << 1)
+#define FM10K_SW_MOD_STATS_BANK_FRAME(b_, i_) \
+ FM10K_SW_MOD_REG(0x25000 + 0x800 * (b_) + 0x2 * (i_))
+#define FM10K_SW_MOD_STATS_BANK_BYTE(b_, i_) \
+ FM10K_SW_MOD_REG(0x26000 + 0x800 * (b_) + 0x2 * (i_))
+
+#define FM10K_SW_MOD_STATS_BANK_7_INDEX 0
+#define FM10K_SW_MOD_STATS_BANK_8_INDEX 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_UCAST 0
+#define FM10K_SW_MOD_STATS_BANK_7_L2_MCAST 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_BCAST 2
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR 3
+#define FM10K_SW_MOD_STATS_BANK_7_TIMEOUT_DROP 4
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR_DROP 5
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ECC_DROP 6
+#define FM10K_SW_MOD_STATS_BANK_7_LOOPBACK_DROP 7
+#define FM10K_SW_MOD_STATS_BANK_7_TTL1_DROP 8
+#define FM10K_SW_MOD_STATS_BANK_7_PAUSE 9
+#define FM10K_SW_MOD_STATS_BANK_7_CB_PAUSE 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_LT_64 0
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_EQ_64 1
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_65_127 2
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_128_255 3
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_256_511 4
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_512_1023 5
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1024_1522 6
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1523_2047 7
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_2048_4095 8
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_4096_8191 9
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_8192_10239 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_GE_10240 11
+#define FM10K_SW_TX_PORT_STAT(b_, s_, p_, f_, p1_) \
+ ((f_) ? \
+ FM10K_SW_MOD_STATS_BANK_FRAME(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_MOD_STATS_##b_##_##s_) : \
+ FM10K_SW_MOD_STATS_BANK_BYTE(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p1_) + FM10K_SW_MOD_STATS_##b_##_##s_))
+
+#define FM10K_SW_SCHED_BASE FM10K_SW_REG_OFF(0xF00000)
+#define FM10K_SW_SCHED_REG(wo_) \
+ (FM10K_SW_SCHED_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * Schedule fields for both FM10K_SW_SCHED_RX_SCHEDULE and
+ * FM10K_SW_SCHED_TX_SCHEDULE
+ */
+#define FM10K_SW_SCHED_SCHEDULE_ENTRY(p_, l_, q_) \
+ (FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_PHYS_PORT, (p_)) | \
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_LOG_PORT, (l_)) | \
+ ((q_) ? FM10K_SW_SCHED_SCHEDULE_QUAD : 0))
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_lsb 0
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_msb 7
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_lsb 8
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_msb 13
+#define FM10K_SW_SCHED_SCHEDULE_QUAD (1 << 14)
+#define FM10K_SW_SCHED_SCHEDULE_COLOR (1 << 15)
+#define FM10K_SW_SCHED_SCHEDULE_IDLE (1 << 16)
+
+#define FM10K_SW_SCHED_RX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20000 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_TX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20400 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_SCHEDULE_CTRL \
+ FM10K_SW_SCHED_REG(0x20800)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE (1 << 0)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_PAGE (1 << 1)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_lsb 2
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_msb 10
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE (1 << 11)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_PAGE (1 << 12)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_lsb 13
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_msb 21
+#define FM10K_SW_SCHED_CONFIG_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x20801)
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x20840 + (l_))
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_lsb 0
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_msb 15
+#define FM10K_SW_SCHED_SSCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30050)
+#define FM10K_SW_SCHED_ESCHED_CFG_1(l_) \
+ FM10K_SW_SCHED_REG(0x30080 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_2(l_) \
+ FM10K_SW_SCHED_REG(0x300C0 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_3(l_) \
+ FM10K_SW_SCHED_REG(0x30100 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_ESCHED \
+ FM10K_SW_SCHED_REG(0x30240)
+#define FM10K_SW_SCHED_ESCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30242)
+#define FM10K_SW_SCHED_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x30244)
+#define FM10K_SW_SCHED_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_FREELIST_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30246)
+#define FM10K_SW_SCHED_MONITOR_DRR_Q_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x30400 + (q_))
+#define FM10K_SW_SCHED_MONITOR_DRR_CFG_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x30600 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_MONITOR \
+ FM10K_SW_SCHED_REG(0x30A40)
+#define FM10K_SW_SCHED_MONITOR_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30A41)
+#define FM10K_SW_SCHED_MCAST_LEN_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x34000 + (n_))
+#define FM10K_SW_SCHED_MCAST_DEST_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x38000 + 0x2 * (n_))
+#define FM10K_SW_SCHED_MCAST_LIMITED_SKEW_MULTICAST \
+ FM10K_SW_SCHED_REG(0x3A000)
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(n_) \
+ FM10K_SW_SCHED_REG(0x60400 + 0x2 * (n_))
+
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES 8
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_lsb 0
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_msb 9
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_lsb 10
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_msb 19
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_lsb 20
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_msb 24
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_lsb 25
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_msb 29
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_lsb 30
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_msb 39
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x60410)
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES 1024
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_msb 9
+
+#define FM10K_SW_SCHED_MGMT_TIMER_RXQ \
+ FM10K_SW_SCHED_REG(0x60412)
+#define FM10K_SW_SCHED_MGMT_TIMER_LG \
+ FM10K_SW_SCHED_REG(0x60413)
+#define FM10K_SW_SCHED_RXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x60414)
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60600 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60800 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60A00 + (q_))
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES 384
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_lsb 0
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_msb 15
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x62000)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_TXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62002)
+#define FM10K_SW_SCHED_FIFO_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62004)
+#define FM10K_SW_SCHED_IP \
+ FM10K_SW_SCHED_REG(0x62006)
+#define FM10K_SW_SCHED_IM \
+ FM10K_SW_SCHED_REG(0x62007)
+
+#endif /* _FM10K_REGS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.c b/drivers/net/fm10k/switch/fm10k_sbus.c
new file mode 100644
index 0000000..d7d656e
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.c
@@ -0,0 +1,292 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+#define FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX 4
+#define FM10K_SW_SBUS_COMMAND_TIMEOUT_US 500000
+#define FM10K_SW_SBUS_RING_DIVIDER 0x01
+
+static int fm10k_sbus_init(struct fm10k_sbus *);
+static int fm10k_sbus_exec(struct fm10k_sbus *, struct fm10k_sbus_req *);
+static int fm10k_sbus_reset_all(struct fm10k_sbus *);
+static int fm10k_sbus_sbm_reset(struct fm10k_sbus *);
+
+struct fm10k_sbus *
+fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name,
+ unsigned int cfg_reg)
+{
+ struct fm10k_sbus *sb;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: attaching", name);
+
+ sb = (struct fm10k_sbus *)rte_zmalloc
+ ("fm10k_sbus", sizeof(struct fm10k_sbus), 0);
+ if (sb == NULL) {
+ FM10K_SW_INFO("sbus %s: failed to allocate context", name);
+ goto fail;
+ }
+
+ sb->sw = sw;
+ sb->name = name;
+ pthread_mutex_init(&sb->lock, NULL);
+
+ sb->cfg_reg = cfg_reg;
+ sb->cmd_reg = cfg_reg + FM10K_SW_REG_OFF(1);
+ sb->req_reg = cfg_reg + FM10K_SW_REG_OFF(2);
+ sb->resp_reg = cfg_reg + FM10K_SW_REG_OFF(3);
+
+ error = fm10k_sbus_init(sb);
+ if (error)
+ goto fail;
+
+ FM10K_SW_TRACE("sbus %s: attach successful", name);
+ return sb;
+fail:
+ if (sb)
+ fm10k_sbus_detach(sb);
+ return NULL;
+}
+
+void
+fm10k_sbus_detach(struct fm10k_sbus *sb)
+{
+ FM10K_SW_TRACE("sbus %s: detaching", sb->name);
+
+ rte_free(sb);
+}
+
+static int
+fm10k_sbus_init(struct fm10k_sbus *sb)
+{
+ uint32_t data;
+ int error;
+
+ error = fm10k_sbus_sbm_reset(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_reset_all(sb);
+ if (error)
+ goto done;
+
+ /* set clock to REFCLK/2 */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ FM10K_SW_SBUS_RING_DIVIDER);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to set ring divider", sb->name);
+ goto done;
+ }
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ &data);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to read back ring divider",
+ sb->name);
+ goto done;
+ }
+ if (data != FM10K_SW_SBUS_RING_DIVIDER) {
+ FM10K_SW_ERR("sbus %s: ring divider "
+ "verify failed (expected %u, got %u)",
+ sb->name, FM10K_SW_SBUS_RING_DIVIDER, data);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return error;
+}
+
+static int
+fm10k_sbus_exec(struct fm10k_sbus *sb, struct fm10k_sbus_req *req)
+{
+ struct fm10k_switch *sw = sb->sw;
+ unsigned int total_usecs;
+ unsigned int drop_lock;
+ unsigned int delay_time;
+ int error = 0;
+ uint32_t data;
+ uint8_t expected_result;
+
+ FM10K_SW_SBUS_LOCK(sb);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (data & FM10K_SW_SBUS_COMMAND_BUSY) {
+ FM10K_SW_INFO("sbus %s: bus busy (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+
+ switch (req->op) {
+ case FM10K_SW_SBUS_OP_RESET:
+ FM10K_SW_TRACE("sbus %s: RESET dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_RESET;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_WRITE:
+ FM10K_SW_TRACE("sbus %s: WRITE dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_WRITE;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_READ:
+ FM10K_SW_TRACE("sbus %s: READ dev=0x%02x reg=0x%02x",
+ sb->name, req->dev, req->reg);
+ expected_result = FM10K_SW_SBUS_RESULT_READ;
+ req->data = 0;
+ break;
+ default:
+ FM10K_SW_INFO("sbus %s: invalid opcode 0x%02x",
+ sb->name, req->op);
+ error = -1;
+ goto done;
+ }
+
+ /* Clear the execute bit */
+ fm10k_write_switch_reg(sw, sb->cmd_reg, 0);
+
+ data =
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_OP, req->op) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_ADDRESS, req->dev) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_REGISTER, req->reg) |
+ FM10K_SW_SBUS_COMMAND_EXECUTE;
+ fm10k_write_switch_reg(sw, sb->cmd_reg, data);
+ fm10k_write_flush(sw);
+
+ total_usecs = 0;
+ delay_time = 1;
+ drop_lock = 0;
+ do {
+ if (drop_lock)
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_udelay(delay_time);
+ if (drop_lock)
+ FM10K_SW_SWITCH_LOCK(sw);
+ total_usecs += delay_time;
+ if (total_usecs >= FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX) {
+ drop_lock = 1;
+ delay_time <<= 1;
+ }
+
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (!(data & FM10K_SW_SBUS_COMMAND_BUSY)) {
+ if (FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE)
+ != expected_result) {
+ FM10K_SW_INFO("sbus %s: expected "
+ "result code %u, got %u", sb->name,
+ expected_result,
+ FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE));
+ error = -1;
+ goto done;
+ }
+ if (req->op == FM10K_SW_SBUS_OP_READ) {
+ req->data =
+ fm10k_read_switch_reg(sw, sb->resp_reg);
+ FM10K_SW_TRACE("sbus %s: READ data=0x%02x",
+ sb->name, req->data);
+ }
+ goto done;
+ }
+
+ } while (total_usecs < FM10K_SW_SBUS_COMMAND_TIMEOUT_US);
+
+ error = -1;
+ FM10K_SW_INFO("sbus %s: command timed out after %u us "
+ "(op=0x%02x dev=0x%02x reg=0x%02x data=0x%08x)", sb->name,
+ total_usecs, req->op, req->dev, req->reg, req->data);
+
+done:
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ FM10K_SW_SBUS_UNLOCK(sb);
+ return error;
+}
+
+static int
+fm10k_sbus_reset_all(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_BROADCAST;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: broadcast reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+static int
+fm10k_sbus_sbm_reset(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_SPICO;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: SBM reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+int
+fm10k_sbus_read(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t *data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_READ;
+ req.dev = dev;
+ req.reg = reg;
+
+ error = fm10k_sbus_exec(sb, &req);
+ *data = error ? 0 : req.data;
+
+ return error;
+}
+
+int
+fm10k_sbus_write(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_WRITE;
+ req.dev = dev;
+ req.reg = reg;
+ req.data = data;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ return error;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.h b/drivers/net/fm10k/switch/fm10k_sbus.h
new file mode 100644
index 0000000..e67967f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SBUS_H_
+#define _FM10K_SW_SBUS_H_
+
+#include <pthread.h>
+
+
+struct fm10k_sbus {
+ struct fm10k_switch *sw;
+ const char *name;
+ pthread_mutex_t lock;
+ unsigned int cfg_reg;
+ unsigned int cmd_reg;
+ unsigned int req_reg;
+ unsigned int resp_reg;
+};
+
+#define FM10K_SW_SBUS_LOCK(sb_) pthread_mutex_lock(&((sb_)->lock))
+#define FM10K_SW_SBUS_UNLOCK(sb_) pthread_mutex_unlock(&((sb_)->lock))
+
+struct fm10k_sbus_req {
+ uint8_t op;
+ uint8_t dev;
+ uint8_t reg;
+ uint32_t data;
+};
+
+
+struct fm10k_sbus *fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name, unsigned int cfg_reg);
+void fm10k_sbus_detach(struct fm10k_sbus *sb);
+int fm10k_sbus_read(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t *data);
+int fm10k_sbus_write(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t data);
+
+#endif /* _FM10K_SW_SBUS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
new file mode 100644
index 0000000..2423dbf
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SWITCH_H_
+#define _FM10K_SW_SWITCH_H_
+
+#include <rte_spinlock.h>
+#include <pthread.h>
+#include <sys/time.h>
+#include <semaphore.h>
+
+#include "../fm10k.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+
+
+/*
+ * EPL
+ */
+#define FM10K_SW_EXT_PORTS_MAX 4
+
+#define FM10K_SW_EPLS_MAX 9
+#define FM10K_SW_EPLS_SUPPORTED 2
+#define FM10K_SW_EPL_LANES 4
+#define FM10K_SW_EPLS_PEP_MAX 2
+
+/*
+ * PEP
+ */
+#define FM10K_SW_PEP_GROUP_MAX 10
+#define FM10K_SW_LOGICAL_PORTS_MAX 48
+
+#define FM10K_SW_PEPS_MAX 9
+#define FM10K_SW_PEPS_SUPPORTED 4
+#define FM10K_SW_PEP_PORTS_MAX 4
+
+/*
+ * GLORT
+ */
+#define FM10K_SW_GROUP_MAX 10
+#define FM10K_SW_VFS_MAX 64
+#define FM10K_SW_PFS_GLORT_START 0x1000
+#define FM10K_SW_VF_GLORT_START 0x2000
+#define FM10K_SW_MULTI_GLORT_START 0x3000
+#define FM10K_SW_DPORT_MASK(lport) (1ULL << (lport))
+
+
+/*
+ * CONFIG
+ */
+#define FM10K_SW_CONFIG_MAX 1000
+#define FM10K_SW_FFU_RULE_MAX 256
+#define FM10K_SW_FFU_RULE_ITEM_MAX 10
+
+/*
+ * FFU COUNT
+ */
+#define FM10K_SW_FFU_CNT_BANK 3
+#define FM10K_SW_POLICER_LAST 19
+#define FM10K_SW_FFU_CNT_START (FM10K_SW_POLICER_LAST + 1)
+#define FM10K_SW_FFU_CNT_MAX 100
+
+#define FM10K_SW_PEP_QUEUES_MAX 256
+#define FM10K_SW_VF_QUEUES_MAX (FM10K_SW_PEP_QUEUES_MAX - 1)
+
+#define FM10K_SW_FTAG_SIZE 8
+#define FM10K_SW_PACKET_SIZE_MIN 17
+#define FM10K_SW_PACKET_SIZE_MAX (15 * 1024)
+#define FM10K_SW_MTU_MAX \
+ (FM10K_SW_PACKET_SIZE_MAX - ETHER_HDR_LEN - ETHER_VLAN_ENCAP_LEN)
+#define FM10K_SW_SEG_SIZE_MAX (16 * 1024)
+#define FM10K_SW_TSO_SIZE_MAX (256 * 1024 - 1)
+
+#define FM10K_SW_MEM_POOL_SEG_SIZE 192
+#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
+#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+
+
+
+#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
+#define FM10K_SW_CARD(vname_, dname_) \
+ FM10K_SW_CARD_ID(FM10K_SW_VENDOR_ID_##vname_, \
+ FM10K_SW_DEV_ID_##dname_)
+
+/*
+ * All IDs that may appear in the vendor ID or subsystem vendor ID.
+ */
+#define FM10K_SW_VENDOR_ID_INTEL 0x8086
+#define FM10K_SW_VENDOR_ID_SILICOM 0x1374
+#define FM10K_SW_VENDOR_ID_SILICOM_RB 0x1B2E
+
+
+/*
+ * All IDs that may appear in the device ID or subsystem device ID.
+ */
+#define FM10K_SW_DEV_ID_FM10K 0x15a4
+
+/* Silicom cards */
+#define FM10K_SW_DEV_ID_PE310G4DBIR_T 0x01B0
+#define FM10K_SW_DEV_ID_PE310G4DBIR_SRD 0x01B1
+#define FM10K_SW_DEV_ID_PE310G4DBIR_LRD 0x01B2
+#define FM10K_SW_DEV_ID_PE310G4DBIR_ER 0x01B3
+#define FM10K_SW_DEV_ID_PE310G4DBIR_DA 0x01B4
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS41 0x01B8
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS43 0x01B9
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QL4 0x01BA
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4 0x01C0
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4_REV_2 0x01C4
+#define FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4 0x01C1
+#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
+#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
+/*
+ * SWITCH
+ */
+
+struct fm10k_device_info {
+ uint16_t subvendor;
+ uint16_t subdevice;
+ const char *desc;
+ uint8_t num_ext_ports;
+ uint8_t ext_port_speed;
+ uint8_t num_epls;
+ uint8_t num_peps;
+};
+
+
+struct fm10k_i2c;
+struct fm10k_sbus;
+struct fm10k_hw;
+struct fm10k_ext_ports;
+struct fm10k_dpdk_cfg;
+
+struct fm10k_sw_port_map {
+ uint32_t glort;
+ uint16_t logical_port; /* logical port number */
+ uint16_t physical_port; /* */
+};
+
+struct fm10k_switch {
+ uint32_t *hw_addr;
+ uint32_t *sw_addr;
+ struct fm10k_hw *master_hw;
+ struct fm10k_device_info *info;
+ pthread_mutex_t lock;
+ struct fm10k_i2c *i2c;
+ struct fm10k_sbus *epl_sbus;
+ struct fm10k_ext_ports *ext_ports;
+ sem_t intr_tq;
+ pthread_t intr_task;
+ pthread_t led_task;
+ pthread_t stats_task;
+ uint32_t detaching;
+ uint32_t glort_cam_ram_idx;
+ uint32_t glort_dest_table_idx;
+ uint32_t mcast_dest_table_idx;
+ uint32_t mcast_len_table_idx;
+ uint32_t mcast_vlan_table_idx;
+ uint32_t epl_serdes_code_version_build_id;
+ uint16_t pep_mask; /* mask of non-master peps */
+ uint8_t pepno;
+ uint8_t serdes_loopback;
+ uint8_t epla_no;
+ uint8_t eplb_no;
+ uint8_t mac_addr[6];
+ struct fm10k_dpdk_cfg *dpdk_cfg;
+ struct fm10k_sw_port_map *pep_map;
+ struct fm10k_sw_port_map *epl_map;
+ int inited;
+};
+
+#define FM10K_SW_SWITCH_LOCK(sw_) pthread_mutex_lock(&((sw_)->lock))
+#define FM10K_SW_SWITCH_UNLOCK(sw_) pthread_mutex_unlock(&((sw_)->lock))
+
+#define FM10K_SW_HOWMANY(x, y, y1) (((x) + (y) - 1) / (y1))
+
+
+static inline uint64_t
+fm10k_uptime_us(void)
+{
+ struct timeval tv;
+
+ gettimeofday(&tv, NULL);
+ return ((uint64_t)tv.tv_sec * 1000000 + tv.tv_usec);
+}
+
+
+static inline uint32_t
+fm10k_read_switch_reg(struct fm10k_switch *sw, uint32_t reg)
+{
+ return ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+}
+
+static inline uint64_t
+fm10k_read_switch_reg64(struct fm10k_switch *sw, uint32_t reg)
+{
+ uint64_t temp, result;
+
+ result = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+ temp = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1];
+ result |= temp << 32;
+
+ return result;
+}
+
+static inline void
+fm10k_read_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *results, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ results[i] =
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i];
+}
+
+
+static inline void
+fm10k_write_switch_reg(struct fm10k_switch *sw, uint32_t reg, uint32_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val;
+}
+
+static inline void
+fm10k_write_switch_reg64(struct fm10k_switch *sw, uint32_t reg, uint64_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] = val >> 32;
+}
+
+static inline void
+fm10k_write_switch_reg128(struct fm10k_switch *sw,
+ uint32_t reg, uint64_t val_hi, uint64_t val_lo)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] =
+ val_lo & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] =
+ val_lo >> 32;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 2] =
+ val_hi & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 3] =
+ val_hi >> 32;
+}
+
+static inline void
+fm10k_write_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *data, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i] =
+ data[i];
+}
+
+static inline uint32_t
+fm10k_write_flush(struct fm10k_switch *sw)
+{
+ return ((volatile uint32_t *)sw->master_hw->hw_addr)[FM10K_SW_CTRL];
+}
+
+static inline void
+fm10k_gpio_output_set(struct fm10k_switch *sw, int gpio_pin, int value)
+{
+ uint32_t data;
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_CFG);
+ data |= 1 << gpio_pin;
+ data &= ~(1 << (gpio_pin + 16));
+
+ /* set gpio output */
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_CFG, data);
+
+ /*
+ * Wait 1 msec (PCA spec specifies reset pulse width = 4 ns and
+ * and reset time = 100 ns)
+ */
+ usec_delay(1000);
+ /* set reset */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_DATA);
+ if (value == 0)
+ data &= ~(1 << gpio_pin);
+ else
+ data |= 1 << gpio_pin;
+
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
+}
+
+#define fm10k_udelay usec_delay
+typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
+
+unsigned int fm10k_switch_eplidx_to_eplno
+ (struct fm10k_switch *sw, unsigned int eplidx);
+void fm10k_switch_intr(struct fm10k_hw *hw);
+
+struct fm10k_switch *fm10k_switch_get(void);
+struct fm10k_device_info *fm10k_get_device_info(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func);
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue);
+int fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw,
+ struct fm10k_hw *hw_list[]);
+int fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+void *fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw);
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw);
+void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+
+uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
+uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
+uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
+uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
+uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
+uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
+uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new);
+
+int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
+
+void fm10k_switch_flowset_switchto(const char *name);
+void fm10k_switch_show_port(void);
+void fm10k_switch_show_ffu(void);
+void fm10k_switch_show_bank(void);
+
+void fm10k_flow_list_init(void *flow_list);
+const struct rte_flow_ops *fm10k_flow_ops_get(void);
+
+#endif
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 2/5] net/fm10k: add epl serdes and port control functions
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
` (2 subsequent siblings)
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add epl serdes include loading spico,
controling pcsl, dma, dfe, ical. Add spico code.
Add state machine for epl lane and port, it creates
a pthread to handle the state changing event.
Add external port management, which will use
state machine to handle the event from lane and port.
The lane state will change between DOWN, WAIT_PLL_LOCK,
WAIT_SIGNAL_OK, WAIT_DFE_ICAL, WAIT_DFE_PCAL, UP.
The port state will change between DOWN, WAIT_LANE_UP, UP.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 4 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 841 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 81 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
9 files changed, 6207 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index a628e2d..ac2d96f 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -85,6 +85,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.c b/drivers/net/fm10k/switch/fm10k_ext_port.c
new file mode 100644
index 0000000..68eab32
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.c
@@ -0,0 +1,841 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_serdes.h"
+#include "fm10k_sm.h"
+#include "fm10k_switch.h"
+
+
+#define LANE_EVENT(e_) FM10K_SW_EXT_PORT_LANE_EVENT_##e_
+#define LANE_TIMER(t_) FM10K_SW_EXT_PORT_LANE_TIMER_##t_
+#define LANE_TIMEOUT(t_) FM10K_SW_EXT_PORT_LANE_##t_##_TIMEOUT_MS
+#define LANE_POLL(t_) FM10K_SW_EXT_PORT_LANE_##t_##_POLL_MS
+#define LANE_STATE(s_) FM10K_SW_EXT_PORT_LANE_STATE_##s_
+#define LANE_FLAG(f_) FM10K_SW_EXT_PORT_LANE_FLAG_##f_
+#define LANE_FLAGS(f_) FM10K_SW_EXT_PORT_LANE_FLAGS_##f_
+#define LANE_FLAG_SET(l_, f_) ((l_)->flags & LANE_FLAG(f_))
+#define LANE_FLAGS_SET(l_, f_, f1_) \
+ (((l_)->flags & LANE_FLAGS(f_)) == LANE_FLAGS(f1_))
+
+#define PORT_EVENT(e_) FM10K_SW_EXT_PORT_EVENT_##e_
+#define PORT_STATE(s_) FM10K_SW_EXT_PORT_STATE_##s_
+
+static void fm10k_ext_ports_epl_intr
+ (struct fm10k_ext_ports *ports, unsigned int eplno);
+static void fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip);
+static void fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_bring_down(struct fm10k_ext_port *port);
+static void fm10k_ext_port_lane_process_event
+ (struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_lane_process_timer
+ (struct fm10k_sm *sm, uint16_t timer);
+static void fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_phy_enable(struct fm10k_ext_port *port);
+static void fm10k_ext_port_phy_disable(struct fm10k_ext_port *port);
+
+struct fm10k_ext_ports *
+fm10k_ext_ports_attach(struct fm10k_switch *sw)
+{
+// struct fm10k_pep *pep = sw->pep;
+ struct fm10k_ext_ports *ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int is_quad, lane_speed;
+ unsigned int i, j;
+ unsigned int eplidx, laneno;
+
+ FM10K_SW_TRACE("attaching external ports\n");
+
+ ports = (struct fm10k_ext_ports *)rte_zmalloc
+ ("fm10k_ext_ports", sizeof(struct fm10k_ext_ports), 0);
+ if (ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports\n");
+ goto fail;
+ }
+
+ ports->ports = (struct fm10k_ext_port *)rte_zmalloc
+ ("fm10k_ext_port_list",
+ sizeof(struct fm10k_ext_port) * cfg->num_ext_ports, 0);
+ if (ports->ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports list\n");
+ goto fail;
+ }
+ ports->sw = sw;
+ ports->num_ports = cfg->num_ext_ports;
+ ports->ports_per_epl = cfg->num_ext_ports / cfg->num_epls;
+ ports->epl_mask = 0;
+
+ switch (cfg->ext_port_speed) {
+ default:
+ case 10:
+ is_quad = 0;
+ lane_speed = 10;
+ break;
+ case 25:
+ is_quad = 0;
+ lane_speed = 25;
+ break;
+ case 40:
+ is_quad = 1;
+ lane_speed = 10;
+ break;
+ case 100:
+ is_quad = 1;
+ lane_speed = 25;
+ break;
+ }
+
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ port->ports = ports;
+ fm10k_ext_port_eplidx_lane(ports, i, &eplidx, &laneno);
+ port->portno = i;
+ port->eplno = eplidx ? sw->eplb_no : sw->epla_no;
+ snprintf(port->name, sizeof(port->name), "EPL[%u][%u]",
+ port->eplno, port->first_lane);
+ port->first_lane = laneno;
+ port->is_quad = is_quad;
+ port->num_lanes = port->is_quad ? 4 : 1;
+ port->lane_speed = lane_speed;
+ port->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_NUM_EVENTS, 0,
+ fm10k_ext_port_process_event, NULL,
+ (void *)port);
+ if (port->sm == NULL)
+ goto fail;
+ port->sm->portno = port->portno;
+ port->sm->laneno = 0;
+ port->an_im = FM10K_SW_AN_IM_ALL;
+ port->link_im = FM10K_SW_LINK_IM_ALL;
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ lane->port = port;
+ lane->abs_laneno = port->eplno * FM10K_SW_EPL_LANES + j;
+ lane->rel_laneno = j;
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+ lane->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_LANE_NUM_EVENTS,
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS,
+ fm10k_ext_port_lane_process_event,
+ fm10k_ext_port_lane_process_timer,
+ (void *)lane);
+ if (lane->sm == NULL)
+ goto fail;
+ lane->sm->portno = lane->port->portno;
+ lane->sm->laneno = lane->rel_laneno;
+ }
+
+ ports->epl_mask |= (1 << port->eplno);
+ }
+
+ return (ports);
+fail:
+ if (ports)
+ fm10k_ext_ports_detach(ports);
+ return NULL;
+}
+
+void
+fm10k_ext_ports_detach(struct fm10k_ext_ports *ports)
+{
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i, j;
+
+ FM10K_SW_TRACE("detaching external ports\n");
+
+ if (ports->ports) {
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+
+ if (port->sm)
+ fm10k_sm_detach(port->sm);
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ if (lane->sm)
+ fm10k_sm_detach(lane->sm);
+ }
+
+ fm10k_ext_port_phy_disable(port);
+ }
+ rte_free(ports->ports);
+ }
+
+ rte_free(ports);
+}
+
+unsigned int
+fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports, uint64_t gid)
+{
+ unsigned int i;
+ uint16_t epl_mask;
+
+ /*
+ * Process EPL interrupts for all EPLs that are in use and have
+ * their bit set in global interrupt detect.
+ */
+ epl_mask = ports->epl_mask &
+ FM10K_SW_REG_FIELD64(gid, GLOBAL_INTERRUPT_DETECT_EPL);
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++)
+ if (epl_mask & (1 << i))
+ fm10k_ext_ports_epl_intr(ports, i);
+
+ return ports->epl_mask;
+}
+
+static void
+fm10k_ext_ports_epl_intr(struct fm10k_ext_ports *ports, unsigned int eplno)
+{
+ struct fm10k_switch *sw = ports->sw;
+ struct fm10k_ext_port *port;
+ uint64_t gid;
+ unsigned int i, j;
+ uint32_t epl_ip;
+
+ /*
+ * The global interrupt detect architecture makes it somewhat
+ * challenging to ensure events are not lost and spurious interrupts
+ * are not generated.
+ *
+ * The EPLs have an internal interrupt hierarchy whose reporting is
+ * rooted at EPL_IP. Whenever an EPL transitions from having no
+ * unmasked interrupts pending to at least one unmasked interrupt
+ * pending or from having at least one unmasked interrupt pending to
+ * no unmasked interrupts pending, it sends a vector to the
+ * interrupt controller over a serial bus indicating the
+ * at-least-one/none pending state, and the controller updates the
+ * bit for that EPL in the global interrupt detect register
+ * accordingly.
+ *
+ * Because of the update-vector-in-flight potential, there is a race
+ * between re-enabling the EPL PCIe interrupt mask after servicing
+ * the EPL interrupts and the all-clear vector from the EPL reaching
+ * the interrupt controller - it's possible for the software to
+ * clear all the EPL-internal interrupt sources during servicing and
+ * then re-enable the EPL interrupt in the PCIe mask before the
+ * all-clear vector reaches the interrupt controller and clears the
+ * EPL pending bit in the global interrupt detect register. This
+ * would result in a spurious EPL interrupt, potentially for every
+ * necessary EPL interrupt.
+ *
+ * The approach taken here to avoid spurious interrupts is to first
+ * mask off all EPL-internal sources, then poll the EPL bits in the
+ * global interrupt detect register until they clear (indicating the
+ * update vector from the EPL has reached the interrupt controller).
+ * Then software interrupt processing proceeds, and the EPL-internal
+ * masks are restored to their desired state at the end. This
+ * ensures that when the EPL PCIe interrupt mask is re-enabled, the
+ * EPL pending bits in the global interrupt detect register will
+ * only be set if there are new, unprocessed EPL events of interest.
+ *
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* Grab epl_ip before it is cleared by the masks below */
+ epl_ip = fm10k_read_switch_reg(sw, FM10K_SW_EPL_IP(eplno));
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_AN_IM(eplno, port->first_lane),
+ FM10K_SW_AN_IM_ALL);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IM(eplno, port->first_lane),
+ FM10K_SW_LINK_IM_ALL);
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(eplno, j),
+ FM10K_SW_SERDES_IM_ALL);
+ }
+ }
+
+ /*
+ * Wait for the all-clear vector to propagate over the internal
+ * interrupt serial bus to the global interrupt detect.
+ */
+ do {
+ gid = fm10k_read_switch_reg64
+ (sw, FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ } while (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(eplno));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ /*
+ * Proceed with interrupt processing per-external-port.
+ */
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+ fm10k_ext_port_intr(port, epl_ip);
+ }
+}
+
+static void
+fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ uint32_t serdes_ip, data = 0;
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (epl_ip & FM10K_SW_EPL_IP_SERDES_INTERRUPT(i)) {
+ FM10K_SW_SWITCH_LOCK(sw);
+ serdes_ip = fm10k_read_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i));
+ serdes_ip &= ~lane->im;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i), serdes_ip);
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK)
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_STATUS(port->eplno,
+ i));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK) {
+ fm10k_sm_send_event(lane->sm,
+ (data &
+ FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK) ?
+ LANE_EVENT(SIGNAL_OK) : LANE_EVENT(SIGNAL_NOK));
+ }
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RX_PLL_LOCK));
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_TX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(TX_PLL_LOCK));
+ } else {
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RESTORE_IM));
+ }
+ }
+}
+
+void
+fm10k_ext_port_up(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_UP));
+}
+
+void
+fm10k_ext_port_down(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_DOWN));
+}
+
+unsigned int
+fm10k_ext_port_isup(struct fm10k_ext_port *port)
+{
+ /* no locked needed as we are just inspecting
+ * the current value of an int
+ */
+ unsigned int port_state = port->sm->state;
+
+ return (port_state == FM10K_SW_EXT_PORT_STATE_UP);
+}
+
+static void
+fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port *port = sm->ctx;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ unsigned int lanes_up;
+
+ switch (sm->state) {
+ case PORT_STATE(DOWN):
+ if (event == PORT_EVENT(BRING_UP)) {
+ fm10k_ext_port_phy_enable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event
+ (lane->sm, LANE_EVENT(BRING_UP));
+ }
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ }
+ break;
+
+ case PORT_STATE(WAITING_FOR_LANES):
+ if (event == PORT_EVENT(LANE_UP)) {
+ lanes_up = 0;
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (lane->sm->state != LANE_STATE(UP))
+ break;
+ lanes_up++;
+ }
+ if (lanes_up == port->num_lanes) {
+ sm->state = PORT_STATE(UP);
+ FM10K_SW_INFO
+ ("Port %d is LINK UP", port->portno);
+ }
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ }
+ break;
+
+ case PORT_STATE(UP):
+ if (event == PORT_EVENT(LANE_DOWN)) {
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ }
+ }
+}
+
+static void
+fm10k_ext_port_bring_down(struct fm10k_ext_port *port)
+{
+ struct fm10k_sm *sm = port->sm;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+
+ sm->state = PORT_STATE(DOWN);
+
+ fm10k_ext_port_phy_disable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event(lane->sm, LANE_EVENT(BRING_DOWN));
+ }
+}
+
+static void
+fm10k_ext_port_lane_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ struct fm10k_ext_port *port = lane->port;
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ int error;
+
+ switch (sm->state) {
+ case LANE_STATE(DOWN):
+ if (event == LANE_EVENT(BRING_UP)) {
+ lane->flags &= ~LANE_FLAGS(PLLS_LOCKED);
+
+ /*
+ * Clear all pending interrupts.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, lane->rel_laneno),
+ FM10K_SW_SERDES_IP_ALL);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->im &= ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+ }
+ break;
+
+ case LANE_STATE(WAIT_PLL_LOCK):
+ if (event == LANE_EVENT(RX_PLL_LOCK) ||
+ event == LANE_EVENT(TX_PLL_LOCK)) {
+ if (event == LANE_EVENT(RX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(RX_LOCKED);
+ if (event == LANE_EVENT(TX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(TX_LOCKED);
+ if (LANE_FLAGS_SET(lane, PLLS_LOCKED, PLLS_LOCKED)) {
+ error = fm10k_epl_serdes_continue_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_SIGNAL_OK);
+ /* unmask RX signal OK interrupt */
+ lane->im |=
+ (FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ lane->im &=
+ ~FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ }
+ }
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_SIGNAL_OK):
+ if (event == LANE_EVENT(SIGNAL_OK)) {
+ sm->state = LANE_STATE(WAIT_DFE_ICAL);
+
+ fm10k_epl_serdes_configure_for_dfe_tuning(lane);
+ fm10k_epl_serdes_start_dfe_ical(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ sm->state = LANE_STATE(WAIT_DFE_PCAL);
+
+ fm10k_epl_serdes_start_dfe_pcal(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+ sm->state = LANE_STATE(UP);
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_UP));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT)) ||
+ (event == LANE_EVENT(SIGNAL_NOK))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(UP):
+ if (event == LANE_EVENT(SIGNAL_NOK))
+ fm10k_ext_port_restart_lane(lane);
+ else if (event == LANE_EVENT(BRING_DOWN))
+ fm10k_ext_port_disable_lane(lane);
+ break;
+ }
+
+ /* Always restore the interrupt mask */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(port->eplno,
+ lane->rel_laneno),
+ lane->im);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_ext_port_lane_process_timer(struct fm10k_sm *sm, uint16_t timer)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ uint32_t status;
+
+ switch (sm->state) {
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_ICAL);
+ fm10k_epl_serdes_dfe_ical_status(lane, &status);
+
+ if (status == FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_ICAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else if (status == FM10K_SW_SERDES_DFE_ICAL_CONVERGED)
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_FAILED));
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_PCAL);
+
+ fm10k_epl_serdes_dfe_pcal_status(lane, &status);
+ if (status == FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_PCAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else {
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ }
+ }
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+ struct fm10k_ext_port *port = lane->port;
+ int error;
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->flags &= ~FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED;
+ lane->im |= FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ lane->im = ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_DOWN));
+}
+
+static void
+fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+
+ fm10k_sm_timer_cancel(sm, LANE_TIMER(DFE_TUNING));
+ fm10k_epl_serdes_disable(lane);
+
+ /*
+ * Disable all interrupts.
+ */
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+
+ sm->state = LANE_STATE(DOWN);
+}
+
+void
+fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane)
+{
+ FM10K_SW_ASSERT(ext_port_idx < ports->num_ports,
+ ("ext_port_idx out of range"));
+
+ *eplidx = ext_port_idx / ports->ports_per_epl;
+ *lane = ext_port_idx % ports->ports_per_epl;
+}
+
+
+static void
+fm10k_ext_port_phy_enable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the
+ * PCA9505 that the QSFP control lines are connected
+ * to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get
+ * at the PCA9505 that the QSFP control lines
+ * are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x10);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x10);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to enable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_phy_disable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x60);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x61);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to disable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.h b/drivers/net/fm10k/switch/fm10k_ext_port.h
new file mode 100644
index 0000000..9cc0574
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_EXT_PORT_H_
+#define _FM10K_SW_EXT_PORT_H_
+
+#include "fm10k_regs.h"
+#include "fm10k_sm.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+
+struct fm10k_sbus;
+struct fm10k_switch;
+
+#define FM10K_SW_TRACE_ENABLE 0
+
+enum fm10k_ext_port_lane_state {
+ FM10K_SW_EXT_PORT_LANE_STATE_DOWN,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_ICAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_PCAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_UP
+};
+
+enum fm10k_ext_port_lane_event {
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_TX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_NOK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_COMPLETE,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_FAILED,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_TIMED_OUT,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RESTORE_IM,
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_DOWN,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_EVENTS
+};
+
+enum fm10k_ext_port_lane_timer {
+ FM10K_SW_EXT_PORT_LANE_TIMER_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_DFE_TUNING,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS
+};
+
+#define FM10K_SW_EXT_PORT_LANE_PLL_LOCK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_SIGNAL_OK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_POLL_MS 200
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_TIMEOUT_MS 3000
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_POLL_MS 100
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_TIMEOUT_MS 2000
+
+
+enum fm10k_ext_port_state {
+ FM10K_SW_EXT_PORT_STATE_DOWN,
+ FM10K_SW_EXT_PORT_STATE_WAITING_FOR_LANES,
+ FM10K_SW_EXT_PORT_STATE_UP
+};
+
+enum fm10k_ext_port_event {
+ FM10K_SW_EXT_PORT_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_EVENT_BRING_DOWN,
+ FM10K_SW_EXT_PORT_EVENT_LANE_UP,
+ FM10K_SW_EXT_PORT_EVENT_LANE_DOWN,
+
+ FM10K_SW_EXT_PORT_NUM_EVENTS
+};
+
+struct fm10k_ext_port;
+
+struct fm10k_ext_port_lane {
+ struct fm10k_ext_port *port;
+ struct fm10k_sm *sm;
+ uint32_t im;
+ uint16_t dfe_poll_time_ms;
+ uint8_t abs_laneno;
+ uint8_t rel_laneno;
+ uint8_t flags;
+};
+
+#define FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED 0x01
+#define FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED 0x02
+
+#define FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED \
+ (FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED | \
+ FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED)
+
+struct fm10k_ext_ports;
+
+struct fm10k_ext_port {
+ struct fm10k_ext_ports *ports;
+ struct fm10k_sm *sm;
+ struct fm10k_ext_port_lane lanes[FM10K_SW_EPL_LANES];
+ char name[10];
+ uint8_t portno;
+ uint8_t eplno;
+ uint8_t lport; /* filled in by switch */
+ uint8_t first_lane; /* range is [0..3] */
+ uint8_t num_lanes;
+ uint8_t is_quad;
+ uint8_t lane_speed;
+ uint8_t last_led_flags;
+ uint32_t an_im;
+ uint32_t link_im;
+};
+
+#define FM10K_SW_EXT_PORT_LED_FLAG_UP 0x01
+#define FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE 0x02
+
+struct fm10k_ext_ports {
+ struct fm10k_switch *sw;
+ struct fm10k_ext_port *ports;
+ uint16_t epl_mask;
+ uint8_t num_ports;
+ uint8_t ports_per_epl;
+};
+
+#define FM10K_SW_EXT_PORTS_EPL_USED(p_, n_) ((p_)->epl_mask & (1 << (n_)))
+
+struct fm10k_ext_ports *fm10k_ext_ports_attach(struct fm10k_switch *sw);
+void fm10k_ext_ports_detach(struct fm10k_ext_ports *ports);
+unsigned int fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports,
+ uint64_t gid);
+void fm10k_ext_port_up(struct fm10k_ext_port *port);
+void fm10k_ext_port_down(struct fm10k_ext_port *port);
+unsigned int fm10k_ext_port_isup(struct fm10k_ext_port *port);
+void fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane);
+
+#endif /* _FM10K_SW_EXT_PORT_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.c b/drivers/net/fm10k/switch/fm10k_serdes.c
new file mode 100644
index 0000000..01b76a3
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.c
@@ -0,0 +1,1936 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_serdes.h"
+#include "fm10k_spico_code.h"
+#include "fm10k_switch.h"
+
+
+#define FM10K_SW_SERDES_BIST_TIMEOUT_MS 5000
+#define FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS 5000
+#define FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS 3000
+#define FM10K_SW_SERDES_INT02_TIMEOUT_MS 500
+
+#define FM10K_SW_SERDES_CONFIG_DELAY_US 20
+#define FM10K_SW_SERDES_RESET_DELAY_US 22
+#define FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US 200
+
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_25G 0xa5
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_10G 0x42
+
+#define FM10K_SW_SERDES_WIDTH_40 3 /* 25G */
+#define FM10K_SW_SERDES_WIDTH_20 1 /* 10G */
+
+#define FM10K_SW_SERDES_EQ_SEL_PRECUR 0
+#define FM10K_SW_SERDES_EQ_SEL_ATTEN 1
+#define FM10K_SW_SERDES_EQ_SEL_POSTCUR 2
+
+#define FM10K_SW_SERDES_DFE_DEFAULT_HF 0x00
+#define FM10K_SW_SERDES_DFE_DEFAULT_LF 0x0c
+#define FM10K_SW_SERDES_DFE_DEFAULT_DC 0x38
+#define FM10K_SW_SERDES_DFE_DEFAULT_BW 0x0f
+
+#define FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES 10
+#define FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES 500
+
+#define FM10K_SERDES_SPICO_REG_0X00 0x00
+#define FM10K_SERDES_SPICO_REG_0X01 0x01
+#define FM10K_SERDES_SPICO_REG_0X02 0x02
+#define FM10K_SERDES_SPICO_REG_0X03 0x03
+#define FM10K_SERDES_SPICO_REG_0X04 0x04
+#define FM10K_SERDES_SPICO_REG_0X05 0x05
+#define FM10K_SERDES_SPICO_REG_0X07 0x07
+#define FM10K_SERDES_SPICO_REG_0X08 0x08
+#define FM10K_SERDES_SPICO_REG_0X0A 0x0A
+#define FM10K_SERDES_SPICO_REG_0X0B 0x0b
+#define FM10K_SERDES_SPICO_REG_0X14 0x14
+#define FM10K_SERDES_SPICO_REG_0X16 0x16
+
+
+/*
+ * These values correspond to IES FM_PORT_LINK_OPTIMIZATION_BALANCE
+ * settings.
+ */
+#define FM10K_SW_LINK_OPT_PARAM_A10G 0x4800
+#define FM10K_SW_LINK_OPT_PARAM_B10G 0xA000
+#define FM10K_SW_LINK_OPT_PARAM_A25G 0x10001
+#define FM10K_SW_LINK_OPT_PARAM_B25G 0x10001
+
+#define FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD 10
+
+
+static int fm10k_spico_ram_bist(struct fm10k_sbus *sb);
+static int fm10k_load_epl_spico_code(struct fm10k_switch *sw);
+static int fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id);
+static int fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb);
+static int fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id);
+static int fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_swap_image_check_crc(struct fm10k_sbus *sb,
+ unsigned int crc_code);
+static int fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words);
+static int fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id);
+static int fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id);
+static int fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider);
+static int fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static void fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static int fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold);
+static int fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq);
+static int fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms);
+static int fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data,
+ uint32_t mask, uint32_t *result);
+static int fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result);
+static int fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param);
+static void fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane);
+static int fm10k_epl_serdes_configure_near_loopback
+ (struct fm10k_ext_port_lane *lane);
+static int fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param);
+static int fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status);
+static int fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged);
+
+
+int
+fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: initializing EPL serdes", sb->name);
+
+ error = fm10k_spico_ram_bist(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_load_epl_spico_code(sw);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_spico_ram_bist(struct fm10k_sbus *sb)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: starting SPICO RAM BIST", sb->name);
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x03);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x05);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_BIST_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, &data);
+ if (error)
+ goto done;
+
+ if (data & 0x18) {
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ if ((data & 0x18) != 0x08) {
+ FM10K_SW_INFO("sbus %s: SPICO RAM "
+ "BIST failed with bad status (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+ goto done;
+ }
+ }
+ fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SPICO RAM BIST timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_load_epl_spico_code(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ const uint16_t *sbm_code_image;
+ uint32_t sbm_code_size;
+ uint32_t sbm_code_version_build_id;
+ const uint16_t *serdes_code_image;
+ uint32_t serdes_code_size;
+ uint32_t serdes_code_version_build_id;
+ const uint16_t *swap_code_image;
+ uint32_t swap_code_size;
+ unsigned int swap_crc_code;
+ int error = 0;
+
+ FM10K_SW_TRACE("sbus %s: loading SPICO code", sb->name);
+
+ sbm_code_image = fm10000_sbus_master_code_prd;
+ sbm_code_size = fm10000_sbus_master_code_size_prd;
+ sbm_code_version_build_id =
+ fm10000_sbus_master_code_version_build_id_prd;
+
+ FM10K_SW_TRACE("sbus %s: sbm code version=%4.4x build=%4.4x",
+ sb->name, sbm_code_version_build_id >> 16,
+ sbm_code_version_build_id & 0xffff);
+
+ serdes_code_image = fm10000_serdes_spico_code_prd2;
+ serdes_code_size = fm10000_serdes_spico_code_size_prd2;
+ serdes_code_version_build_id =
+ fm10000_serdes_spico_code_version_build_id_prd2;
+ sw->epl_serdes_code_version_build_id = serdes_code_version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: serdes code version=%4.4x build=%4.4x",
+ sb->name, serdes_code_version_build_id >> 16,
+ serdes_code_version_build_id & 0xffff);
+
+ swap_code_image = fm10000_serdes_swap_code_prd2;
+ swap_code_size = fm10000_serdes_swap_code_size_prd2;
+
+ error = fm10k_sbm_spico_upload_image(sb, sbm_code_image,
+ sbm_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_check_crc_version_build_id(sb,
+ sbm_code_version_build_id);
+ if (error)
+ goto done;
+
+ if (swap_code_size > 0) {
+ if ((sbm_code_version_build_id & 0x00008000) == 0) {
+ error = fm10k_serdes_swap_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x1a;
+ } else {
+ error = fm10k_serdes_swap_alt_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x04;
+ }
+ if (error)
+ goto done;
+
+ error = fm10k_swap_image_check_crc(sb, swap_crc_code);
+ if (error)
+ goto done;
+ }
+
+ error = fm10k_serdes_spico_upload_image(sb,
+ FM10K_SW_SBUS_ADDR_BROADCAST,
+ serdes_code_image, serdes_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_check_crc_version_build_id(sw,
+ serdes_code_version_build_id);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+static int
+fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM SPICO image "
+ "(%u words)", sb->name, num_words);
+
+ data = (1 << 6) | (1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ data = 0x80000000 | ((uint32_t)image[i] << 16) | i;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM image upload time %u ms",
+ sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ data = (1 << 6);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X16, data);
+ if (error)
+ goto done;
+
+ data = (1 << 6) | (1 << 8);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id)
+{
+ int error;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SBM SPICO code CRC "
+ "and version", sb->name);
+
+ error = fm10k_sbm_spico_do_crc(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_get_version_build_id(sb, &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO code version "
+ "compare failed (expected 0x%08x, got 0x%08x)",
+ sb->name, expected_version_build_id, version_build_id);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM SPICO image "
+ "version=0x%4.4x build_id=0x%4.4x", sb->name,
+ version_build_id >> 16,
+ version_build_id & 0xffff);
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM SPICO CRC "
+ "check", sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x02, 0, &crc);
+ if (error)
+ goto done;
+
+ crc = crc >> 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO CRC check failed "
+ "(CRC interrupt returned 0xffff)", sb->name);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM SPICO CRC check "
+ "passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM SPICO CRC "
+ "check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id)
+{
+ int error;
+ uint32_t result;
+
+ FM10K_SW_TRACE("sbus %s: getting SBM code version",
+ sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x00, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id = result & 0xffff0000;
+ error = fm10k_sbm_spico_int(sb, 0x01, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id |= (result >> 16) & 0xffff;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ error = fm10k_sbm_spico_int(sb, 0x1c, 0, &addr);
+ if (error)
+ goto done;
+
+ addr >>= 16;
+ if (addr == 0xffff) {
+ FM10K_SW_INFO("sbus %s: invalid build - cannot "
+ "upload SBM swap image", sb->name);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, addr);
+ if (error)
+ goto done;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, 0x80000000 | addr);
+ if (error)
+ goto done;
+
+ for (i = 0; i < num_words - 2; i += 3) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0xc0000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10) |
+ ((uint32_t)image[i + 2] << 20));
+ if (error)
+ goto done;
+ }
+
+ if (num_words - i == 2) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x80000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10));
+ if (error)
+ goto done;
+ } else if (num_words - i == 1) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x40000000 | image[i]);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image upload time "
+ "%u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~(1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: alt-uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 10) | (1 << 11);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ addr = 0x0400;
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X04,
+ 0x8000 | ((uint32_t)image[i + 1] << 16) |
+ (addr + i));
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image alt-upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~((1 << 10) | (1 << 11));
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+static int
+fm10k_swap_image_check_crc(struct fm10k_sbus *sb, unsigned int crc_code)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM swap image CRC "
+ "check", sb->name);
+
+ error =
+ fm10k_sbm_spico_int(sb, crc_code, 0, &crc);
+ if (error)
+ goto done;
+
+ crc >>= 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM swap image CRC check "
+ "failed (CRC interrupt 0x%02x returned 0xffff)",
+ sb->name, crc_code);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM swap image CRC "
+ "check passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM swap image "
+ "CRC check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ uint32_t reg07;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading serdes SPICO image "
+ "to bus address 0x%02x (%u words)", sb->name, dev, num_words);
+
+ reg07 = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X03, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ data = (1 << 30);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, data);
+ if (error)
+ goto done;
+
+ data = (1 << 4) | (1 << 5);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i += 3) {
+ data = 0xc0000000 | image[i];
+ if (i + 1 < num_words) {
+ data |= (uint32_t)image[i + 1] << 10;
+
+ if (i + 2 < num_words)
+ data |= (uint32_t)image[i + 2] << 20;
+ }
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X0A, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: serdes SPICO image upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, 0);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X01, 0x20000000);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X0B, data);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 1);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ unsigned int first = 0;
+ unsigned int last = FM10K_SW_EPLS_MAX * FM10K_SW_EPL_LANES - 1;
+ unsigned int i;
+ int error;
+ uint32_t data;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SPICO code CRC and "
+ "version for serdes %u through %u", sb->name, first, last);
+
+ for (i = first; i <= last; i++) {
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(i), 0xff, &data);
+ if (error || data != 0x01) {
+ FM10K_SW_TRACE("sbus %s: skipping "
+ "serdes %u (error=%d, data=0x%08x)", sb->name, i,
+ error, data);
+ continue;
+ }
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, i);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_get_version_build_id(sw, i,
+ &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SERDES SPICO code "
+ "version compare failed (expected 0x%08x, got "
+ "0x%08x)", sb->name, expected_version_build_id,
+ version_build_id);
+ error = -1;
+ goto done;
+ }
+ }
+ FM10K_SW_TRACE("sbus %s: serdes %u through %u are OK",
+ sb->name, first, last);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: performing SPICO "
+ "CRC check", sb->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3c, 0, &crc);
+ if (error)
+ goto done;
+
+ if (crc != 0) {
+ FM10K_SW_INFO("sbus %s: serdes %u: SPICO CRC check "
+ "failed (CRC interrupt returned 0x%08x)",
+ sb->name, serdes, crc);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t data;
+ uint8_t addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: resetting SPICO",
+ sb->name, serdes);
+
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+
+ data = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X0B, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 1);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X08, 0);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: getting code "
+ "version", sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x00, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id = data << 16;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3f, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id |= data & 0xffff;
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: SBM interrupt 0x%02x "
+ "(param=0x%08x)", sb->name, int_num, param);
+
+ /* int write */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X02,
+ (param << 16) | int_num);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, &data);
+ if (error)
+ goto done;
+
+ data |= (1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X08, &data);
+ if (error)
+ goto done;
+
+ if ((data & 0x8000) || ((data & 0x3ff) == 0)) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ FM10K_SW_TRACE("sbus %s: SBM "
+ "interrupt 0x%02x (param=0x%08x, result=0x%08x)",
+ sb->name, int_num, param, data);
+ *result = data;
+ goto done;
+ }
+ }
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SBM interrupt timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ uint64_t start_time_us, elapsed_time_ms;
+ unsigned int sbus_addr;
+ int error;
+ uint32_t data, intr;
+
+ sbus_addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: (sbus addr 0x%02x) "
+ "interrupt 0x%04x (param=0x%08x)", sb->name, serdes, sbus_addr,
+ int_num, param);
+
+ /* int write */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ error = fm10k_sbus_write(sb, sbus_addr,
+ FM10K_SERDES_SPICO_REG_0X03,
+ ((uint32_t)int_num << 16) | param);
+ if (error)
+ goto done;
+ }
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "(2) after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ FM10K_SW_TRACE("sbus %s: serdes %u: interrupt "
+ "0x%04x (param=0x%08x, result=0x%08x)", sb->name, serdes,
+ int_num, param, data);
+ if (result)
+ *result = data;
+ }
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int divider, width_mode;
+ int error = 0;
+ uint32_t data;
+
+ /*
+ * Bit rate and width
+ */
+ if (speed == 25) {
+ /* 25G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_25G;
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_10G;
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ /*
+ * state = CONFIGURED
+ */
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring",
+ sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_bit_rate(sw, serdes, divider);
+ if (error)
+ goto done;
+
+ fm10k_epl_serdes_set_pcsl_width_mode(sw, serdes,
+ width_mode);
+
+ /*
+ * Data select
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX data select",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_spico_int02_retry(sw, serdes, 0x1ff,
+ FM10K_SW_SERDES_INT02_TIMEOUT_MS);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_reg_write(sw, serdes, 0x21, 0x0c00);
+ if (error)
+ goto done;
+
+ /*
+ * Configure TX equalization
+ *
+ * Use defaults - copper and optical are the same.
+ *
+ * DEFAULTS| precursor | cursor | postcursor |
+ * --------+-----------+--------+------------+
+ * copper | 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ * optical| 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursors",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_ATTEN, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_PRECUR, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_POSTCUR, 15);
+ if (error)
+ goto done;
+
+ /*
+ * Configure 'options', which means:
+ * - PLL calibration mode
+ * - Phase slip
+ * - RX termination
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting PLL calibration mode",
+ sw->epl_sbus->name, serdes);
+ /* enable PLL calibration */
+ data = (1 << 0) | (1 << 1);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x11, data);
+ if (error)
+ goto done;
+ /* There appears to be no action to take for phase slip */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring RX termination",
+ sw->epl_sbus->name, serdes);
+ /* RX termination appears to default to AVDD */
+ data = (1 << 0);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x2b, data);
+ if (error)
+ goto done;
+
+ /*
+ * Set TX and RX lane polarity.
+ * Assuming no inversion required.
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring TX/RX lane polarity",
+ sw->epl_sbus->name, serdes);
+ data = 0x0300;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x13, data, NULL);
+ if (error)
+ goto done;
+
+ /*
+ * Force local fault
+ */
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /*
+ * Enable TX/RX
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX/RX",
+ sw->epl_sbus->name, serdes);
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /*
+ * Clear and enable interrupts
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring interrupts",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_sbus_write(sw->epl_sbus,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes),
+ FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int width_mode;
+ int error = 0;
+ int near_loopback = sw->serdes_loopback;
+
+ if (speed == 25) {
+ /* 25G */
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_pcsl_bit_slip(sw, serdes);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_width_mode(sw, serdes, width_mode);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX output",
+ sw->epl_sbus->name, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x07);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_init_signal_ok(sw, serdes, 0);
+ if (error)
+ goto done;
+
+ if (near_loopback) {
+ /* enable near loopback */
+ error = fm10k_epl_serdes_configure_near_loopback(lane);
+ if (error)
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error = 0;
+
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /* is this delay necessary? in some places the IES SDK does this */
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting bit rate",
+ sw->epl_sbus->name, serdes);
+
+ data = divider & 0x7ff;
+ data |= (1 << 12);
+ data |= (1 << 15);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x05, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting width mode to %u",
+ sw->epl_sbus->name, serdes, width_mode);
+
+ data = FM10K_SW_SERDES_WIDTH_20 | (FM10K_SW_SERDES_WIDTH_20 << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+ data = width_mode | (width_mode << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+done:
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ return error;
+}
+
+static int
+fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursor %u to %u",
+ sw->epl_sbus->name, serdes, which, tx_eq);
+
+ data = (tx_eq & 0xff) | (which << 14);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x15, data);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+void
+fm10k_epl_serdes_set_signal_detect(struct fm10k_ext_port_lane *lane,
+ unsigned int override)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int eplno = lane->port->eplno;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal detect to %s",
+ sw->epl_sbus->name, lane->abs_laneno,
+ (override == 0) ? "NORMAL" :
+ (override == 1) ? "FORCE_GOOD" :
+ (override == 2) ? "FORCE_BAD" : "<unknown>");
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno));
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE,
+ override, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno), data);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ unsigned int eplno, lane;
+ uint32_t pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl width mode",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ if (width_mode == FM10K_SW_SERDES_WIDTH_40) {
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ } else {
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ }
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static int
+fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes)
+{
+ unsigned int eplno, lane;
+ int error;
+ uint32_t int_return, pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl bit slip",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x0c, (1 << 7), &int_return);
+ if (error)
+ goto done;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ if (int_return & 0x1)
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0c, (1 << 8));
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold)
+{
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal OK threshold to %u",
+ sw->epl_sbus->name, serdes, threshold);
+
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x20, 0x20, NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_read_modify_write(sw, serdes, 0x080,
+ (threshold & 0xf) << 2, 0x3c, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t result;
+ int error;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < timeout_ms) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x02, data, &result);
+ if (result == 0x02) {
+ error = 0;
+ break;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ }
+
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x8000 | (addr & 0xff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), result);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x1a, 0x00, result);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr,
+ uint32_t data, uint32_t mask, uint32_t *result)
+{
+ int error;
+ uint32_t read_data;
+
+ error = fm10k_epl_serdes_dma_esb_read(sw, serdes, addr, &read_data);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_write(sw, serdes, addr,
+ (data & mask) | (read_data & ~mask));
+ if (error)
+ goto done;
+ if (result)
+ *result = (data & mask) | (read_data & ~mask);
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param)
+{
+ int error;
+ uint32_t result;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, int_num, param, &result);
+ if (error)
+ goto done;
+ if (result != int_num) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+static void
+fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane)
+{
+ if (sw) {
+ *eplno = serdes / FM10K_SW_EPL_LANES;
+ *lane = serdes % FM10K_SW_EPL_LANES;
+ }
+}
+
+static int
+fm10k_epl_serdes_configure_near_loopback(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring near loopback",
+ sw->epl_sbus->name, serdes);
+
+ /* assuming no lane polarity has been set */
+ /* set near loopback mode */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ data = (1 << 8) | (1 << 0);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x08, data);
+ if (error)
+ goto done;
+
+ /* disable tx output, leave tx/ex enabled */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /* set signal detect override to normal */
+ fm10k_epl_serdes_set_signal_detect(lane, FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring serdes for dfe tuning",
+ sw->epl_sbus->name, serdes);
+
+ /*
+ * Configure the serdes to run DFE tuning
+ */
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (0 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_HF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (1 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_LF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (2 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_DC & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (3 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_BW & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ /* allow early link up */
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26, 0x5b01, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning initial calibration (ical)
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe ical",
+ sw->epl_sbus->name, serdes);
+
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_A25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_A10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x01);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_ical_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status, converged;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ if ((dfe_status & 0x11) == 0x11) {
+ *status = FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS;
+ } else {
+ error = fm10k_epl_serdes_get_ical_result(sw,
+ serdes, &converged);
+ if (error)
+ goto done;
+ if (converged)
+ *status = FM10K_SW_SERDES_DFE_ICAL_CONVERGED;
+ else
+ *status = FM10K_SW_SERDES_DFE_ICAL_FAILED;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning periodic calibration (pcal), just one
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe pcal",
+ sw->epl_sbus->name, serdes);
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_B25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_B10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x02);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_pcal_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+ if (dfe_status & 0x02)
+ *status = FM10K_SW_SERDES_DFE_PCAL_COMPLETE;
+ else
+ *status = FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_stop_dfe_tuning(struct fm10k_ext_port_lane *lane,
+ unsigned int force_reset)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ unsigned int ical_stopped, pcal_stopped;
+ unsigned int i;
+ uint32_t dfe_status, status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ ical_stopped = 0;
+ pcal_stopped = 0;
+
+ if (!force_reset) {
+ if (dfe_status & 0x3) {
+ /* stop all DFE tuning */
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes,
+ 0x0a, 0);
+ if (error)
+ goto done;
+
+ for (i = 0;
+ i < FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES;
+ i++) {
+ error = fm10k_epl_serdes_dfe_ical_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ ical_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+
+ if (ical_stopped)
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES;
+ i++) {
+ error =
+ fm10k_epl_serdes_dfe_pcal_status
+ (lane, &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 0;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+
+ if (ical_stopped && pcal_stopped && (dfe_status & 0x40)) {
+ error = fm10k_epl_serdes_start_dfe_pcal(lane);
+ if (error)
+ goto done;
+ pcal_stopped = 0;
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES; i++) {
+ error = fm10k_epl_serdes_dfe_pcal_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+ }
+
+ if (!ical_stopped || !pcal_stopped) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x18, 0x07);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param & 0xffff);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param >> 16);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126, (0x0b << 8),
+ status);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: dfe status: "
+ "coarse_active=%u fine_active=%u adaptive=%u",
+ sw->epl_sbus->name, serdes, *status & (1 << 0),
+ *status & (1 << 1), *status & (1 << 6));
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged)
+{
+ uint32_t val1;
+ uint32_t val2;
+ uint32_t diff;
+ int error;
+
+ *converged = 0;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12), &val1);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12) | (1 << 8), &val2);
+ if (error)
+ goto done;
+
+ /* sign extension */
+ if (val1 & 0x8000)
+ val1 |= 0xffff0000;
+ if (val2 & 0x8000)
+ val2 |= 0xffff0000;
+ diff = abs(val2 - val1);
+
+ if (diff >= FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD)
+ *converged = 1;
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.h b/drivers/net/fm10k/switch/fm10k_serdes.h
new file mode 100644
index 0000000..f13ad3a
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SERDES_H_
+#define _FM10K_SW_SERDES_H_
+
+
+#define FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_ICAL_CONVERGED 1
+#define FM10K_SW_SERDES_DFE_ICAL_FAILED 2
+#define FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_PCAL_COMPLETE 1
+
+
+int fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw);
+int fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_ical_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_pcal_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_stop_dfe_tuning
+ (struct fm10k_ext_port_lane *lane, unsigned int force_reset);
+void fm10k_epl_serdes_set_signal_detect
+ (struct fm10k_ext_port_lane *lane, unsigned int override);
+
+#endif /* _FM10K_SW_SERDES_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sm.c b/drivers/net/fm10k/switch/fm10k_sm.c
new file mode 100644
index 0000000..26fe7dd
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.c
@@ -0,0 +1,190 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <rte_malloc.h>
+
+#include "../base/fm10k_osdep.h"
+#include "fm10k_debug.h"
+#include "fm10k_sm.h"
+
+static void *fm10k_sm_handle_event(void *ptr);
+static void *fm10k_sm_handle_timer(void *ptr);
+static void fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap);
+static uint16_t fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap);
+static void fm10k_sm_event_heap_push
+ (struct fm10k_sm_event_heap *heap, uint16_t id);
+
+
+struct fm10k_sm *
+fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx)
+{
+ struct fm10k_sm *sm;
+ struct fm10k_sm_timer *timer;
+ unsigned int i;
+
+ sm = (struct fm10k_sm *)rte_zmalloc("fm10k_sm",
+ sizeof(struct fm10k_sm), 0);
+ if (sm == NULL)
+ goto fail;
+
+ sm->is_running = true;
+ sm->num_events = num_events;
+ fm10k_sm_event_heap_init(&sm->events);
+ pthread_create(&sm->event_task, NULL, fm10k_sm_handle_event, sm);
+ sm->num_timers = num_timers;
+ if (sm->num_timers > 0) {
+ sm->timers = (struct fm10k_sm_timer *)rte_zmalloc
+ ("fm10k_sm_timers",
+ sizeof(struct fm10k_sm_timer) * sm->num_timers, 0);
+ if (sm->timers == NULL)
+ goto fail;
+ }
+
+ sm->event_handler = event_handler;
+ sm->timer_handler = timer_handler;
+ sm->ctx = ctx;
+
+ for (i = 0; i < sm->num_timers; i++) {
+ timer = &sm->timers[i];
+ sem_init(&timer->tq, 0, 0);
+ timer->sm = sm;
+ pthread_create(&timer->t, NULL, fm10k_sm_handle_timer, timer);
+ timer->timer_id = i;
+ }
+
+ return sm;
+
+fail:
+ if (sm)
+ fm10k_sm_detach(sm);
+
+ return NULL;
+}
+
+void
+fm10k_sm_detach(struct fm10k_sm *sm)
+{
+ int i;
+
+ sm->is_running = false;
+ pthread_join(sm->event_task, NULL);
+ for (i = 0; i < sm->num_timers; i++)
+ pthread_join(sm->timers[i].t, NULL);
+ rte_free(sm);
+}
+
+void
+fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id)
+{
+ fm10k_sm_event_heap_push(&sm->events, event_id);
+}
+
+void
+fm10k_sm_timer_start(struct fm10k_sm *sm, uint16_t timer_id,
+ unsigned int timeout_ms)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+
+ gettimeofday(&timer->start, 0);
+ timer->timeout_ms = timeout_ms;
+ timer->running = 1;
+ sem_post(&timer->tq);
+}
+
+/*
+ * It is possible that an expiration events from this timer will
+ * occur after this is called.
+ */
+void
+fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+ timer->running = 0;
+}
+
+static void *
+fm10k_sm_handle_event(void *ctx)
+{
+ struct fm10k_sm *sm = ctx;
+ uint16_t event_id;
+
+ while (sm->is_running) {
+ event_id = fm10k_sm_event_heap_pop(&sm->events);
+ sm->event_handler(sm, event_id);
+ }
+ return NULL;
+}
+
+static void *
+fm10k_sm_handle_timer(void *ctx)
+{
+ struct fm10k_sm_timer *timer = ctx;
+ struct fm10k_sm *sm = timer->sm;
+ struct timeval now;
+ struct timeval sleep;
+ uint64_t cost, timeout;
+
+ while (sm->is_running) {
+ sem_wait(&timer->tq);
+ if (timer->running) {
+ gettimeofday(&now, 0);
+ cost = (now.tv_sec - timer->start.tv_sec) * 1000000;
+ cost += (now.tv_usec - timer->start.tv_usec);
+ if (cost < timer->timeout_ms * 1000) {
+ timeout = timer->timeout_ms * 1000;
+ timeout -= cost;
+ sleep.tv_sec = timeout / 1000000;
+ sleep.tv_usec = timeout % 1000000;
+ select(0, NULL, NULL, NULL, &sleep);
+ }
+ if (timer->running)
+ sm->timer_handler(sm, timer->timer_id);
+ }
+ }
+
+ return NULL;
+}
+
+
+static void
+fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap)
+{
+ heap->count = 0;
+ pthread_mutex_init(&heap->lock, NULL);
+ sem_init(&heap->s, 0, 0);
+}
+
+static uint16_t
+fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap)
+{
+ int i;
+ uint16_t id;
+
+ sem_wait(&heap->s);
+ pthread_mutex_lock(&heap->lock);
+ id = heap->heap[0];
+ for (i = 1; i < heap->count; i++)
+ heap->heap[i - 1] = heap->heap[i];
+ heap->count--;
+ pthread_mutex_unlock(&heap->lock);
+ return id;
+}
+
+static void
+fm10k_sm_event_heap_push(struct fm10k_sm_event_heap *heap, uint16_t id)
+{
+ pthread_mutex_lock(&heap->lock);
+ if (heap->count >= FM10K_SM_ID_HEAP_MAX) {
+ pthread_mutex_unlock(&heap->lock);
+ return;
+ }
+ heap->heap[heap->count] = id;
+ heap->count++;
+ pthread_mutex_unlock(&heap->lock);
+ sem_post(&heap->s);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sm.h b/drivers/net/fm10k/switch/fm10k_sm.h
new file mode 100644
index 0000000..8daadfa
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.h
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SM_H_
+#define _FM10K_SW_SM_H_
+
+#include <stdint.h>
+
+#include <signal.h>
+#include <sys/time.h>
+#include <pthread.h>
+#include <semaphore.h>
+#include <stdbool.h>
+
+
+/*
+ * This framework provides state machine implementation support for use in
+ * the rest of the driver.
+ *
+ * The model is that for a given state machine, there are N>0 events
+ * identified by the integers [0..N-1] and there are M>=0 timers identified
+ * by the integers [0..M-1].
+ *
+ * The timers are one-shot timers.
+ *
+ * Since all state machine processing occurs in the context of the
+ * task queue, if that task queue was created with only a single thread, then
+ * no locking is required in the provided event and timer handlers to
+ * prevent overlapping execution.
+ *
+ */
+
+struct fm10k_sm;
+
+struct fm10k_sm_timer {
+ struct fm10k_sm *sm;
+ pthread_t t;
+ sem_t tq;
+ struct timeval start;
+ uint16_t timeout_ms;
+ uint16_t timer_id;
+ uint8_t running;
+};
+
+typedef void (*fm10k_sm_process_event_t)(struct fm10k_sm *sm, uint16_t event);
+typedef void (*fm10k_sm_process_timer_t)(struct fm10k_sm *sm, uint16_t timer);
+
+#define FM10K_SM_ID_HEAP_MAX 128
+struct fm10k_sm_event_heap {
+ pthread_mutex_t lock;
+ sem_t s;
+ uint16_t count;
+ uint16_t heap[FM10K_SM_ID_HEAP_MAX];
+};
+
+struct fm10k_sm {
+ bool is_running;
+ pthread_t event_task;
+ struct fm10k_sm_timer *timers;
+ struct fm10k_sm_event_heap events;
+ fm10k_sm_process_event_t event_handler;
+ fm10k_sm_process_timer_t timer_handler;
+ void *ctx;
+ unsigned int state;
+ uint16_t num_events;
+ uint16_t num_timers;
+ uint8_t portno;
+ uint8_t laneno;
+};
+
+struct fm10k_sm *fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx);
+void fm10k_sm_detach(struct fm10k_sm *sm);
+void fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id);
+void fm10k_sm_timer_start(struct fm10k_sm *sm,
+ uint16_t timer_id, unsigned int timeout_ms);
+void fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id);
+
+#endif /* _FM10K_SW_SM_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.c b/drivers/net/fm10k/switch/fm10k_spico_code.c
new file mode 100644
index 0000000..cfab632
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.c
@@ -0,0 +1,2966 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include "fm10k_spico_code.h"
+
+
+/* Production versions */
+
+
+/* SBus master code (production version)
+ * source file: release_2015_01_14/sbus_master.0x1013_0001.rom
+ */
+
+const uint32_t fm10000_sbus_master_code_version_build_id_prd = 0x10130001;
+const uint32_t fm10000_sbus_master_code_size_prd = 3338;
+
+const uint16_t fm10000_sbus_master_code_prd[] = {
+ 0x132, 0x002, 0x006, 0x000, 0x390, 0x002, 0x080, 0x247,
+ 0x071, 0x247, 0x01f, 0x247, 0x020, 0x247, 0x02e, 0x247,
+ 0x02f, 0x247, 0x03f, 0x247, 0x040, 0x247, 0x042, 0x247,
+ 0x043, 0x247, 0x03d, 0x0c0, 0x02d, 0x247, 0x031, 0x247,
+ 0x032, 0x081, 0x18b, 0x247, 0x039, 0x248, 0x03a, 0x181,
+ 0x247, 0x034, 0x248, 0x036, 0x209, 0x247, 0x033, 0x248,
+ 0x035, 0x305, 0x034, 0x247, 0x037, 0x248, 0x038, 0x190,
+ 0x247, 0x03b, 0x248, 0x03c, 0x3d4, 0x1ff, 0x00c, 0x3d5,
+ 0x100, 0x00c, 0x3d4, 0x2ff, 0x00c, 0x3c7, 0x0fe, 0x018,
+ 0x0c2, 0x019, 0x044, 0x233, 0x000, 0x152, 0x01b, 0x347,
+ 0x01b, 0x030, 0x085, 0x187, 0x247, 0x03e, 0x3c7, 0x100,
+ 0x041, 0x080, 0x247, 0x065, 0x3c4, 0x3fc, 0x010, 0x3d4,
+ 0x22f, 0x00e, 0x347, 0x033, 0x008, 0x247, 0x008, 0x081,
+ 0x189, 0x227, 0x2c7, 0x044, 0x220, 0x2c7, 0x045, 0x220,
+ 0x2c7, 0x046, 0x220, 0x2c7, 0x047, 0x220, 0x2c7, 0x048,
+ 0x220, 0x2c7, 0x049, 0x220, 0x2c7, 0x04a, 0x220, 0x2c7,
+ 0x04b, 0x3a7, 0x05b, 0x1a7, 0x3a5, 0x08d, 0x2c7, 0x04c,
+ 0x0c0, 0x04d, 0x044, 0x07d, 0x001, 0x0df, 0x02c, 0x000,
+ 0x009, 0x322, 0x02c, 0x009, 0x005, 0x347, 0x02d, 0x02c,
+ 0x3d4, 0x3df, 0x00e, 0x307, 0x02c, 0x384, 0x01f, 0x229,
+ 0x220, 0x3b0, 0x0aa, 0x287, 0x199, 0x390, 0x000, 0x041,
+ 0x040, 0x045, 0x020, 0x12c, 0x000, 0x020, 0x163, 0x000,
+ 0x020, 0x175, 0x000, 0x020, 0x27e, 0x000, 0x020, 0x362,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x1de, 0x000, 0x020, 0x330, 0x001, 0x020, 0x270, 0x001,
+ 0x020, 0x2af, 0x001, 0x020, 0x2a1, 0x001, 0x020, 0x338,
+ 0x001, 0x020, 0x285, 0x001, 0x020, 0x2b9, 0x001, 0x020,
+ 0x2a8, 0x001, 0x020, 0x001, 0x002, 0x020, 0x105, 0x001,
+ 0x020, 0x19e, 0x001, 0x020, 0x3e6, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000,
+ 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x044, 0x140, 0x000, 0x054, 0x362, 0x03d,
+ 0x000, 0x001, 0x0b0, 0x364, 0x03d, 0x001, 0x001, 0x0c6,
+ 0x307, 0x02e, 0x044, 0x187, 0x000, 0x055, 0x141, 0x02e,
+ 0x307, 0x030, 0x302, 0x02e, 0x00b, 0x374, 0x0c0, 0x02e,
+ 0x327, 0x02d, 0x000, 0x367, 0x044, 0x140, 0x000, 0x3d4,
+ 0x3df, 0x00e, 0x207, 0x009, 0x00a, 0x04f, 0x04e, 0x055,
+ 0x3d5, 0x200, 0x014, 0x000, 0x35d, 0x054, 0x000, 0x3f8,
+ 0x055, 0x081, 0x364, 0x010, 0x001, 0x001, 0x00d, 0x364,
+ 0x011, 0x002, 0x009, 0x009, 0x3c4, 0x3fe, 0x010, 0x3d4,
+ 0x3bf, 0x00e, 0x080, 0x101, 0x364, 0x010, 0x002, 0x001,
+ 0x00a, 0x364, 0x011, 0x001, 0x001, 0x006, 0x3c4, 0x3fd,
+ 0x010, 0x111, 0x045, 0x347, 0x026, 0x018, 0x347, 0x027,
+ 0x019, 0x347, 0x028, 0x01b, 0x347, 0x029, 0x01c, 0x044,
+ 0x222, 0x000, 0x0a1, 0x000, 0x31e, 0x347, 0x026, 0x018,
+ 0x347, 0x027, 0x019, 0x044, 0x233, 0x000, 0x347, 0x01b,
+ 0x028, 0x347, 0x01c, 0x029, 0x0a2, 0x000, 0x30c, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x000, 0x001, 0x016, 0x362,
+ 0x01b, 0x001, 0x001, 0x013, 0x362, 0x01b, 0x002, 0x001,
+ 0x00d, 0x362, 0x01b, 0x003, 0x001, 0x013, 0x362, 0x01b,
+ 0x004, 0x001, 0x013, 0x04e, 0x045, 0x364, 0x03d, 0x002,
+ 0x001, 0x005, 0x044, 0x01a, 0x001, 0x000, 0x3f7, 0x348,
+ 0x01b, 0x01a, 0x000, 0x3f2, 0x348, 0x01b, 0x01a, 0x000,
+ 0x3ed, 0x101, 0x247, 0x018, 0x3d7, 0x0ff, 0x019, 0x000,
+ 0x074, 0x3c7, 0x100, 0x014, 0x364, 0x004, 0x100, 0x009,
+ 0x00f, 0x044, 0x08d, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x055, 0x0c1,
+ 0x03d, 0x0bf, 0x000, 0x2b7, 0x000, 0x2bc, 0x3c7, 0x3ff,
+ 0x028, 0x087, 0x0b6, 0x0c1, 0x04e, 0x044, 0x37c, 0x000,
+ 0x347, 0x067, 0x029, 0x2c7, 0x02a, 0x322, 0x04e, 0x009,
+ 0x00c, 0x3c2, 0x016, 0x067, 0x009, 0x00c, 0x0c1, 0x028,
+ 0x0a7, 0x000, 0x298, 0x347, 0x033, 0x028, 0x000, 0x3fa,
+ 0x347, 0x034, 0x028, 0x000, 0x3f5, 0x111, 0x009, 0x3ff,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x040,
+ 0x103, 0x051, 0x042, 0x2a7, 0x001, 0x00c, 0x040, 0x280,
+ 0x051, 0x1a7, 0x101, 0x050, 0x225, 0x042, 0x280, 0x045,
+ 0x287, 0x045, 0x364, 0x005, 0x008, 0x009, 0x3fd, 0x347,
+ 0x01b, 0x00a, 0x347, 0x01c, 0x00b, 0x3c7, 0x041, 0x00c,
+ 0x020, 0x260, 0x000, 0x364, 0x005, 0x008, 0x009, 0x3fd,
+ 0x3d5, 0x100, 0x00c, 0x3c7, 0x042, 0x00c, 0x044, 0x260,
+ 0x000, 0x364, 0x005, 0x010, 0x001, 0x3fd, 0x307, 0x005,
+ 0x384, 0x007, 0x262, 0x004, 0x009, 0x009, 0x347, 0x002,
+ 0x01b, 0x347, 0x003, 0x01c, 0x045, 0x262, 0x006, 0x009,
+ 0x3ef, 0x141, 0x020, 0x0c0, 0x01b, 0x0c0, 0x01c, 0x045,
+ 0x347, 0x019, 0x00d, 0x1c7, 0x00d, 0x345, 0x018, 0x00d,
+ 0x3d5, 0x200, 0x00c, 0x364, 0x005, 0x008, 0x001, 0x3fd,
+ 0x3d4, 0x1ff, 0x00c, 0x045, 0x364, 0x005, 0x008, 0x009,
+ 0x3fd, 0x0c0, 0x00c, 0x020, 0x260, 0x000, 0x054, 0x347,
+ 0x032, 0x031, 0x307, 0x031, 0x044, 0x2ad, 0x000, 0x141,
+ 0x031, 0x307, 0x030, 0x302, 0x031, 0x00b, 0x004, 0x0c0,
+ 0x031, 0x2a7, 0x001, 0x00a, 0x347, 0x031, 0x032, 0x347,
+ 0x031, 0x070, 0x000, 0x00b, 0x307, 0x032, 0x302, 0x031,
+ 0x009, 0x3e2, 0x3c7, 0x3ff, 0x070, 0x055, 0x3d4, 0x2ff,
+ 0x00e, 0x0a3, 0x020, 0x091, 0x000, 0x044, 0x1b9, 0x000,
+ 0x362, 0x01b, 0x001, 0x001, 0x009, 0x362, 0x01b, 0x016,
+ 0x001, 0x004, 0x000, 0x00b, 0x0cf, 0x019, 0x044, 0x233,
+ 0x000, 0x307, 0x01b, 0x003, 0x004, 0x0a0, 0x045, 0x304,
+ 0x035, 0x227, 0x1bc, 0x2e2, 0x003, 0x001, 0x093, 0x2e2,
+ 0x002, 0x001, 0x046, 0x2e2, 0x001, 0x001, 0x063, 0x227,
+ 0x001, 0x03d, 0x2e4, 0x001, 0x009, 0x010, 0x2e4, 0x002,
+ 0x009, 0x019, 0x2e4, 0x004, 0x009, 0x029, 0x2e4, 0x008,
+ 0x009, 0x029, 0x000, 0x02b, 0x041, 0x0a2, 0x1a7, 0x3b5,
+ 0x03a, 0x044, 0x1b1, 0x001, 0x043, 0x3a4, 0x3fe, 0x000,
+ 0x3e1, 0x041, 0x327, 0x018, 0x041, 0x044, 0x08d, 0x001,
+ 0x247, 0x01e, 0x043, 0x2c7, 0x018, 0x044, 0x0fb, 0x001,
+ 0x043, 0x3a4, 0x3fd, 0x000, 0x3cd, 0x3a4, 0x3fb, 0x000,
+ 0x3c9, 0x3a4, 0x3f7, 0x000, 0x3c5, 0x0a1, 0x045, 0x227,
+ 0x3a4, 0x0ff, 0x197, 0x384, 0x01f, 0x001, 0x010, 0x262,
+ 0x001, 0x001, 0x00e, 0x262, 0x002, 0x001, 0x00f, 0x262,
+ 0x003, 0x001, 0x00d, 0x000, 0x3ea, 0x000, 0x3e8, 0x044,
+ 0x079, 0x002, 0x000, 0x3e3, 0x000, 0x3e1, 0x000, 0x3df,
+ 0x0bf, 0x1aa, 0x2a8, 0x224, 0x19a, 0x384, 0x003, 0x001,
+ 0x00c, 0x262, 0x001, 0x001, 0x017, 0x262, 0x002, 0x001,
+ 0x015, 0x000, 0x015, 0x0c5, 0x019, 0x247, 0x01b, 0x0a2,
+ 0x1a7, 0x3b5, 0x03a, 0x2c7, 0x01c, 0x044, 0x222, 0x000,
+ 0x000, 0x3bd, 0x000, 0x3bb, 0x000, 0x3b9, 0x000, 0x3b7,
+ 0x000, 0x3b5, 0x054, 0x307, 0x066, 0x191, 0x384, 0x07f,
+ 0x3c7, 0x0fe, 0x018, 0x0c2, 0x019, 0x044, 0x233, 0x000,
+ 0x347, 0x01b, 0x064, 0x0a7, 0x044, 0x0bf, 0x003, 0x055,
+ 0x0a4, 0x020, 0x091, 0x000, 0x186, 0x285, 0x181, 0x0a7,
+ 0x1ab, 0x324, 0x004, 0x285, 0x055, 0x364, 0x010, 0x002,
+ 0x009, 0x026, 0x364, 0x011, 0x001, 0x009, 0x029, 0x247,
+ 0x064, 0x044, 0x0cd, 0x003, 0x3c4, 0x2ff, 0x065, 0x04f,
+ 0x04e, 0x055, 0x364, 0x065, 0x100, 0x001, 0x3fa, 0x327,
+ 0x065, 0x3a4, 0x007, 0x364, 0x065, 0x040, 0x001, 0x007,
+ 0x287, 0x302, 0x04e, 0x009, 0x3ec, 0x045, 0x364, 0x011,
+ 0x001, 0x001, 0x005, 0x3c4, 0x3fd, 0x010, 0x054, 0x04e,
+ 0x000, 0x3cc, 0x309, 0x023, 0x003, 0x024, 0x347, 0x023,
+ 0x04f, 0x00b, 0x01f, 0x3d5, 0x010, 0x00e, 0x08f, 0x187,
+ 0x395, 0x0a0, 0x247, 0x017, 0x080, 0x247, 0x016, 0x0c0,
+ 0x015, 0x3c5, 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4,
+ 0x3fe, 0x014, 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e,
+ 0x347, 0x04f, 0x009, 0x020, 0x20b, 0x002, 0x307, 0x04f,
+ 0x00b, 0x028, 0x3c5, 0x002, 0x03d, 0x3d5, 0x010, 0x00e,
+ 0x387, 0x0b2, 0x187, 0x385, 0x080, 0x247, 0x017, 0x08e,
+ 0x187, 0x385, 0x0e6, 0x247, 0x016, 0x0c0, 0x015, 0x3c5,
+ 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4, 0x3fe, 0x014,
+ 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e, 0x000, 0x006,
+ 0x082, 0x208, 0x244, 0x03d, 0x0df, 0x02c, 0x0b3, 0x020,
+ 0x091, 0x000, 0x307, 0x018, 0x3a7, 0x051, 0x044, 0x061,
+ 0x001, 0x2c7, 0x061, 0x04c, 0x324, 0x061, 0x001, 0x020,
+ 0x040, 0x3b7, 0x126, 0x08b, 0x187, 0x044, 0x1c4, 0x001,
+ 0x009, 0x005, 0x3c7, 0x022, 0x01b, 0x042, 0x364, 0x01b,
+ 0x022, 0x009, 0x027, 0x04c, 0x348, 0x061, 0x061, 0x324,
+ 0x061, 0x04d, 0x151, 0x050, 0x000, 0x01c, 0x327, 0x04f,
+ 0x3a4, 0x0ff, 0x322, 0x050, 0x00f, 0x014, 0x040, 0x0aa,
+ 0x084, 0x044, 0x1c4, 0x001, 0x042, 0x362, 0x01b, 0x00a,
+ 0x009, 0x008, 0x04c, 0x325, 0x061, 0x04d, 0x141, 0x050,
+ 0x045, 0x384, 0x0ff, 0x040, 0x193, 0x220, 0x042, 0x384,
+ 0x00f, 0x041, 0x0a1, 0x111, 0x003, 0x005, 0x2a9, 0x000,
+ 0x3fc, 0x042, 0x045, 0x0a0, 0x0d0, 0x061, 0x04d, 0x101,
+ 0x151, 0x061, 0x009, 0x3fc, 0x045, 0x0c0, 0x04f, 0x0c0,
+ 0x050, 0x387, 0x051, 0x044, 0x073, 0x001, 0x044, 0x209,
+ 0x000, 0x051, 0x101, 0x051, 0x045, 0x307, 0x03f, 0x001,
+ 0x006, 0x247, 0x018, 0x000, 0x01f, 0x0a0, 0x287, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x012, 0x001, 0x012, 0x362,
+ 0x01b, 0x011, 0x009, 0x005, 0x347, 0x018, 0x040, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3eb, 0x0a0, 0x045, 0x347,
+ 0x018, 0x03f, 0x0c0, 0x019, 0x3c7, 0x070, 0x01b, 0x3c5,
+ 0x001, 0x01b, 0x044, 0x222, 0x000, 0x3d4, 0x3fe, 0x01b,
+ 0x044, 0x222, 0x000, 0x307, 0x03e, 0x044, 0x205, 0x000,
+ 0x3c5, 0x002, 0x01b, 0x044, 0x222, 0x000, 0x307, 0x03e,
+ 0x194, 0x044, 0x205, 0x000, 0x0c1, 0x019, 0x044, 0x233,
+ 0x000, 0x327, 0x01b, 0x0c0, 0x019, 0x3c5, 0x001, 0x01b,
+ 0x044, 0x222, 0x000, 0x3b5, 0x200, 0x045, 0x2c7, 0x01e,
+ 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001, 0x007, 0x041,
+ 0x044, 0x0fb, 0x001, 0x043, 0x121, 0x307, 0x030, 0x282,
+ 0x00b, 0x3f1, 0x045, 0x3b7, 0x03b, 0x307, 0x01e, 0x000,
+ 0x0b2, 0x091, 0x020, 0x362, 0x002, 0x054, 0x327, 0x06f,
+ 0x009, 0x009, 0x044, 0x08d, 0x001, 0x001, 0x080, 0x000,
+ 0x00d, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x012, 0x009,
+ 0x076, 0x044, 0x0b2, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x287, 0x001, 0x06b, 0x2c7, 0x01d, 0x327, 0x040,
+ 0x3a4, 0x0ff, 0x2c7, 0x018, 0x0c9, 0x01a, 0x044, 0x2d7,
+ 0x001, 0x044, 0x206, 0x001, 0x324, 0x035, 0x0c0, 0x01a,
+ 0x087, 0x187, 0x385, 0x06c, 0x282, 0x00b, 0x015, 0x0c1,
+ 0x01a, 0x087, 0x187, 0x385, 0x0bc, 0x282, 0x00b, 0x00c,
+ 0x0c2, 0x01a, 0x088, 0x187, 0x08c, 0x282, 0x00b, 0x004,
+ 0x0c3, 0x01a, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x010, 0x041, 0x307, 0x01a, 0x18b, 0x305, 0x042, 0x385,
+ 0x0e0, 0x3b7, 0x33b, 0x044, 0x1b1, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3e8, 0x307, 0x03e, 0x044,
+ 0x205, 0x000, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x00b, 0x041, 0x307, 0x01d, 0x3b7, 0x03b, 0x044, 0x1b1,
+ 0x001, 0x043, 0x121, 0x307, 0x030, 0x282, 0x00b, 0x3ed,
+ 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387, 0x3ff, 0x3c7,
+ 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296, 0x001, 0x347,
+ 0x023, 0x043, 0x092, 0x020, 0x365, 0x002, 0x054, 0x044,
+ 0x1de, 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296,
+ 0x001, 0x040, 0x044, 0x1c9, 0x001, 0x001, 0x00c, 0x042,
+ 0x0c5, 0x019, 0x2c7, 0x01c, 0x247, 0x01b, 0x020, 0x222,
+ 0x000, 0x042, 0x080, 0x045, 0x044, 0x1b1, 0x001, 0x001,
+ 0x3fb, 0x041, 0x3a7, 0x0ff, 0x0c6, 0x019, 0x131, 0x001,
+ 0x00c, 0x044, 0x233, 0x000, 0x30a, 0x01c, 0x002, 0x3f8,
+ 0x081, 0x000, 0x003, 0x080, 0x043, 0x045, 0x307, 0x040,
+ 0x384, 0x0ff, 0x009, 0x017, 0x0a0, 0x287, 0x044, 0x1b9,
+ 0x000, 0x362, 0x01b, 0x011, 0x001, 0x00a, 0x121, 0x307,
+ 0x030, 0x282, 0x00b, 0x3f3, 0x0a0, 0x045, 0x347, 0x018,
+ 0x040, 0x0c0, 0x01a, 0x327, 0x040, 0x044, 0x2c3, 0x001,
+ 0x324, 0x034, 0x009, 0x3f2, 0x227, 0x045, 0x247, 0x01c,
+ 0x2c7, 0x01b, 0x280, 0x001, 0x3e9, 0x0c3, 0x019, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222,
+ 0x000, 0x0c2, 0x01b, 0x044, 0x222, 0x000, 0x347, 0x01e,
+ 0x019, 0x3d0, 0x041, 0x019, 0x307, 0x041, 0x044, 0x205,
+ 0x000, 0x044, 0x233, 0x000, 0x327, 0x01b, 0x00b, 0x3f6,
+ 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x287,
+ 0x045, 0x2c7, 0x01e, 0x0a0, 0x287, 0x044, 0x300, 0x001,
+ 0x001, 0x007, 0x041, 0x044, 0x24e, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3f1, 0x045, 0x09f, 0x18a,
+ 0x0a1, 0x1aa, 0x324, 0x01e, 0x001, 0x006, 0x305, 0x01e,
+ 0x000, 0x005, 0x208, 0x304, 0x01e, 0x300, 0x043, 0x00b,
+ 0x005, 0x080, 0x000, 0x008, 0x272, 0x3e8, 0x00b, 0x004,
+ 0x397, 0x3e8, 0x305, 0x033, 0x3b7, 0x13b, 0x000, 0x343,
+ 0x327, 0x06e, 0x0c0, 0x01a, 0x044, 0x2c3, 0x001, 0x0a9,
+ 0x305, 0x033, 0x247, 0x007, 0x344, 0x035, 0x007, 0x3d4,
+ 0x37f, 0x00e, 0x020, 0x091, 0x000, 0x327, 0x06f, 0x0c0,
+ 0x01a, 0x044, 0x2c3, 0x001, 0x324, 0x034, 0x009, 0x005,
+ 0x347, 0x06f, 0x040, 0x0ad, 0x0c1, 0x008, 0x247, 0x009,
+ 0x345, 0x033, 0x008, 0x344, 0x035, 0x008, 0x020, 0x091,
+ 0x000, 0x327, 0x06e, 0x054, 0x055, 0x0ab, 0x000, 0x3d2,
+ 0x327, 0x06f, 0x054, 0x055, 0x0af, 0x000, 0x3e7, 0x327,
+ 0x06e, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001, 0x0aa, 0x000,
+ 0x3c1, 0x327, 0x06f, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001,
+ 0x0ae, 0x000, 0x3d3, 0x044, 0x2f8, 0x001, 0x362, 0x01b,
+ 0x011, 0x009, 0x00a, 0x044, 0x2d7, 0x001, 0x044, 0x206,
+ 0x001, 0x055, 0x045, 0x307, 0x034, 0x000, 0x3fc, 0x080,
+ 0x1bb, 0x2e2, 0x008, 0x00b, 0x00d, 0x0a1, 0x0c0, 0x01e,
+ 0x340, 0x01a, 0x01e, 0x001, 0x014, 0x1a8, 0x000, 0x011,
+ 0x320, 0x01a, 0x2c7, 0x01e, 0x2c7, 0x01a, 0x0a1, 0x151,
+ 0x01a, 0x003, 0x006, 0x2a9, 0x20b, 0x000, 0x3fa, 0x045,
+ 0x054, 0x287, 0x384, 0x0ff, 0x111, 0x020, 0x1b9, 0x000,
+ 0x044, 0x1b9, 0x000, 0x362, 0x01b, 0x016, 0x001, 0x007,
+ 0x362, 0x01b, 0x001, 0x009, 0x023, 0x3d7, 0x0fe, 0x019,
+ 0x044, 0x233, 0x000, 0x387, 0x0bb, 0x187, 0x385, 0x031,
+ 0x302, 0x01b, 0x001, 0x014, 0x387, 0x0b9, 0x187, 0x385,
+ 0x031, 0x302, 0x01b, 0x001, 0x00b, 0x387, 0x0b6, 0x187,
+ 0x385, 0x031, 0x302, 0x01b, 0x009, 0x003, 0x080, 0x045,
+ 0x327, 0x06e, 0x044, 0x340, 0x001, 0x0a8, 0x000, 0x342,
+ 0x327, 0x06f, 0x044, 0x340, 0x001, 0x0ac, 0x000, 0x356,
+ 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x00b, 0x009, 0x38d,
+ 0x0c0, 0x01c, 0x041, 0x1bb, 0x3a4, 0x007, 0x001, 0x047,
+ 0x131, 0x001, 0x02f, 0x131, 0x001, 0x017, 0x083, 0x3a7,
+ 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0, 0x044, 0x3f6,
+ 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x081, 0x044, 0x3f6,
+ 0x001, 0x000, 0x03e, 0x082, 0x3a7, 0x3ff, 0x044, 0x3f6,
+ 0x001, 0x084, 0x0a0, 0x044, 0x3f6, 0x001, 0x083, 0x044,
+ 0x3f6, 0x001, 0x081, 0x044, 0x3f6, 0x001, 0x000, 0x029,
+ 0x081, 0x3a7, 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0,
+ 0x044, 0x3f6, 0x001, 0x083, 0x044, 0x3f6, 0x001, 0x082,
+ 0x044, 0x3f6, 0x001, 0x000, 0x014, 0x081, 0x3a7, 0x3ff,
+ 0x044, 0x3f6, 0x001, 0x084, 0x044, 0x3f6, 0x001, 0x083,
+ 0x044, 0x3f6, 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x0c6,
+ 0x019, 0x0c0, 0x01b, 0x044, 0x222, 0x000, 0x0c5, 0x019,
+ 0x387, 0x0ff, 0x3a7, 0x3ff, 0x322, 0x04c, 0x303, 0x04d,
+ 0x2c7, 0x01b, 0x247, 0x01c, 0x044, 0x222, 0x000, 0x0c0,
+ 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x397, 0x3ff,
+ 0x044, 0x205, 0x000, 0x044, 0x233, 0x000, 0x3c4, 0x001,
+ 0x01b, 0x009, 0x3f5, 0x0c6, 0x019, 0x044, 0x233, 0x000,
+ 0x307, 0x01b, 0x043, 0x2a7, 0x003, 0x016, 0x0c0, 0x01a,
+ 0x397, 0x044, 0x04c, 0x322, 0x01b, 0x00b, 0x00b, 0x272,
+ 0x04b, 0x001, 0x007, 0x101, 0x141, 0x01a, 0x000, 0x3f4,
+ 0x307, 0x01a, 0x305, 0x033, 0x055, 0x045, 0x247, 0x019,
+ 0x2c7, 0x01b, 0x020, 0x222, 0x000, 0x090, 0x020, 0x362,
+ 0x002, 0x327, 0x06f, 0x287, 0x384, 0x0ff, 0x262, 0x0df,
+ 0x00b, 0x00a, 0x262, 0x0e7, 0x003, 0x006, 0x247, 0x018,
+ 0x000, 0x00a, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x001,
+ 0x009, 0x057, 0x0c7, 0x019, 0x0d1, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0d0, 0x01b, 0x044, 0x222, 0x000,
+ 0x0c0, 0x019, 0x0c0, 0x01b, 0x347, 0x034, 0x01c, 0x044,
+ 0x222, 0x000, 0x0ca, 0x019, 0x081, 0x189, 0x247, 0x012,
+ 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x133, 0x347, 0x013,
+ 0x01b, 0x347, 0x013, 0x01c, 0x044, 0x222, 0x000, 0x132,
+ 0x007, 0x3f6, 0x0c0, 0x019, 0x0c0, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0cb, 0x019, 0x0cc, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c7, 0x019, 0x0c2, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0c8, 0x019, 0x0c0, 0x01b, 0x044,
+ 0x222, 0x000, 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b0, 0x020, 0x296,
+ 0x001, 0x0c1, 0x019, 0x0c0, 0x01b, 0x0c1, 0x01c, 0x1cc,
+ 0x01c, 0x044, 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x054,
+ 0x01c, 0x044, 0x222, 0x000, 0x345, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x01d, 0x102, 0x050, 0x282, 0x00f,
+ 0x08f, 0x307, 0x01d, 0x104, 0x1a0, 0x280, 0x051, 0x1a7,
+ 0x101, 0x050, 0x285, 0x300, 0x01d, 0x051, 0x1a7, 0x2c7,
+ 0x01d, 0x101, 0x051, 0x2c5, 0x01d, 0x101, 0x051, 0x1a7,
+ 0x2c7, 0x01e, 0x101, 0x051, 0x2c5, 0x01e, 0x347, 0x034,
+ 0x01c, 0x227, 0x044, 0x222, 0x000, 0x287, 0x044, 0x11b,
+ 0x002, 0x044, 0x11b, 0x002, 0x347, 0x01d, 0x01b, 0x347,
+ 0x034, 0x01c, 0x044, 0x222, 0x000, 0x327, 0x01e, 0x001,
+ 0x057, 0x0ca, 0x019, 0x101, 0x052, 0x01b, 0x101, 0x051,
+ 0x041, 0x1a9, 0x2c5, 0x01b, 0x043, 0x1b5, 0x2c7, 0x01c,
+ 0x101, 0x051, 0x1a3, 0x325, 0x037, 0x2c5, 0x01c, 0x044,
+ 0x222, 0x000, 0x153, 0x01e, 0x007, 0x3e7, 0x0c0, 0x019,
+ 0x0c0, 0x01c, 0x044, 0x222, 0x000, 0x0ca, 0x01b, 0x0c5,
+ 0x019, 0x090, 0x187, 0x395, 0x03a, 0x247, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x020, 0x01b, 0x0c0,
+ 0x01c, 0x044, 0x222, 0x000, 0x347, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x045, 0x101, 0x051, 0x1a7, 0x2c7, 0x01b,
+ 0x101, 0x051, 0x2c5, 0x01b, 0x101, 0x051, 0x325, 0x037,
+ 0x2c7, 0x01c, 0x044, 0x222, 0x000, 0x045, 0x0c5, 0x01b,
+ 0x000, 0x3cf, 0x347, 0x000, 0x022, 0x347, 0x001, 0x023,
+ 0x0c0, 0x008, 0x0c0, 0x009, 0x307, 0x022, 0x384, 0x03f,
+ 0x229, 0x220, 0x3b0, 0x14b, 0x287, 0x199, 0x390, 0x002,
+ 0x041, 0x040, 0x045, 0x020, 0x21a, 0x002, 0x020, 0x223,
+ 0x002, 0x020, 0x22c, 0x002, 0x020, 0x351, 0x002, 0x020,
+ 0x281, 0x002, 0x020, 0x304, 0x002, 0x020, 0x309, 0x002,
+ 0x020, 0x30e, 0x002, 0x020, 0x313, 0x002, 0x020, 0x318,
+ 0x002, 0x020, 0x31d, 0x002, 0x020, 0x322, 0x002, 0x020,
+ 0x327, 0x002, 0x020, 0x32c, 0x002, 0x020, 0x331, 0x002,
+ 0x020, 0x336, 0x002, 0x020, 0x339, 0x002, 0x020, 0x33f,
+ 0x002, 0x020, 0x344, 0x002, 0x020, 0x348, 0x002, 0x020,
+ 0x354, 0x002, 0x020, 0x358, 0x002, 0x020, 0x361, 0x002,
+ 0x020, 0x36a, 0x002, 0x020, 0x36d, 0x002, 0x020, 0x370,
+ 0x002, 0x020, 0x2f1, 0x002, 0x020, 0x373, 0x002, 0x020,
+ 0x384, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x38e, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x3fd, 0x001, 0x020, 0x101,
+ 0x001, 0x020, 0x197, 0x001, 0x020, 0x3ba, 0x000, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x0c1, 0x008, 0x345, 0x033, 0x008,
+ 0x344, 0x035, 0x008, 0x04e, 0x047, 0x3c7, 0x3ff, 0x008,
+ 0x000, 0x3f8, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7,
+ 0x009, 0x000, 0x3ea, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001,
+ 0x2c7, 0x009, 0x000, 0x3e1, 0x080, 0x3b7, 0x003, 0x1a9,
+ 0x3b5, 0x105, 0x2c7, 0x023, 0x0c0, 0x028, 0x0c0, 0x029,
+ 0x052, 0x022, 0x044, 0x2d6, 0x002, 0x001, 0x005, 0x101,
+ 0x000, 0x3f8, 0x101, 0x052, 0x022, 0x346, 0x028, 0x022,
+ 0x3c4, 0x0ff, 0x022, 0x362, 0x022, 0x0ef, 0x009, 0x02e,
+ 0x101, 0x052, 0x022, 0x327, 0x028, 0x1b7, 0x2c6, 0x022,
+ 0x362, 0x022, 0x0be, 0x009, 0x021, 0x101, 0x052, 0x022,
+ 0x346, 0x029, 0x022, 0x3c4, 0x0ff, 0x022, 0x362, 0x022,
+ 0x0ad, 0x009, 0x013, 0x101, 0x052, 0x022, 0x327, 0x029,
+ 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0de, 0x009, 0x006,
+ 0x0c1, 0x009, 0x000, 0x391, 0x3c7, 0x3ff, 0x009, 0x000,
+ 0x38c, 0x081, 0x189, 0x247, 0x012, 0x0c0, 0x028, 0x0c0,
+ 0x029, 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x2c7, 0x023,
+ 0x247, 0x012, 0x080, 0x347, 0x013, 0x022, 0x044, 0x2d6,
+ 0x002, 0x001, 0x005, 0x101, 0x000, 0x3f7, 0x347, 0x013,
+ 0x022, 0x346, 0x028, 0x022, 0x3c4, 0x0ff, 0x022, 0x362,
+ 0x022, 0x0ef, 0x009, 0x3d2, 0x347, 0x013, 0x022, 0x327,
+ 0x028, 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0be, 0x009,
+ 0x3c5, 0x347, 0x013, 0x022, 0x346, 0x029, 0x022, 0x3c4,
+ 0x0ff, 0x022, 0x362, 0x022, 0x0ad, 0x009, 0x3b7, 0x347,
+ 0x013, 0x022, 0x327, 0x029, 0x1b7, 0x2c6, 0x022, 0x362,
+ 0x022, 0x0de, 0x009, 0x3aa, 0x000, 0x3a4, 0x340, 0x022,
+ 0x028, 0x3c1, 0x000, 0x029, 0x3c6, 0x0d8, 0x028, 0x349,
+ 0x028, 0x028, 0x34b, 0x029, 0x029, 0x00a, 0x008, 0x3c0,
+ 0x0e5, 0x028, 0x3c1, 0x000, 0x029, 0x327, 0x023, 0x222,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x051,
+ 0x1a7, 0x2c7, 0x023, 0x101, 0x051, 0x2c5, 0x023, 0x111,
+ 0x240, 0x023, 0x000, 0x332, 0x347, 0x023, 0x026, 0x000,
+ 0x304, 0x347, 0x023, 0x027, 0x000, 0x2ff, 0x347, 0x023,
+ 0x028, 0x000, 0x2fa, 0x347, 0x023, 0x029, 0x000, 0x2f5,
+ 0x347, 0x023, 0x02a, 0x000, 0x2f0, 0x347, 0x023, 0x02b,
+ 0x000, 0x2eb, 0x347, 0x028, 0x009, 0x000, 0x2e6, 0x347,
+ 0x029, 0x009, 0x000, 0x2e1, 0x347, 0x02a, 0x009, 0x000,
+ 0x2dc, 0x347, 0x02b, 0x009, 0x000, 0x2d7, 0x347, 0x023,
+ 0x02d, 0x055, 0x347, 0x023, 0x02c, 0x000, 0x2ce, 0x347,
+ 0x023, 0x021, 0x000, 0x2c9, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x327, 0x023, 0x04d, 0x000,
+ 0x2bc, 0x347, 0x023, 0x021, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x04c, 0x2c7, 0x009, 0x000,
+ 0x2ac, 0x08c, 0x347, 0x023, 0x06f, 0x055, 0x247, 0x02c,
+ 0x000, 0x2ab, 0x08d, 0x000, 0x3f7, 0x08e, 0x000, 0x3f4,
+ 0x08f, 0x000, 0x3f1, 0x307, 0x022, 0x183, 0x19b, 0x262,
+ 0x009, 0x00b, 0x007, 0x347, 0x037, 0x008, 0x000, 0x292,
+ 0x390, 0x044, 0x000, 0x3ca, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x009, 0x000, 0x27f, 0x000, 0x27d,
+ 0x364, 0x004, 0x002, 0x009, 0x090, 0x364, 0x004, 0x008,
+ 0x009, 0x165, 0x364, 0x004, 0x004, 0x009, 0x08e, 0x0c0,
+ 0x007, 0x3d5, 0x080, 0x00e, 0x347, 0x006, 0x062, 0x0c0,
+ 0x064, 0x307, 0x062, 0x197, 0x384, 0x00f, 0x229, 0x220,
+ 0x3b0, 0x3ca, 0x287, 0x199, 0x390, 0x002, 0x041, 0x040,
+ 0x045, 0x345, 0x033, 0x064, 0x347, 0x064, 0x007, 0x344,
+ 0x035, 0x007, 0x3d4, 0x37f, 0x00e, 0x3c5, 0x020, 0x00e,
+ 0x04e, 0x047, 0x020, 0x3ff, 0x002, 0x020, 0x00a, 0x003,
+ 0x020, 0x011, 0x003, 0x020, 0x01a, 0x003, 0x020, 0x01d,
+ 0x003, 0x020, 0x020, 0x003, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002,
+ 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa,
+ 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x345, 0x037, 0x064, 0x000, 0x3bf, 0x3b7,
+ 0x010, 0x1a7, 0x3a5, 0x013, 0x324, 0x038, 0x2c7, 0x064,
+ 0x000, 0x3b1, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001, 0x000,
+ 0x3f5, 0x088, 0x055, 0x247, 0x02c, 0x347, 0x062, 0x06e,
+ 0x000, 0x3ad, 0x089, 0x000, 0x3f7, 0x08a, 0x000, 0x3f4,
+ 0x08b, 0x000, 0x3f1, 0x055, 0x0c3, 0x02c, 0x3d5, 0x100,
+ 0x00e, 0x000, 0x39c, 0x347, 0x011, 0x066, 0x3c4, 0x3fc,
+ 0x066, 0x364, 0x065, 0x040, 0x009, 0x0a3, 0x044, 0x085,
+ 0x003, 0x001, 0x06f, 0x044, 0x06b, 0x003, 0x001, 0x005,
+ 0x055, 0x000, 0x384, 0x044, 0x074, 0x003, 0x3c4, 0x37f,
+ 0x065, 0x1b5, 0x3a4, 0x038, 0x3c4, 0x3c7, 0x065, 0x2c5,
+ 0x065, 0x1b2, 0x2e2, 0x007, 0x001, 0x047, 0x2e2, 0x006,
+ 0x001, 0x043, 0x2e2, 0x002, 0x001, 0x047, 0x2e2, 0x001,
+ 0x001, 0x031, 0x3d7, 0x07f, 0x064, 0x0a6, 0x044, 0x0bf,
+ 0x003, 0x000, 0x35c, 0x364, 0x010, 0x002, 0x009, 0x005,
+ 0x364, 0x011, 0x001, 0x045, 0x3d5, 0x040, 0x00e, 0x3c5,
+ 0x001, 0x010, 0x3c4, 0x3bf, 0x065, 0x327, 0x066, 0x00b,
+ 0x005, 0x3c5, 0x040, 0x065, 0x045, 0x327, 0x004, 0x307,
+ 0x066, 0x198, 0x384, 0x038, 0x1b8, 0x3a4, 0x038, 0x222,
+ 0x045, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7, 0x064,
+ 0x0a7, 0x000, 0x3cd, 0x347, 0x066, 0x064, 0x044, 0x0cd,
+ 0x003, 0x000, 0x324, 0x055, 0x0c4, 0x02c, 0x000, 0x31f,
+ 0x044, 0x074, 0x003, 0x3c5, 0x080, 0x065, 0x3c4, 0x3f8,
+ 0x065, 0x3a4, 0x1fc, 0x081, 0x245, 0x065, 0x1b1, 0x390,
+ 0x066, 0x04d, 0x3c5, 0x100, 0x065, 0x000, 0x308, 0x3c4,
+ 0x07f, 0x064, 0x1c1, 0x064, 0x307, 0x066, 0x198, 0x384,
+ 0x038, 0x285, 0x188, 0x245, 0x064, 0x3c4, 0x003, 0x010,
+ 0x345, 0x064, 0x010, 0x3c5, 0x002, 0x010, 0x045, 0x044,
+ 0x085, 0x003, 0x009, 0x009, 0x307, 0x065, 0x364, 0x065,
+ 0x080, 0x009, 0x00c, 0x044, 0x06b, 0x003, 0x009, 0x35a,
+ 0x044, 0x074, 0x003, 0x000, 0x3b0, 0x044, 0x074, 0x003,
+ 0x384, 0x007, 0x262, 0x007, 0x001, 0x2d1, 0x141, 0x065,
+ 0x101, 0x324, 0x035, 0x000, 0x3bb, 0x3d5, 0x010, 0x00e,
+ 0x055, 0x0d3, 0x02c, 0x000, 0x2c2, 0x04e, 0x062, 0x0b1,
+ 0x0e7, 0x079
+
+}; /* end fm10000_sbus_master_code_prd */
+
+/* Spico SerDes code (production version 2)
+ * source file: serdes.0x2055/serdes.0x2055_0045.rom
+ * KR support
+ */
+const uint32_t fm10000_serdes_spico_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_spico_code_size_prd2 = 12167;
+
+const uint16_t fm10000_serdes_spico_code_prd2[] = {
+ 0x035, 0x006, 0x00a, 0x000, 0x1a7, 0x006, 0x020, 0x055,
+ 0x000, 0x045, 0x3c7, 0x03f, 0x000, 0x0c0, 0x200, 0x307,
+ 0x200, 0x304, 0x02b, 0x009, 0x00b, 0x0a0, 0x397, 0x3bb,
+ 0x04d, 0x111, 0x272, 0x1ff, 0x009, 0x3fc, 0x3c8, 0x000,
+ 0x21e, 0x3ce, 0x3ff, 0x21f, 0x348, 0x21f, 0x21d, 0x34e,
+ 0x21d, 0x221, 0x34e, 0x221, 0x21c, 0x34e, 0x21c, 0x220,
+ 0x3c7, 0x3fd, 0x23d, 0x3c7, 0x020, 0x23e, 0x3c7, 0x077,
+ 0x22d, 0x0c0, 0x202, 0x044, 0x36d, 0x003, 0x044, 0x360,
+ 0x002, 0x3d7, 0x303, 0x030, 0x347, 0x032, 0x236, 0x3b7,
+ 0x213, 0x090, 0x044, 0x1a9, 0x000, 0x3b7, 0x204, 0x387,
+ 0x07c, 0x044, 0x1a9, 0x000, 0x0a4, 0x397, 0x27c, 0x044,
+ 0x1a9, 0x000, 0x3d5, 0x230, 0x232, 0x345, 0x21d, 0x232,
+ 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000, 0x3b7,
+ 0x080, 0x044, 0x192, 0x000, 0x385, 0x00c, 0x384, 0x3cf,
+ 0x044, 0x1bd, 0x000, 0x3c5, 0x040, 0x021, 0x3b7, 0x020,
+ 0x387, 0x021, 0x189, 0x044, 0x1a9, 0x000, 0x0c0, 0x008,
+ 0x0c0, 0x005, 0x0c0, 0x00b, 0x0c0, 0x000, 0x3c5, 0x020,
+ 0x027, 0x055, 0x307, 0x221, 0x304, 0x233, 0x009, 0x012,
+ 0x374, 0x027, 0x200, 0x001, 0x00d, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x009, 0x006, 0x3b8, 0x200, 0x2c4, 0x027,
+ 0x054, 0x364, 0x200, 0x004, 0x029, 0x3de, 0x003, 0x364,
+ 0x33b, 0x010, 0x029, 0x3de, 0x003, 0x054, 0x364, 0x200,
+ 0x040, 0x029, 0x034, 0x008, 0x364, 0x2cf, 0x040, 0x009,
+ 0x01c, 0x307, 0x026, 0x384, 0x003, 0x266, 0x003, 0x009,
+ 0x014, 0x362, 0x26b, 0x0e0, 0x009, 0x00f, 0x364, 0x234,
+ 0x002, 0x001, 0x00a, 0x3c5, 0x060, 0x2cf, 0x0c0, 0x26b,
+ 0x3c5, 0x040, 0x200, 0x054, 0x364, 0x200, 0x100, 0x029,
+ 0x22f, 0x003, 0x054, 0x044, 0x141, 0x001, 0x055, 0x364,
+ 0x23f, 0x003, 0x001, 0x012, 0x044, 0x34c, 0x001, 0x029,
+ 0x0af, 0x006, 0x044, 0x340, 0x001, 0x227, 0x326, 0x249,
+ 0x1ba, 0x029, 0x0af, 0x006, 0x3c4, 0x3df, 0x000, 0x364,
+ 0x33c, 0x020, 0x001, 0x017, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x055, 0x364, 0x200, 0x080, 0x001, 0x00d, 0x3c4,
+ 0x3fd, 0x26f, 0x3c4, 0x3fe, 0x200, 0x366, 0x200, 0x080,
+ 0x001, 0x00f, 0x140, 0x200, 0x009, 0x365, 0x364, 0x23f,
+ 0x003, 0x009, 0x360, 0x04f, 0x04e, 0x000, 0x35c, 0x3c5,
+ 0x030, 0x000, 0x347, 0x244, 0x054, 0x374, 0x000, 0x2c0,
+ 0x009, 0x34c, 0x387, 0x0c1, 0x187, 0x245, 0x054, 0x04f,
+ 0x055, 0x141, 0x26c, 0x347, 0x245, 0x200, 0x347, 0x245,
+ 0x26d, 0x0c0, 0x245, 0x364, 0x026, 0x002, 0x001, 0x336,
+ 0x364, 0x020, 0x005, 0x009, 0x007, 0x0c2, 0x26f, 0x044,
+ 0x1b8, 0x001, 0x055, 0x364, 0x020, 0x005, 0x009, 0x326,
+ 0x362, 0x26b, 0x0e0, 0x009, 0x321, 0x364, 0x234, 0x002,
+ 0x001, 0x31c, 0x3c5, 0x060, 0x2cf, 0x3c5, 0x040, 0x200,
+ 0x000, 0x314, 0x044, 0x290, 0x001, 0x3c5, 0x080, 0x200,
+ 0x347, 0x2cf, 0x26b, 0x3c4, 0x0e0, 0x26b, 0x3c4, 0x3bf,
+ 0x2cf, 0x045, 0x387, 0x080, 0x020, 0x2f8, 0x001, 0x387,
+ 0x02f, 0x111, 0x009, 0x3ff, 0x045, 0x054, 0x111, 0x009,
+ 0x3fe, 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x032, 0x054,
+ 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x033, 0x054, 0x045,
+ 0x055, 0x2c7, 0x030, 0x364, 0x035, 0x001, 0x009, 0x3fd,
+ 0x045, 0x044, 0x1a0, 0x000, 0x305, 0x033, 0x000, 0x012,
+ 0x044, 0x1a0, 0x000, 0x306, 0x033, 0x000, 0x00b, 0x208,
+ 0x044, 0x1a0, 0x000, 0x304, 0x033, 0x055, 0x2c7, 0x030,
+ 0x247, 0x031, 0x345, 0x21d, 0x030, 0x140, 0x035, 0x009,
+ 0x3fe, 0x346, 0x21d, 0x030, 0x054, 0x045, 0x040, 0x140,
+ 0x036, 0x00b, 0x020, 0x044, 0x206, 0x000, 0x344, 0x21f,
+ 0x036, 0x3c4, 0x3ef, 0x036, 0x09f, 0x044, 0x189, 0x000,
+ 0x000, 0x010, 0x040, 0x140, 0x036, 0x003, 0x00c, 0x044,
+ 0x206, 0x000, 0x345, 0x21d, 0x036, 0x3c4, 0x3ef, 0x036,
+ 0x054, 0x327, 0x22b, 0x3a4, 0x003, 0x133, 0x009, 0x00e,
+ 0x0a1, 0x1aa, 0x2a8, 0x2c4, 0x021, 0x08f, 0x044, 0x189,
+ 0x000, 0x2a8, 0x2c5, 0x021, 0x042, 0x045, 0x055, 0x088,
+ 0x3c5, 0x010, 0x036, 0x000, 0x37e, 0x3c4, 0x3fe, 0x026,
+ 0x327, 0x22c, 0x3a4, 0x007, 0x2e6, 0x001, 0x001, 0x01b,
+ 0x044, 0x1e2, 0x000, 0x364, 0x234, 0x001, 0x009, 0x013,
+ 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x3b6, 0x003,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x066, 0x004,
+ 0x054, 0x045, 0x364, 0x233, 0x002, 0x009, 0x00d, 0x347,
+ 0x271, 0x270, 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x044,
+ 0x02b, 0x008, 0x374, 0x232, 0x200, 0x001, 0x0a5, 0x3d7,
+ 0x200, 0x27b, 0x044, 0x29c, 0x001, 0x044, 0x2f6, 0x002,
+ 0x009, 0x005, 0x044, 0x028, 0x008, 0x3b7, 0x221, 0x364,
+ 0x22b, 0x003, 0x009, 0x007, 0x083, 0x18b, 0x044, 0x1a9,
+ 0x000, 0x398, 0x200, 0x244, 0x232, 0x3b7, 0x221, 0x091,
+ 0x189, 0x385, 0x128, 0x044, 0x1a9, 0x000, 0x083, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x220, 0x044, 0x192, 0x000, 0x197,
+ 0x264, 0x007, 0x009, 0x005, 0x091, 0x000, 0x003, 0x081,
+ 0x3b7, 0x221, 0x189, 0x044, 0x1b7, 0x000, 0x364, 0x246,
+ 0x010, 0x009, 0x002, 0x3b7, 0x221, 0x081, 0x18d, 0x044,
+ 0x1a9, 0x000, 0x081, 0x189, 0x044, 0x1a9, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x364, 0x22b, 0x003, 0x009,
+ 0x009, 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000,
+ 0x364, 0x02b, 0x020, 0x009, 0x020, 0x0a3, 0x364, 0x22b,
+ 0x003, 0x001, 0x006, 0x044, 0x025, 0x008, 0x0a0, 0x307,
+ 0x251, 0x197, 0x384, 0x00f, 0x220, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x327, 0x251, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x044, 0x1ce, 0x000, 0x364, 0x232,
+ 0x100, 0x001, 0x005, 0x044, 0x0a9, 0x001, 0x3c4, 0x3f0,
+ 0x22d, 0x3c5, 0x001, 0x026, 0x364, 0x026, 0x002, 0x001,
+ 0x1c5, 0x044, 0x283, 0x001, 0x3b7, 0x300, 0x387, 0x020,
+ 0x000, 0x2c1, 0x307, 0x036, 0x00b, 0x3ea, 0x044, 0x1ce,
+ 0x000, 0x000, 0x3e5, 0x387, 0x3ff, 0x327, 0x22c, 0x3a4,
+ 0x007, 0x131, 0x001, 0x01a, 0x3d5, 0x200, 0x232, 0x000,
+ 0x006, 0x080, 0x131, 0x001, 0x011, 0x3b7, 0x205, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x222, 0x044, 0x1bd, 0x000, 0x3b7,
+ 0x251, 0x044, 0x1bd, 0x000, 0x3b7, 0x214, 0x100, 0x009,
+ 0x00a, 0x044, 0x1bd, 0x000, 0x3c4, 0x37b, 0x021, 0x000,
+ 0x29e, 0x3c5, 0x084, 0x021, 0x000, 0x299, 0x364, 0x233,
+ 0x002, 0x009, 0x00d, 0x347, 0x272, 0x270, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x02e, 0x008, 0x307, 0x22d,
+ 0x384, 0x070, 0x386, 0x010, 0x001, 0x04a, 0x364, 0x02b,
+ 0x020, 0x009, 0x045, 0x3b7, 0x071, 0x387, 0x128, 0x044,
+ 0x1a9, 0x000, 0x387, 0x368, 0x044, 0x1b8, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x0a1, 0x307, 0x22b, 0x384,
+ 0x030, 0x001, 0x003, 0x0ac, 0x307, 0x256, 0x384, 0x01f,
+ 0x220, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x080, 0x327,
+ 0x22b, 0x3a4, 0x030, 0x3a2, 0x030, 0x009, 0x00e, 0x0a1,
+ 0x1ab, 0x2a8, 0x2c4, 0x024, 0x08f, 0x044, 0x189, 0x000,
+ 0x2a8, 0x2c5, 0x024, 0x327, 0x256, 0x1b7, 0x3a4, 0x07f,
+ 0x220, 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x044, 0x2f6,
+ 0x002, 0x009, 0x007, 0x0c0, 0x27b, 0x044, 0x29c, 0x001,
+ 0x044, 0x0e6, 0x001, 0x3c4, 0x30f, 0x22d, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x031, 0x008, 0x345, 0x21d,
+ 0x01c, 0x307, 0x21d, 0x304, 0x233, 0x001, 0x007, 0x3a7,
+ 0x071, 0x044, 0x1a9, 0x000, 0x3c5, 0x002, 0x026, 0x044,
+ 0x141, 0x001, 0x020, 0x283, 0x001, 0x055, 0x3c4, 0x3ef,
+ 0x027, 0x3c4, 0x3fd, 0x026, 0x364, 0x26f, 0x002, 0x001,
+ 0x008, 0x3c4, 0x3fd, 0x26f, 0x044, 0x290, 0x001, 0x344,
+ 0x21f, 0x01c, 0x3c4, 0x3bf, 0x200, 0x3c4, 0x39f, 0x2cf,
+ 0x0c0, 0x2d8, 0x0c0, 0x26b, 0x364, 0x234, 0x002, 0x009,
+ 0x012, 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x066,
+ 0x004, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x3b6,
+ 0x003, 0x054, 0x0a0, 0x081, 0x189, 0x020, 0x1a9, 0x000,
+ 0x044, 0x3fd, 0x000, 0x387, 0x3ff, 0x044, 0x02b, 0x001,
+ 0x3a7, 0x022, 0x020, 0x1bd, 0x000, 0x0a0, 0x081, 0x189,
+ 0x020, 0x1a9, 0x000, 0x080, 0x044, 0x02b, 0x001, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x364, 0x024,
+ 0x002, 0x001, 0x006, 0x399, 0x305, 0x000, 0x004, 0x397,
+ 0x200, 0x044, 0x322, 0x001, 0x3a7, 0x022, 0x044, 0x1bd,
+ 0x000, 0x364, 0x232, 0x002, 0x001, 0x005, 0x044, 0x382,
+ 0x001, 0x000, 0x0c9, 0x3a7, 0x061, 0x044, 0x1bd, 0x000,
+ 0x0a5, 0x044, 0x1bd, 0x000, 0x3a7, 0x072, 0x044, 0x1bd,
+ 0x000, 0x3a7, 0x0f1, 0x044, 0x1bd, 0x000, 0x385, 0x004,
+ 0x0b1, 0x020, 0x1bd, 0x000, 0x364, 0x22c, 0x002, 0x009,
+ 0x2ac, 0x364, 0x234, 0x001, 0x001, 0x058, 0x020, 0x232,
+ 0x000, 0x3c4, 0x3fe, 0x027, 0x364, 0x234, 0x002, 0x001,
+ 0x04d, 0x000, 0x2cd, 0x3b7, 0x211, 0x044, 0x199, 0x000,
+ 0x384, 0x00f, 0x140, 0x239, 0x00b, 0x004, 0x385, 0x010,
+ 0x140, 0x237, 0x00b, 0x004, 0x385, 0x020, 0x044, 0x1bd,
+ 0x000, 0x307, 0x239, 0x00b, 0x004, 0x208, 0x101, 0x384,
+ 0x01f, 0x183, 0x327, 0x237, 0x00b, 0x004, 0x2a8, 0x121,
+ 0x3a4, 0x00f, 0x285, 0x184, 0x3c4, 0x01f, 0x238, 0x362,
+ 0x238, 0x017, 0x00b, 0x005, 0x0a1, 0x1ad, 0x285, 0x305,
+ 0x238, 0x32a, 0x238, 0x3a4, 0x008, 0x285, 0x3b7, 0x210,
+ 0x044, 0x1bd, 0x000, 0x347, 0x237, 0x23a, 0x347, 0x238,
+ 0x23c, 0x347, 0x239, 0x23b, 0x045, 0x345, 0x21c, 0x021,
+ 0x045, 0x308, 0x21c, 0x244, 0x021, 0x0c0, 0x240, 0x307,
+ 0x022, 0x384, 0x001, 0x245, 0x240, 0x142, 0x240, 0x364,
+ 0x240, 0x100, 0x009, 0x3eb, 0x0a1, 0x044, 0x0d9, 0x001,
+ 0x307, 0x240, 0x327, 0x022, 0x286, 0x384, 0x001, 0x001,
+ 0x3ee, 0x0a2, 0x307, 0x22b, 0x384, 0x003, 0x001, 0x008,
+ 0x0aa, 0x262, 0x003, 0x001, 0x003, 0x0a5, 0x289, 0x240,
+ 0x240, 0x387, 0x084, 0x020, 0x2f8, 0x001, 0x094, 0x020,
+ 0x2f2, 0x001, 0x090, 0x020, 0x2f2, 0x001, 0x0a0, 0x082,
+ 0x18b, 0x304, 0x020, 0x009, 0x3b9, 0x081, 0x189, 0x020,
+ 0x1b7, 0x000, 0x0a1, 0x1a9, 0x364, 0x234, 0x002, 0x001,
+ 0x018, 0x364, 0x22c, 0x020, 0x001, 0x016, 0x364, 0x22c,
+ 0x010, 0x001, 0x009, 0x364, 0x232, 0x080, 0x001, 0x00c,
+ 0x000, 0x007, 0x364, 0x232, 0x040, 0x001, 0x005, 0x2c5,
+ 0x024, 0x045, 0x309, 0x024, 0x003, 0x3fb, 0x2a8, 0x2c4,
+ 0x024, 0x045, 0x3b7, 0x213, 0x364, 0x234, 0x004, 0x009,
+ 0x017, 0x307, 0x233, 0x182, 0x00a, 0x007, 0x3c5, 0x040,
+ 0x021, 0x000, 0x005, 0x3c4, 0x3bf, 0x021, 0x082, 0x044,
+ 0x1b7, 0x000, 0x088, 0x020, 0x1a9, 0x000, 0x082, 0x044,
+ 0x1a9, 0x000, 0x3c4, 0x3bf, 0x021, 0x088, 0x020, 0x1b7,
+ 0x000, 0x055, 0x364, 0x026, 0x002, 0x001, 0x031, 0x364,
+ 0x024, 0x002, 0x009, 0x007, 0x364, 0x025, 0x004, 0x009,
+ 0x027, 0x309, 0x01c, 0x003, 0x004, 0x00a, 0x021, 0x364,
+ 0x2cf, 0x011, 0x009, 0x01c, 0x364, 0x33b, 0x007, 0x001,
+ 0x007, 0x364, 0x33b, 0x002, 0x001, 0x012, 0x3c5, 0x010,
+ 0x027, 0x364, 0x234, 0x040, 0x001, 0x005, 0x3c4, 0x37f,
+ 0x024, 0x3c5, 0x008, 0x025, 0x000, 0x010, 0x364, 0x234,
+ 0x040, 0x001, 0x005, 0x3c5, 0x080, 0x024, 0x3c4, 0x3ef,
+ 0x027, 0x3c5, 0x010, 0x026, 0x054, 0x045, 0x364, 0x026,
+ 0x001, 0x001, 0x008, 0x3c5, 0x004, 0x232, 0x044, 0x20d,
+ 0x000, 0x09f, 0x18a, 0x208, 0x244, 0x243, 0x244, 0x250,
+ 0x327, 0x243, 0x326, 0x250, 0x001, 0x308, 0x347, 0x243,
+ 0x250, 0x087, 0x187, 0x208, 0x244, 0x020, 0x364, 0x026,
+ 0x002, 0x001, 0x2fb, 0x3c5, 0x008, 0x232, 0x020, 0x3b5,
+ 0x000, 0x364, 0x233, 0x008, 0x001, 0x004, 0x307, 0x220,
+ 0x3c5, 0x001, 0x00c, 0x247, 0x01b, 0x083, 0x18d, 0x244,
+ 0x01c, 0x044, 0x1c9, 0x001, 0x0a3, 0x1a8, 0x2c5, 0x01f,
+ 0x045, 0x3c4, 0x0ff, 0x01f, 0x085, 0x044, 0x189, 0x000,
+ 0x3d5, 0x100, 0x01f, 0x3b7, 0x300, 0x397, 0x200, 0x044,
+ 0x1a9, 0x000, 0x3c7, 0x07f, 0x252, 0x055, 0x151, 0x252,
+ 0x001, 0x006, 0x140, 0x01e, 0x00b, 0x3fa, 0x397, 0x200,
+ 0x044, 0x1b7, 0x000, 0x3c4, 0x2ff, 0x01f, 0x045, 0x327,
+ 0x22c, 0x307, 0x22d, 0x2e4, 0x004, 0x009, 0x00d, 0x286,
+ 0x384, 0x007, 0x001, 0x00f, 0x2e4, 0x007, 0x001, 0x032,
+ 0x000, 0x039, 0x384, 0x007, 0x3c4, 0x3f8, 0x22c, 0x245,
+ 0x22c, 0x327, 0x22c, 0x307, 0x22d, 0x2e4, 0x040, 0x009,
+ 0x016, 0x286, 0x264, 0x070, 0x001, 0x018, 0x3a4, 0x070,
+ 0x001, 0x029, 0x1aa, 0x002, 0x02e, 0x044, 0x3b5, 0x000,
+ 0x044, 0x3fd, 0x000, 0x000, 0x009, 0x384, 0x070, 0x3c4,
+ 0x38f, 0x22c, 0x245, 0x22c, 0x347, 0x22c, 0x22d, 0x045,
+ 0x227, 0x044, 0x301, 0x000, 0x044, 0x232, 0x000, 0x000,
+ 0x3d2, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x000,
+ 0x3ca, 0x044, 0x003, 0x001, 0x044, 0x326, 0x000, 0x000,
+ 0x3e5, 0x044, 0x0f2, 0x001, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x000, 0x3da, 0x3a7, 0x080, 0x384, 0x001,
+ 0x001, 0x011, 0x082, 0x044, 0x1a9, 0x000, 0x081, 0x044,
+ 0x1b7, 0x000, 0x3c5, 0x004, 0x024, 0x3c5, 0x008, 0x025,
+ 0x045, 0x366, 0x22e, 0x020, 0x001, 0x010, 0x082, 0x044,
+ 0x1b7, 0x000, 0x081, 0x044, 0x1a9, 0x000, 0x3c4, 0x3fb,
+ 0x024, 0x3c4, 0x3f7, 0x025, 0x045, 0x308, 0x248, 0x19e,
+ 0x020, 0x254, 0x001, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x020, 0x1b6, 0x001,
+ 0x3c5, 0x001, 0x00c, 0x0c0, 0x01b, 0x044, 0x1c9, 0x001,
+ 0x3c4, 0x3fe, 0x00c, 0x045, 0x0a1, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x247, 0x24f, 0x397, 0x3e0, 0x044, 0x1a9,
+ 0x000, 0x0a0, 0x325, 0x27b, 0x387, 0x140, 0x044, 0x1a9,
+ 0x000, 0x044, 0x187, 0x000, 0x084, 0x044, 0x1a9, 0x000,
+ 0x044, 0x187, 0x000, 0x387, 0x040, 0x044, 0x1b7, 0x000,
+ 0x387, 0x106, 0x044, 0x189, 0x000, 0x387, 0x100, 0x044,
+ 0x1b7, 0x000, 0x387, 0x099, 0x044, 0x189, 0x000, 0x0a1,
+ 0x325, 0x27b, 0x307, 0x24f, 0x044, 0x1bd, 0x000, 0x0a0,
+ 0x325, 0x27b, 0x084, 0x044, 0x1b7, 0x000, 0x387, 0x099,
+ 0x044, 0x2f6, 0x002, 0x009, 0x004, 0x399, 0x3ff, 0x020,
+ 0x189, 0x000, 0x041, 0x327, 0x236, 0x3a4, 0x00c, 0x134,
+ 0x043, 0x045, 0x055, 0x3d7, 0x071, 0x030, 0x000, 0x006,
+ 0x055, 0x3d7, 0x221, 0x030, 0x347, 0x034, 0x258, 0x0c0,
+ 0x034, 0x140, 0x035, 0x009, 0x3fe, 0x347, 0x032, 0x031,
+ 0x345, 0x21d, 0x030, 0x246, 0x031, 0x374, 0x030, 0x200,
+ 0x001, 0x007, 0x246, 0x031, 0x044, 0x18c, 0x000, 0x131,
+ 0x009, 0x3f3, 0x346, 0x21d, 0x030, 0x347, 0x258, 0x034,
+ 0x054, 0x045, 0x364, 0x233, 0x040, 0x001, 0x006, 0x3b7,
+ 0x200, 0x2a8, 0x284, 0x045, 0x227, 0x055, 0x040, 0x397,
+ 0x203, 0x300, 0x202, 0x04d, 0x141, 0x202, 0x362, 0x202,
+ 0x018, 0x009, 0x004, 0x0c0, 0x202, 0x042, 0x054, 0x045,
+ 0x055, 0x3c7, 0x0f2, 0x030, 0x364, 0x035, 0x001, 0x009,
+ 0x3fd, 0x307, 0x032, 0x045, 0x0a1, 0x1aa, 0x324, 0x233,
+ 0x045, 0x327, 0x222, 0x3a4, 0x07f, 0x001, 0x02c, 0x055,
+ 0x307, 0x034, 0x0c0, 0x034, 0x040, 0x140, 0x035, 0x009,
+ 0x3fe, 0x3c7, 0x071, 0x030, 0x387, 0x0d7, 0x347, 0x032,
+ 0x031, 0x345, 0x21d, 0x030, 0x141, 0x23d, 0x009, 0x007,
+ 0x153, 0x23d, 0x346, 0x21c, 0x031, 0x246, 0x031, 0x131,
+ 0x009, 0x3f4, 0x346, 0x21d, 0x030, 0x042, 0x247, 0x034,
+ 0x054, 0x045, 0x083, 0x18d, 0x345, 0x21d, 0x233, 0x364,
+ 0x2cf, 0x004, 0x001, 0x005, 0x245, 0x305, 0x045, 0x3a7,
+ 0x071, 0x020, 0x1a9, 0x000, 0x083, 0x18d, 0x364, 0x2cf,
+ 0x004, 0x001, 0x006, 0x208, 0x244, 0x305, 0x045, 0x328,
+ 0x21d, 0x2c4, 0x233, 0x3a7, 0x071, 0x020, 0x1b7, 0x000,
+ 0x120, 0x001, 0x027, 0x041, 0x2f4, 0x200, 0x001, 0x00a,
+ 0x3a7, 0x073, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x009,
+ 0x3a7, 0x073, 0x387, 0x3fd, 0x044, 0x1b8, 0x000, 0x043,
+ 0x2e4, 0x180, 0x001, 0x016, 0x2e4, 0x080, 0x009, 0x017,
+ 0x3c5, 0x002, 0x232, 0x044, 0x382, 0x001, 0x000, 0x012,
+ 0x3c4, 0x3fd, 0x232, 0x044, 0x394, 0x001, 0x000, 0x00a,
+ 0x044, 0x353, 0x001, 0x000, 0x005, 0x081, 0x043, 0x040,
+ 0x020, 0x181, 0x006, 0x3c4, 0x3fe, 0x03a, 0x247, 0x25e,
+ 0x0c5, 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263,
+ 0x3c4, 0x3bf, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03a,
+ 0x3c5, 0x040, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042,
+ 0x100, 0x00b, 0x008, 0x247, 0x03c, 0x042, 0x247, 0x03b,
+ 0x042, 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03a,
+ 0x041, 0x307, 0x25d, 0x040, 0x152, 0x03a, 0x345, 0x263,
+ 0x03a, 0x045, 0x3c4, 0x3fe, 0x03d, 0x247, 0x25f, 0x0c5,
+ 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263, 0x3c4,
+ 0x37f, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03d, 0x3c5,
+ 0x080, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042, 0x100,
+ 0x00b, 0x008, 0x247, 0x03f, 0x042, 0x247, 0x03e, 0x042,
+ 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03d, 0x041,
+ 0x307, 0x25d, 0x040, 0x152, 0x03d, 0x345, 0x263, 0x03d,
+ 0x045, 0x3c4, 0x3fe, 0x040, 0x0c1, 0x041, 0x3c4, 0x3cf,
+ 0x00c, 0x151, 0x041, 0x0c0, 0x263, 0x247, 0x262, 0x0c0,
+ 0x040, 0x384, 0x003, 0x227, 0x1a0, 0x285, 0x384, 0x3fd,
+ 0x187, 0x245, 0x040, 0x245, 0x041, 0x245, 0x263, 0x3c5,
+ 0x030, 0x00c, 0x3c7, 0x3ff, 0x039, 0x142, 0x040, 0x152,
+ 0x040, 0x144, 0x040, 0x364, 0x262, 0x100, 0x001, 0x004,
+ 0x148, 0x040, 0x042, 0x043, 0x247, 0x261, 0x2c7, 0x260,
+ 0x044, 0x0de, 0x002, 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x04a,
+ 0x307, 0x262, 0x384, 0x030, 0x187, 0x245, 0x04a, 0x0a1,
+ 0x307, 0x262, 0x384, 0x030, 0x193, 0x262, 0x000, 0x001,
+ 0x008, 0x262, 0x001, 0x001, 0x007, 0x000, 0x006, 0x0a0,
+ 0x000, 0x003, 0x0a2, 0x1ad, 0x2c5, 0x263, 0x0a3, 0x307,
+ 0x262, 0x384, 0x00c, 0x191, 0x262, 0x001, 0x001, 0x008,
+ 0x262, 0x002, 0x001, 0x007, 0x000, 0x006, 0x0a1, 0x000,
+ 0x003, 0x0a2, 0x1a5, 0x2c5, 0x263, 0x374, 0x262, 0x200,
+ 0x001, 0x00e, 0x307, 0x261, 0x327, 0x260, 0x041, 0x040,
+ 0x347, 0x263, 0x041, 0x141, 0x040, 0x045, 0x347, 0x263,
+ 0x041, 0x141, 0x040, 0x020, 0x089, 0x000, 0x042, 0x043,
+ 0x247, 0x25c, 0x042, 0x264, 0x3ff, 0x00b, 0x008, 0x247,
+ 0x043, 0x042, 0x247, 0x042, 0x042, 0x247, 0x039, 0x042,
+ 0x247, 0x038, 0x142, 0x040, 0x152, 0x040, 0x307, 0x25c,
+ 0x041, 0x040, 0x045, 0x055, 0x140, 0x259, 0x001, 0x009,
+ 0x322, 0x259, 0x001, 0x007, 0x0a1, 0x054, 0x045, 0x2c7,
+ 0x259, 0x264, 0x008, 0x009, 0x06e, 0x0c0, 0x012, 0x043,
+ 0x2c7, 0x25a, 0x043, 0x2c7, 0x25b, 0x227, 0x274, 0x200,
+ 0x009, 0x022, 0x3a4, 0x006, 0x2c7, 0x00d, 0x227, 0x3c4,
+ 0x3fd, 0x00c, 0x3c4, 0x3fc, 0x018, 0x384, 0x001, 0x245,
+ 0x018, 0x142, 0x00c, 0x042, 0x247, 0x014, 0x042, 0x247,
+ 0x013, 0x120, 0x00b, 0x008, 0x042, 0x247, 0x011, 0x042,
+ 0x247, 0x010, 0x3c7, 0x050, 0x26e, 0x142, 0x012, 0x141,
+ 0x00d, 0x080, 0x2e4, 0x100, 0x001, 0x003, 0x108, 0x151,
+ 0x26e, 0x001, 0x007, 0x364, 0x012, 0x040, 0x009, 0x3f9,
+ 0x152, 0x012, 0x151, 0x00d, 0x2e4, 0x080, 0x001, 0x003,
+ 0x104, 0x245, 0x012, 0x307, 0x25b, 0x040, 0x307, 0x25a,
+ 0x040, 0x2a9, 0x003, 0x017, 0x151, 0x26e, 0x003, 0x007,
+ 0x364, 0x012, 0x080, 0x009, 0x3f9, 0x364, 0x026, 0x002,
+ 0x001, 0x007, 0x364, 0x025, 0x004, 0x009, 0x007, 0x141,
+ 0x012, 0x0a0, 0x054, 0x045, 0x364, 0x2cf, 0x007, 0x001,
+ 0x3f8, 0x3c4, 0x3fc, 0x2cf, 0x3d5, 0x200, 0x2cf, 0x020,
+ 0x33a, 0x002, 0x247, 0x017, 0x287, 0x00b, 0x006, 0x3a7,
+ 0x3ff, 0x000, 0x003, 0x1bb, 0x384, 0x0ff, 0x264, 0x080,
+ 0x001, 0x004, 0x385, 0x300, 0x380, 0x011, 0x00b, 0x018,
+ 0x041, 0x0a0, 0x131, 0x380, 0x014, 0x003, 0x3fd, 0x041,
+ 0x0bf, 0x1a6, 0x2a8, 0x2c4, 0x018, 0x186, 0x245, 0x018,
+ 0x083, 0x18d, 0x245, 0x017, 0x042, 0x043, 0x103, 0x121,
+ 0x18a, 0x1a7, 0x285, 0x247, 0x01a, 0x387, 0x3ff, 0x247,
+ 0x019, 0x18b, 0x245, 0x018, 0x307, 0x017, 0x19d, 0x262,
+ 0x003, 0x001, 0x00e, 0x262, 0x001, 0x001, 0x013, 0x262,
+ 0x002, 0x009, 0x037, 0x3a7, 0x0aa, 0x000, 0x00d, 0x0c1,
+ 0x019, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x000, 0x039,
+ 0x3a7, 0x055, 0x041, 0x3a7, 0x071, 0x044, 0x192, 0x000,
+ 0x00b, 0x010, 0x040, 0x3a7, 0x0f2, 0x044, 0x192, 0x000,
+ 0x198, 0x043, 0x3a4, 0x001, 0x009, 0x004, 0x386, 0x001,
+ 0x043, 0x264, 0x001, 0x001, 0x004, 0x3a6, 0x0ff, 0x287,
+ 0x187, 0x285, 0x247, 0x019, 0x18b, 0x103, 0x244, 0x018,
+ 0x364, 0x24d, 0x030, 0x009, 0x00c, 0x08f, 0x18b, 0x208,
+ 0x244, 0x018, 0x397, 0x1fe, 0x182, 0x244, 0x019, 0x327,
+ 0x017, 0x1ba, 0x3a4, 0x007, 0x001, 0x019, 0x2e2, 0x003,
+ 0x001, 0x015, 0x041, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x043, 0x384, 0x008, 0x001, 0x00a, 0x2e2, 0x003, 0x003,
+ 0x004, 0x3a6, 0x002, 0x3a6, 0x001, 0x387, 0x002, 0x189,
+ 0x395, 0x23f, 0x280, 0x050, 0x245, 0x01a, 0x045, 0x0ff,
+ 0x00c, 0x030, 0x03c, 0x00f, 0x0f0, 0x004, 0x020, 0x307,
+ 0x26f, 0x384, 0x003, 0x229, 0x220, 0x3b0, 0x04b, 0x287,
+ 0x199, 0x390, 0x008, 0x041, 0x040, 0x045, 0x055, 0x329,
+ 0x01f, 0x00b, 0x08b, 0x364, 0x234, 0x080, 0x001, 0x00e,
+ 0x364, 0x024, 0x002, 0x009, 0x009, 0x364, 0x025, 0x004,
+ 0x001, 0x004, 0x000, 0x012, 0x327, 0x01d, 0x009, 0x00a,
+ 0x081, 0x18d, 0x245, 0x020, 0x327, 0x01d, 0x246, 0x020,
+ 0x3a2, 0x020, 0x003, 0x02b, 0x309, 0x01c, 0x003, 0x022,
+ 0x3c4, 0x3ef, 0x027, 0x3c5, 0x010, 0x026, 0x0a0, 0x387,
+ 0x3df, 0x044, 0x1b8, 0x000, 0x081, 0x189, 0x044, 0x1a9,
+ 0x000, 0x140, 0x01c, 0x00b, 0x00d, 0x0a1, 0x044, 0x192,
+ 0x000, 0x247, 0x27f, 0x395, 0x3ff, 0x044, 0x1bd, 0x000,
+ 0x344, 0x21f, 0x01c, 0x000, 0x02a, 0x307, 0x01c, 0x003,
+ 0x026, 0x345, 0x21d, 0x01c, 0x209, 0x003, 0x020, 0x307,
+ 0x020, 0x181, 0x003, 0x01b, 0x364, 0x025, 0x004, 0x009,
+ 0x005, 0x3c5, 0x008, 0x025, 0x0a0, 0x081, 0x189, 0x044,
+ 0x1b7, 0x000, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1,
+ 0x307, 0x27f, 0x044, 0x1bd, 0x000, 0x055, 0x364, 0x234,
+ 0x002, 0x001, 0x00f, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x044, 0x1b6, 0x001,
+ 0x054, 0x020, 0x0a3, 0x006, 0x346, 0x21d, 0x232, 0x02b,
+ 0x181, 0x006, 0x364, 0x233, 0x001, 0x021, 0x08d, 0x006,
+ 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4, 0x000, 0x327, 0x21d,
+ 0x324, 0x020, 0x045, 0x307, 0x28d, 0x384, 0x07f, 0x22a,
+ 0x286, 0x040, 0x0b0, 0x044, 0x192, 0x000, 0x384, 0x380,
+ 0x043, 0x285, 0x0b0, 0x020, 0x1bd, 0x000, 0x0a2, 0x141,
+ 0x041, 0x141, 0x2f1, 0x397, 0x200, 0x151, 0x041, 0x044,
+ 0x0fb, 0x002, 0x055, 0x364, 0x200, 0x080, 0x009, 0x00e,
+ 0x364, 0x234, 0x002, 0x001, 0x009, 0x054, 0x364, 0x012,
+ 0x040, 0x001, 0x3f1, 0x045, 0x364, 0x2cf, 0x007, 0x009,
+ 0x3fc, 0x044, 0x33f, 0x002, 0x3c4, 0x080, 0x2cf, 0x3c4,
+ 0x2bf, 0x200, 0x080, 0x046, 0x020, 0x0d3, 0x000, 0x0c0,
+ 0x013, 0x0c0, 0x014, 0x0c0, 0x259, 0x045, 0x088, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x364, 0x259, 0x002, 0x009, 0x007,
+ 0x042, 0x042, 0x020, 0x0d3, 0x000, 0x387, 0x085, 0x187,
+ 0x385, 0x080, 0x247, 0x041, 0x3c5, 0x020, 0x00c, 0x045,
+ 0x387, 0x07f, 0x247, 0x2a1, 0x247, 0x2a2, 0x247, 0x2a3,
+ 0x247, 0x2a4, 0x247, 0x2a7, 0x247, 0x2a8, 0x247, 0x2a5,
+ 0x247, 0x2a6, 0x044, 0x150, 0x003, 0x0c3, 0x2e0, 0x364,
+ 0x02b, 0x010, 0x009, 0x011, 0x044, 0x38c, 0x002, 0x044,
+ 0x39f, 0x002, 0x347, 0x290, 0x2c6, 0x347, 0x28f, 0x2c5,
+ 0x347, 0x28e, 0x2c4, 0x045, 0x0c0, 0x2ce, 0x0c8, 0x2d5,
+ 0x0cf, 0x2ca, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x0c5, 0x2cc,
+ 0x0cf, 0x2d1, 0x0c0, 0x2d2, 0x0dd, 0x2cd, 0x045, 0x0c0,
+ 0x2d8, 0x044, 0x3b3, 0x002, 0x3c7, 0x038, 0x290, 0x0cc,
+ 0x28f, 0x0c0, 0x28e, 0x0cd, 0x291, 0x044, 0x034, 0x003,
+ 0x020, 0x06e, 0x003, 0x364, 0x02b, 0x010, 0x009, 0x013,
+ 0x0ca, 0x293, 0x0a0, 0x397, 0x294, 0x04d, 0x101, 0x272,
+ 0x2a0, 0x009, 0x3fc, 0x044, 0x06e, 0x003, 0x044, 0x096,
+ 0x003, 0x090, 0x3a7, 0x022, 0x044, 0x1a9, 0x000, 0x347,
+ 0x2aa, 0x2a1, 0x347, 0x2ab, 0x2a2, 0x347, 0x2ac, 0x2a3,
+ 0x347, 0x2ad, 0x2a4, 0x020, 0x3f0, 0x002, 0x044, 0x034,
+ 0x003, 0x020, 0x096, 0x003, 0x3a7, 0x020, 0x084, 0x020,
+ 0x1b7, 0x000, 0x3a7, 0x020, 0x084, 0x020, 0x1a9, 0x000,
+ 0x307, 0x2a1, 0x187, 0x327, 0x2a3, 0x00b, 0x004, 0x141,
+ 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x024, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a2, 0x187, 0x327, 0x2a4, 0x00b,
+ 0x004, 0x141, 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7,
+ 0x026, 0x044, 0x1bd, 0x000, 0x307, 0x2a7, 0x187, 0x327,
+ 0x2a5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x025, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a8, 0x187, 0x327, 0x2a6, 0x3a4,
+ 0x0ff, 0x285, 0x208, 0x3a7, 0x027, 0x020, 0x1bd, 0x000,
+ 0x0c0, 0x28f, 0x000, 0x00d, 0x140, 0x28f, 0x003, 0x3fa,
+ 0x362, 0x28f, 0x00f, 0x00b, 0x004, 0x0cf, 0x28f, 0x140,
+ 0x28e, 0x003, 0x00b, 0x362, 0x28e, 0x00f, 0x00b, 0x008,
+ 0x0cf, 0x28e, 0x000, 0x004, 0x0c0, 0x28e, 0x307, 0x28f,
+ 0x183, 0x305, 0x28e, 0x22a, 0x3a4, 0x3f7, 0x286, 0x327,
+ 0x290, 0x1a7, 0x285, 0x3a7, 0x02b, 0x044, 0x1bd, 0x000,
+ 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x0c2, 0x2fa, 0x045,
+ 0x045, 0x347, 0x2ca, 0x293, 0x000, 0x011, 0x307, 0x2ca,
+ 0x302, 0x293, 0x003, 0x3f7, 0x327, 0x2c9, 0x322, 0x293,
+ 0x003, 0x005, 0x347, 0x2c9, 0x293, 0x307, 0x291, 0x044,
+ 0x106, 0x003, 0x0a1, 0x364, 0x236, 0x100, 0x009, 0x003,
+ 0x0a5, 0x1a3, 0x285, 0x183, 0x305, 0x292, 0x183, 0x305,
+ 0x293, 0x3a7, 0x02c, 0x020, 0x1bd, 0x000, 0x307, 0x296,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x295, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x294, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x028, 0x044,
+ 0x1bd, 0x000, 0x307, 0x299, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x298, 0x044, 0x106, 0x003, 0x040, 0x307, 0x297,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x029, 0x044, 0x1bd, 0x000, 0x307, 0x29f,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x29e, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x29d, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x031, 0x044,
+ 0x1bd, 0x000, 0x307, 0x29c, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x29b, 0x044, 0x106, 0x003, 0x040, 0x307, 0x29a,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x02a, 0x020, 0x1bd, 0x000, 0x100, 0x003,
+ 0x005, 0x22a, 0x286, 0x045, 0x228, 0x121, 0x28a, 0x286,
+ 0x385, 0x010, 0x045, 0x083, 0x182, 0x0a1, 0x182, 0x385,
+ 0x001, 0x181, 0x285, 0x181, 0x385, 0x002, 0x181, 0x285,
+ 0x3a7, 0x030, 0x020, 0x1bd, 0x000, 0x044, 0x15b, 0x003,
+ 0x382, 0x081, 0x240, 0x2a5, 0x240, 0x2a6, 0x044, 0x134,
+ 0x003, 0x020, 0x3f0, 0x002, 0x364, 0x2a5, 0x300, 0x001,
+ 0x007, 0x003, 0x010, 0x3c7, 0x0ff, 0x2a5, 0x364, 0x2a6,
+ 0x300, 0x001, 0x007, 0x003, 0x00a, 0x3c7, 0x0ff, 0x2a6,
+ 0x045, 0x0c0, 0x2a5, 0x000, 0x3f3, 0x0c0, 0x2a6, 0x045,
+ 0x397, 0x2a1, 0x04c, 0x109, 0x04d, 0x118, 0x272, 0x2a9,
+ 0x009, 0x3fa, 0x045, 0x347, 0x2ae, 0x2a5, 0x347, 0x2af,
+ 0x2a6, 0x045, 0x342, 0x2fe, 0x00f, 0x003, 0x03f, 0x045,
+ 0x302, 0x2fe, 0x003, 0x019, 0x042, 0x042, 0x045, 0x327,
+ 0x00e, 0x309, 0x00f, 0x002, 0x005, 0x2a8, 0x208, 0x121,
+ 0x20e, 0x040, 0x041, 0x322, 0x2fd, 0x001, 0x3eb, 0x303,
+ 0x2fe, 0x00b, 0x3eb, 0x042, 0x247, 0x2fd, 0x042, 0x247,
+ 0x2fe, 0x000, 0x02e, 0x0c0, 0x2ff, 0x044, 0x30e, 0x002,
+ 0x364, 0x017, 0x004, 0x009, 0x3dc, 0x340, 0x046, 0x047,
+ 0x001, 0x021, 0x342, 0x2fd, 0x00e, 0x001, 0x3c5, 0x343,
+ 0x2fe, 0x00f, 0x00b, 0x017, 0x347, 0x00e, 0x2fd, 0x347,
+ 0x00f, 0x2fe, 0x009, 0x00d, 0x140, 0x2fd, 0x003, 0x009,
+ 0x342, 0x300, 0x2fd, 0x00b, 0x004, 0x0c0, 0x2fd, 0x0c1,
+ 0x2ff, 0x045, 0x30e, 0x21d, 0x102, 0x000, 0x004, 0x30e,
+ 0x21d, 0x327, 0x2c7, 0x041, 0x327, 0x2c8, 0x041, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x045, 0x366, 0x200, 0x080, 0x009,
+ 0x019, 0x364, 0x026, 0x002, 0x001, 0x014, 0x054, 0x140,
+ 0x2fa, 0x00b, 0x3f3, 0x3c7, 0x3ff, 0x2fa, 0x360, 0x2fa,
+ 0x00a, 0x003, 0x007, 0x054, 0x140, 0x01c, 0x00b, 0x3f8,
+ 0x045, 0x0c0, 0x301, 0x3ce, 0x3ff, 0x2fe, 0x040, 0x044,
+ 0x3f0, 0x002, 0x044, 0x18b, 0x003, 0x042, 0x04c, 0x140,
+ 0x301, 0x001, 0x010, 0x346, 0x301, 0x00f, 0x003, 0x029,
+ 0x320, 0x303, 0x2e4, 0x100, 0x009, 0x02a, 0x04d, 0x000,
+ 0x3e7, 0x0c1, 0x303, 0x347, 0x00f, 0x301, 0x00b, 0x004,
+ 0x152, 0x303, 0x009, 0x004, 0x0c1, 0x301, 0x040, 0x3a7,
+ 0x060, 0x044, 0x192, 0x000, 0x264, 0x008, 0x001, 0x007,
+ 0x348, 0x303, 0x303, 0x141, 0x303, 0x000, 0x3d0, 0x140,
+ 0x2ff, 0x009, 0x005, 0x322, 0x303, 0x04d, 0x045, 0x364,
+ 0x2cf, 0x080, 0x009, 0x019, 0x044, 0x346, 0x002, 0x0a0,
+ 0x397, 0x294, 0x04d, 0x101, 0x272, 0x2a0, 0x009, 0x3fc,
+ 0x044, 0x096, 0x003, 0x0c0, 0x293, 0x044, 0x06e, 0x003,
+ 0x044, 0x2b1, 0x003, 0x140, 0x2d3, 0x009, 0x002, 0x3c4,
+ 0x2fd, 0x00d, 0x044, 0x33f, 0x002, 0x3c4, 0x2ff, 0x200,
+ 0x020, 0x089, 0x000, 0x0cd, 0x2ce, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x307, 0x2e3, 0x140, 0x304, 0x009,
+ 0x004, 0x208, 0x101, 0x240, 0x2a6, 0x240, 0x2a5, 0x044,
+ 0x134, 0x003, 0x044, 0x1ba, 0x003, 0x0c0, 0x309, 0x387,
+ 0x071, 0x140, 0x304, 0x001, 0x003, 0x108, 0x364, 0x309,
+ 0x001, 0x001, 0x004, 0x380, 0x040, 0x187, 0x106, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x397, 0x2a5, 0x300, 0x309, 0x044,
+ 0x1e9, 0x003, 0x141, 0x309, 0x362, 0x309, 0x002, 0x009,
+ 0x3e0, 0x044, 0x3f0, 0x002, 0x347, 0x2a5, 0x2e3, 0x342,
+ 0x2ae, 0x2e3, 0x00b, 0x007, 0x348, 0x2e3, 0x2e3, 0x141,
+ 0x2e3, 0x140, 0x307, 0x009, 0x005, 0x347, 0x2e3, 0x307,
+ 0x045, 0x364, 0x2cf, 0x080, 0x009, 0x0ac, 0x0c1, 0x2ce,
+ 0x055, 0x3c5, 0x004, 0x2cf, 0x3a7, 0x071, 0x044, 0x192,
+ 0x000, 0x247, 0x305, 0x054, 0x080, 0x044, 0x1bd, 0x000,
+ 0x3a7, 0x02c, 0x387, 0x30f, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x022, 0x08a, 0x044, 0x1bd, 0x000, 0x081, 0x189, 0x0a0,
+ 0x044, 0x1a9, 0x000, 0x30e, 0x21d, 0x102, 0x0b0, 0x041,
+ 0x0a0, 0x041, 0x0a2, 0x044, 0x0fb, 0x002, 0x080, 0x0a0,
+ 0x262, 0x001, 0x00b, 0x004, 0x3a7, 0x0ff, 0x040, 0x397,
+ 0x2a1, 0x04d, 0x101, 0x04d, 0x101, 0x04d, 0x101, 0x04d,
+ 0x113, 0x043, 0x280, 0x3a7, 0x080, 0x04d, 0x040, 0x392,
+ 0x2a1, 0x262, 0x005, 0x00b, 0x007, 0x040, 0x044, 0x3e4,
+ 0x002, 0x042, 0x227, 0x3a4, 0x001, 0x121, 0x1a5, 0x191,
+ 0x285, 0x187, 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x042,
+ 0x044, 0x1e9, 0x003, 0x041, 0x101, 0x392, 0x2a1, 0x264,
+ 0x008, 0x001, 0x3c6, 0x390, 0x2a1, 0x111, 0x043, 0x04d,
+ 0x272, 0x2a1, 0x003, 0x3fb, 0x044, 0x150, 0x003, 0x044,
+ 0x3ea, 0x002, 0x044, 0x06e, 0x003, 0x364, 0x292, 0x008,
+ 0x009, 0x008, 0x080, 0x3a7, 0x022, 0x044, 0x1bd, 0x000,
+ 0x307, 0x020, 0x192, 0x002, 0x013, 0x19a, 0x002, 0x010,
+ 0x081, 0x189, 0x0a0, 0x044, 0x1b7, 0x000, 0x055, 0x3a7,
+ 0x071, 0x307, 0x305, 0x044, 0x1bd, 0x000, 0x044, 0x3f0,
+ 0x002, 0x3c4, 0x3fb, 0x2cf, 0x3c5, 0x080, 0x2cf, 0x054,
+ 0x045, 0x397, 0x36a, 0x247, 0x369, 0x0a0, 0x101, 0x04d,
+ 0x272, 0x38e, 0x009, 0x3fc, 0x045, 0x0c3, 0x34e, 0x1cb,
+ 0x34e, 0x3c5, 0x03f, 0x34e, 0x0c3, 0x362, 0x0c4, 0x363,
+ 0x0c0, 0x33c, 0x0c0, 0x356, 0x0c0, 0x357, 0x3d7, 0x2f9,
+ 0x352, 0x1c2, 0x352, 0x3d7, 0x254, 0x353, 0x1c0, 0x353,
+ 0x08d, 0x18b, 0x385, 0x05e, 0x247, 0x354, 0x3c7, 0x0b2,
+ 0x355, 0x345, 0x21d, 0x355, 0x0c4, 0x341, 0x0c2, 0x33f,
+ 0x0d2, 0x340, 0x0d3, 0x342, 0x0ce, 0x344, 0x0de, 0x343,
+ 0x0c0, 0x348, 0x045, 0x055, 0x3c4, 0x008, 0x33b, 0x144,
+ 0x33b, 0x0c6, 0x33d, 0x3c4, 0x3fb, 0x200, 0x307, 0x027,
+ 0x384, 0x324, 0x101, 0x247, 0x027, 0x045, 0x055, 0x3c4,
+ 0x3ec, 0x052, 0x044, 0x05b, 0x001, 0x3c4, 0x3fb, 0x00c,
+ 0x0c0, 0x359, 0x3c4, 0x1ff, 0x35b, 0x045, 0x055, 0x0c0,
+ 0x33d, 0x3c4, 0x3fe, 0x03d, 0x0df, 0x053, 0x3c4, 0x373,
+ 0x00c, 0x0c0, 0x349, 0x0c0, 0x35a, 0x3c4, 0x330, 0x027,
+ 0x0c0, 0x33b, 0x344, 0x21f, 0x33c, 0x045, 0x044, 0x050,
+ 0x005, 0x307, 0x33c, 0x00b, 0x018, 0x362, 0x33d, 0x003,
+ 0x007, 0x013, 0x009, 0x005, 0x044, 0x073, 0x004, 0x055,
+ 0x364, 0x33c, 0x040, 0x001, 0x008, 0x3c4, 0x3fb, 0x200,
+ 0x044, 0x103, 0x004, 0x044, 0x1bb, 0x005, 0x020, 0x0ad,
+ 0x000, 0x364, 0x33c, 0x002, 0x001, 0x055, 0x3c5, 0x008,
+ 0x00c, 0x364, 0x33c, 0x080, 0x009, 0x023, 0x0c0, 0x392,
+ 0x347, 0x354, 0x03e, 0x347, 0x355, 0x03f, 0x347, 0x352,
+ 0x038, 0x347, 0x353, 0x039, 0x307, 0x352, 0x040, 0x307,
+ 0x353, 0x040, 0x307, 0x354, 0x040, 0x307, 0x355, 0x305,
+ 0x21d, 0x040, 0x397, 0x2a0, 0x044, 0x01a, 0x002, 0x0c0,
+ 0x290, 0x044, 0x034, 0x003, 0x044, 0x361, 0x003, 0x0c7,
+ 0x2d1, 0x0c3, 0x2cc, 0x0c0, 0x2c8, 0x0c1, 0x2c7, 0x1cd,
+ 0x2c7, 0x0c1, 0x33d, 0x142, 0x027, 0x0c0, 0x33e, 0x0c0,
+ 0x365, 0x020, 0x04c, 0x004, 0x085, 0x364, 0x33b, 0x020,
+ 0x001, 0x003, 0x102, 0x18a, 0x385, 0x11f, 0x247, 0x053,
+ 0x045, 0x0c2, 0x33b, 0x0c5, 0x33d, 0x3c5, 0x010, 0x027,
+ 0x0df, 0x053, 0x045, 0x044, 0x3b6, 0x003, 0x044, 0x3c6,
+ 0x003, 0x044, 0x38c, 0x002, 0x044, 0x39f, 0x002, 0x3c4,
+ 0x3fb, 0x200, 0x045, 0x055, 0x364, 0x33c, 0x020, 0x009,
+ 0x007, 0x0c4, 0x33d, 0x044, 0x103, 0x004, 0x0c5, 0x33d,
+ 0x3c4, 0x3fe, 0x03d, 0x3c5, 0x00a, 0x33b, 0x307, 0x027,
+ 0x384, 0x334, 0x305, 0x392, 0x247, 0x027, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x377, 0x00c, 0x020, 0x38c, 0x002, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x044, 0x006,
+ 0x005, 0x364, 0x33b, 0x010, 0x3c4, 0x3bf, 0x33c, 0x364,
+ 0x33c, 0x020, 0x009, 0x007, 0x364, 0x33c, 0x002, 0x001,
+ 0x015, 0x3c5, 0x004, 0x00c, 0x364, 0x33c, 0x020, 0x001,
+ 0x005, 0x3c5, 0x080, 0x058, 0x0c0, 0x350, 0x044, 0x0c5,
+ 0x004, 0x3c5, 0x001, 0x052, 0x045, 0x0c0, 0x04c, 0x347,
+ 0x357, 0x04d, 0x347, 0x356, 0x04e, 0x347, 0x357, 0x364,
+ 0x347, 0x356, 0x365, 0x387, 0x067, 0x187, 0x385, 0x080,
+ 0x3c4, 0x04c, 0x052, 0x245, 0x052, 0x309, 0x026, 0x003,
+ 0x00c, 0x0c0, 0x029, 0x3c7, 0x022, 0x029, 0x345, 0x21d,
+ 0x029, 0x000, 0x00f, 0x347, 0x362, 0x02c, 0x399, 0x380,
+ 0x208, 0x244, 0x029, 0x307, 0x363, 0x187, 0x245, 0x029,
+ 0x081, 0x18c, 0x245, 0x053, 0x347, 0x021, 0x35c, 0x3c4,
+ 0x3cf, 0x021, 0x045, 0x055, 0x364, 0x052, 0x001, 0x001,
+ 0x02a, 0x142, 0x052, 0x086, 0x187, 0x385, 0x072, 0x364,
+ 0x33c, 0x020, 0x001, 0x008, 0x044, 0x18d, 0x000, 0x055,
+ 0x000, 0x005, 0x044, 0x189, 0x000, 0x347, 0x35c, 0x021,
+ 0x3c7, 0x080, 0x052, 0x3c4, 0x3fb, 0x00c, 0x364, 0x33c,
+ 0x200, 0x009, 0x008, 0x3c5, 0x020, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x045, 0x055, 0x140, 0x33d, 0x001, 0x3fc, 0x055,
+ 0x307, 0x368, 0x304, 0x34e, 0x0c0, 0x351, 0x227, 0x1a2,
+ 0x002, 0x071, 0x00b, 0x08b, 0x044, 0x03b, 0x005, 0x044,
+ 0x006, 0x005, 0x141, 0x374, 0x140, 0x346, 0x001, 0x00b,
+ 0x342, 0x342, 0x346, 0x00b, 0x019, 0x0c4, 0x350, 0x000,
+ 0x019, 0x340, 0x347, 0x345, 0x001, 0x00a, 0x342, 0x347,
+ 0x345, 0x364, 0x33c, 0x004, 0x001, 0x3ec, 0x0cc, 0x350,
+ 0x141, 0x384, 0x000, 0x009, 0x0c8, 0x350, 0x141, 0x383,
+ 0x340, 0x342, 0x346, 0x140, 0x345, 0x001, 0x00c, 0x342,
+ 0x344, 0x345, 0x00b, 0x00e, 0x3c5, 0x001, 0x350, 0x000,
+ 0x00e, 0x3c5, 0x003, 0x350, 0x141, 0x37c, 0x000, 0x00a,
+ 0x3c5, 0x002, 0x350, 0x141, 0x37b, 0x340, 0x344, 0x345,
+ 0x140, 0x347, 0x001, 0x00c, 0x342, 0x343, 0x347, 0x00b,
+ 0x00f, 0x3c5, 0x010, 0x350, 0x000, 0x00f, 0x3c5, 0x030,
+ 0x350, 0x141, 0x38c, 0x020, 0x3fa, 0x004, 0x3c5, 0x020,
+ 0x350, 0x141, 0x38b, 0x340, 0x343, 0x347, 0x020, 0x3fa,
+ 0x004, 0x044, 0x03b, 0x005, 0x141, 0x375, 0x0c0, 0x346,
+ 0x0c0, 0x347, 0x0c0, 0x345, 0x044, 0x01e, 0x005, 0x3c7,
+ 0x03f, 0x350, 0x141, 0x37c, 0x141, 0x384, 0x141, 0x38c,
+ 0x0c0, 0x348, 0x020, 0x3fa, 0x004, 0x205, 0x001, 0x182,
+ 0x044, 0x03b, 0x005, 0x227, 0x3a4, 0x00c, 0x1b1, 0x001,
+ 0x013, 0x364, 0x350, 0x00c, 0x009, 0x05c, 0x364, 0x04d,
+ 0x00c, 0x009, 0x057, 0x132, 0x003, 0x00b, 0x001, 0x027,
+ 0x000, 0x050, 0x3c4, 0x3f3, 0x350, 0x000, 0x04b, 0x140,
+ 0x351, 0x009, 0x004, 0x141, 0x37f, 0x140, 0x346, 0x001,
+ 0x00b, 0x151, 0x348, 0x151, 0x346, 0x3c5, 0x004, 0x350,
+ 0x000, 0x038, 0x3c5, 0x00c, 0x350, 0x140, 0x351, 0x001,
+ 0x031, 0x141, 0x384, 0x000, 0x02d, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x380, 0x342, 0x23e, 0x348, 0x001, 0x016,
+ 0x342, 0x342, 0x346, 0x001, 0x00e, 0x141, 0x348, 0x141,
+ 0x346, 0x340, 0x342, 0x346, 0x3c5, 0x004, 0x350, 0x000,
+ 0x00e, 0x340, 0x342, 0x346, 0x3c5, 0x008, 0x350, 0x140,
+ 0x351, 0x001, 0x004, 0x141, 0x383, 0x340, 0x23e, 0x348,
+ 0x227, 0x3a4, 0x003, 0x001, 0x013, 0x364, 0x350, 0x003,
+ 0x009, 0x071, 0x364, 0x04d, 0x003, 0x009, 0x06c, 0x132,
+ 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065, 0x3c4, 0x3fc,
+ 0x350, 0x000, 0x060, 0x140, 0x351, 0x009, 0x004, 0x141,
+ 0x377, 0x140, 0x345, 0x009, 0x00d, 0x3c5, 0x003, 0x350,
+ 0x140, 0x351, 0x001, 0x04f, 0x141, 0x37c, 0x000, 0x04b,
+ 0x342, 0x342, 0x346, 0x009, 0x007, 0x340, 0x342, 0x346,
+ 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151, 0x348, 0x141,
+ 0x346, 0x152, 0x345, 0x3c5, 0x001, 0x350, 0x000, 0x033,
+ 0x140, 0x351, 0x009, 0x004, 0x141, 0x378, 0x342, 0x23e,
+ 0x348, 0x001, 0x01c, 0x342, 0x344, 0x345, 0x001, 0x014,
+ 0x140, 0x346, 0x001, 0x010, 0x141, 0x348, 0x151, 0x346,
+ 0x142, 0x345, 0x340, 0x344, 0x345, 0x3c5, 0x001, 0x350,
+ 0x000, 0x00e, 0x340, 0x344, 0x345, 0x3c5, 0x002, 0x350,
+ 0x140, 0x351, 0x001, 0x004, 0x141, 0x37b, 0x340, 0x23e,
+ 0x348, 0x227, 0x1b3, 0x3a4, 0x003, 0x001, 0x013, 0x364,
+ 0x350, 0x030, 0x009, 0x071, 0x364, 0x04d, 0x030, 0x009,
+ 0x06c, 0x132, 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065,
+ 0x3c4, 0x3cf, 0x350, 0x000, 0x060, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x387, 0x140, 0x347, 0x009, 0x00d, 0x3c5,
+ 0x030, 0x350, 0x140, 0x351, 0x001, 0x04f, 0x141, 0x38c,
+ 0x000, 0x04b, 0x342, 0x342, 0x346, 0x009, 0x007, 0x340,
+ 0x342, 0x346, 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151,
+ 0x348, 0x141, 0x346, 0x152, 0x347, 0x3c5, 0x010, 0x350,
+ 0x000, 0x033, 0x140, 0x351, 0x009, 0x004, 0x141, 0x388,
+ 0x342, 0x23e, 0x348, 0x001, 0x01c, 0x342, 0x343, 0x347,
+ 0x001, 0x014, 0x140, 0x346, 0x001, 0x010, 0x141, 0x348,
+ 0x151, 0x346, 0x142, 0x347, 0x340, 0x343, 0x347, 0x3c5,
+ 0x010, 0x350, 0x000, 0x00e, 0x340, 0x343, 0x347, 0x3c5,
+ 0x020, 0x350, 0x140, 0x351, 0x001, 0x004, 0x141, 0x38b,
+ 0x340, 0x23e, 0x348, 0x328, 0x04d, 0x324, 0x350, 0x2e4,
+ 0x02a, 0x001, 0x026, 0x140, 0x351, 0x009, 0x022, 0x141,
+ 0x351, 0x2e4, 0x020, 0x001, 0x005, 0x3c4, 0x3cf, 0x350,
+ 0x2e4, 0x008, 0x001, 0x005, 0x3c4, 0x3f3, 0x350, 0x2e4,
+ 0x002, 0x001, 0x28a, 0x3c4, 0x3fc, 0x350, 0x000, 0x285,
+ 0x0c0, 0x350, 0x3c4, 0x3bf, 0x027, 0x000, 0x0a8, 0x328,
+ 0x04d, 0x324, 0x350, 0x287, 0x3a4, 0x003, 0x131, 0x009,
+ 0x029, 0x140, 0x345, 0x001, 0x01c, 0x342, 0x344, 0x345,
+ 0x044, 0x030, 0x005, 0x003, 0x006, 0x140, 0x345, 0x009,
+ 0x00b, 0x364, 0x368, 0x002, 0x001, 0x006, 0x141, 0x350,
+ 0x141, 0x37b, 0x340, 0x344, 0x345, 0x000, 0x00b, 0x364,
+ 0x368, 0x001, 0x001, 0x006, 0x142, 0x350, 0x141, 0x37c,
+ 0x227, 0x3a4, 0x00c, 0x134, 0x009, 0x039, 0x140, 0x346,
+ 0x001, 0x01c, 0x342, 0x342, 0x346, 0x044, 0x030, 0x005,
+ 0x003, 0x006, 0x140, 0x346, 0x009, 0x00b, 0x364, 0x368,
+ 0x008, 0x001, 0x006, 0x144, 0x350, 0x141, 0x383, 0x340,
+ 0x342, 0x346, 0x000, 0x01b, 0x364, 0x368, 0x008, 0x009,
+ 0x3e3, 0x364, 0x368, 0x004, 0x001, 0x011, 0x327, 0x345,
+ 0x320, 0x347, 0x001, 0x007, 0x364, 0x33c, 0x004, 0x001,
+ 0x006, 0x148, 0x350, 0x141, 0x384, 0x227, 0x3a4, 0x030,
+ 0x3a2, 0x010, 0x009, 0x02b, 0x140, 0x347, 0x001, 0x01d,
+ 0x342, 0x343, 0x347, 0x044, 0x030, 0x005, 0x003, 0x006,
+ 0x140, 0x347, 0x009, 0x00c, 0x364, 0x368, 0x020, 0x001,
+ 0x007, 0x3c0, 0x010, 0x350, 0x141, 0x38b, 0x340, 0x343,
+ 0x347, 0x000, 0x00c, 0x364, 0x368, 0x010, 0x001, 0x007,
+ 0x3c0, 0x020, 0x350, 0x141, 0x38c, 0x044, 0x3fa, 0x004,
+ 0x000, 0x026, 0x3c5, 0x040, 0x027, 0x307, 0x04d, 0x384,
+ 0x200, 0x305, 0x350, 0x247, 0x04d, 0x045, 0x08f, 0x3b7,
+ 0x211, 0x044, 0x1b8, 0x000, 0x347, 0x33f, 0x346, 0x347,
+ 0x340, 0x347, 0x347, 0x341, 0x345, 0x347, 0x347, 0x348,
+ 0x340, 0x345, 0x348, 0x340, 0x346, 0x348, 0x307, 0x347,
+ 0x183, 0x305, 0x345, 0x184, 0x305, 0x346, 0x32a, 0x346,
+ 0x3a4, 0x008, 0x285, 0x3b7, 0x210, 0x020, 0x1bd, 0x000,
+ 0x364, 0x33c, 0x008, 0x001, 0x007, 0x327, 0x23e, 0x322,
+ 0x348, 0x131, 0x045, 0x040, 0x227, 0x306, 0x373, 0x001,
+ 0x00f, 0x307, 0x369, 0x101, 0x272, 0x372, 0x009, 0x003,
+ 0x118, 0x04d, 0x247, 0x369, 0x2c7, 0x373, 0x042, 0x045,
+ 0x140, 0x35a, 0x00b, 0x0b0, 0x344, 0x21f, 0x35a, 0x0a3,
+ 0x1ad, 0x2a8, 0x2c4, 0x027, 0x055, 0x044, 0x340, 0x001,
+ 0x054, 0x264, 0x002, 0x001, 0x005, 0x0c0, 0x33d, 0x045,
+ 0x083, 0x18b, 0x304, 0x050, 0x305, 0x221, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3f4, 0x308, 0x221, 0x244, 0x027, 0x387,
+ 0x030, 0x304, 0x050, 0x187, 0x044, 0x0d3, 0x005, 0x001,
+ 0x3e6, 0x08c, 0x304, 0x050, 0x189, 0x044, 0x0d3, 0x005,
+ 0x001, 0x3dd, 0x083, 0x304, 0x050, 0x18b, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3d4, 0x083, 0x18b, 0x304, 0x365, 0x044,
+ 0x0d3, 0x005, 0x001, 0x3cb, 0x387, 0x030, 0x304, 0x365,
+ 0x187, 0x044, 0x0d3, 0x005, 0x001, 0x3c1, 0x08c, 0x304,
+ 0x365, 0x189, 0x044, 0x0d3, 0x005, 0x001, 0x3b8, 0x083,
+ 0x304, 0x365, 0x18b, 0x044, 0x0d3, 0x005, 0x001, 0x3af,
+ 0x307, 0x21d, 0x304, 0x364, 0x192, 0x364, 0x052, 0x020,
+ 0x001, 0x00d, 0x327, 0x21d, 0x324, 0x04f, 0x001, 0x005,
+ 0x345, 0x21d, 0x33c, 0x1b1, 0x285, 0x044, 0x0d3, 0x005,
+ 0x001, 0x395, 0x045, 0x364, 0x33b, 0x001, 0x001, 0x02c,
+ 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x027, 0x245, 0x027, 0x345,
+ 0x21d, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x01d, 0x044,
+ 0x103, 0x005, 0x364, 0x35b, 0x002, 0x001, 0x3f5, 0x344,
+ 0x21f, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x00d, 0x044,
+ 0x103, 0x005, 0x307, 0x35b, 0x264, 0x002, 0x009, 0x3f4,
+ 0x264, 0x001, 0x045, 0x055, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x044, 0x340, 0x001, 0x247, 0x35b, 0x374, 0x027,
+ 0x200, 0x001, 0x00a, 0x264, 0x080, 0x009, 0x006, 0x3b8,
+ 0x200, 0x2c4, 0x027, 0x054, 0x045, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x001, 0x088, 0x3b8, 0x300, 0x2c4, 0x027,
+ 0x264, 0x040, 0x001, 0x00f, 0x140, 0x359, 0x001, 0x00b,
+ 0x0c0, 0x33d, 0x0c0, 0x359, 0x3d5, 0x200, 0x027, 0x000,
+ 0x060, 0x384, 0x030, 0x327, 0x359, 0x2e2, 0x008, 0x003,
+ 0x3f1, 0x2a9, 0x3b0, 0x15a, 0x041, 0x1b9, 0x3b0, 0x005,
+ 0x041, 0x045, 0x000, 0x012, 0x000, 0x015, 0x000, 0x015,
+ 0x000, 0x016, 0x000, 0x017, 0x000, 0x01a, 0x000, 0x01c,
+ 0x000, 0x01f, 0x000, 0x022, 0x187, 0x247, 0x367, 0x000,
+ 0x033, 0x000, 0x02f, 0x191, 0x000, 0x02c, 0x193, 0x000,
+ 0x029, 0x187, 0x247, 0x366, 0x000, 0x026, 0x245, 0x366,
+ 0x000, 0x022, 0x191, 0x245, 0x366, 0x000, 0x01d, 0x193,
+ 0x245, 0x366, 0x000, 0x018, 0x229, 0x3a4, 0x040, 0x2c5,
+ 0x33c, 0x329, 0x04d, 0x18b, 0x2cc, 0x04d, 0x347, 0x367,
+ 0x368, 0x347, 0x366, 0x04e, 0x0c0, 0x359, 0x000, 0x006,
+ 0x245, 0x367, 0x141, 0x359, 0x3d5, 0x300, 0x027, 0x140,
+ 0x359, 0x009, 0x011, 0x140, 0x04d, 0x00b, 0x00a, 0x364,
+ 0x33c, 0x040, 0x001, 0x005, 0x020, 0x103, 0x004, 0x044,
+ 0x137, 0x004, 0x045, 0x362, 0x33e, 0x007, 0x009, 0x003,
+ 0x045, 0x055, 0x362, 0x33d, 0x000, 0x006, 0x077, 0x362,
+ 0x33d, 0x005, 0x00f, 0x072, 0x140, 0x34d, 0x009, 0x007,
+ 0x044, 0x23e, 0x005, 0x000, 0x069, 0x308, 0x356, 0x304,
+ 0x365, 0x001, 0x007, 0x044, 0x36a, 0x005, 0x000, 0x05e,
+ 0x140, 0x349, 0x001, 0x017, 0x364, 0x04f, 0x03f, 0x009,
+ 0x055, 0x0c0, 0x349, 0x362, 0x34f, 0x000, 0x006, 0x04e,
+ 0x307, 0x34f, 0x044, 0x000, 0x006, 0x0c0, 0x34f, 0x000,
+ 0x045, 0x140, 0x33e, 0x009, 0x007, 0x044, 0x23e, 0x005,
+ 0x000, 0x03c, 0x362, 0x33e, 0x001, 0x009, 0x007, 0x044,
+ 0x269, 0x005, 0x000, 0x032, 0x362, 0x33e, 0x002, 0x009,
+ 0x007, 0x044, 0x26f, 0x005, 0x000, 0x028, 0x362, 0x33e,
+ 0x003, 0x009, 0x007, 0x044, 0x27f, 0x005, 0x000, 0x01e,
+ 0x362, 0x33e, 0x004, 0x009, 0x007, 0x044, 0x29a, 0x005,
+ 0x000, 0x014, 0x362, 0x33e, 0x005, 0x009, 0x007, 0x044,
+ 0x2b7, 0x005, 0x000, 0x00a, 0x362, 0x33e, 0x006, 0x009,
+ 0x005, 0x044, 0x34f, 0x005, 0x054, 0x045, 0x081, 0x189,
+ 0x304, 0x053, 0x009, 0x00c, 0x0c1, 0x33d, 0x083, 0x189,
+ 0x395, 0x3f7, 0x244, 0x027, 0x000, 0x01c, 0x0c2, 0x33d,
+ 0x0c1, 0x34d, 0x3c5, 0x008, 0x027, 0x0c0, 0x34b, 0x0c0,
+ 0x34c, 0x140, 0x35d, 0x009, 0x00b, 0x0c0, 0x34f, 0x088,
+ 0x189, 0x245, 0x365, 0x044, 0x3c2, 0x005, 0x0c1, 0x33e,
+ 0x045, 0x347, 0x35e, 0x34a, 0x0c2, 0x33e, 0x045, 0x307,
+ 0x34b, 0x302, 0x360, 0x006, 0x009, 0x141, 0x34b, 0x044,
+ 0x3f3, 0x005, 0x000, 0x004, 0x0c3, 0x33e, 0x045, 0x307,
+ 0x34c, 0x302, 0x361, 0x006, 0x014, 0x141, 0x34c, 0x088,
+ 0x3c7, 0x020, 0x34f, 0x3c7, 0x028, 0x372, 0x141, 0x38a,
+ 0x141, 0x382, 0x044, 0x000, 0x006, 0x000, 0x004, 0x0c4,
+ 0x33e, 0x045, 0x0c5, 0x33e, 0x0c0, 0x2d3, 0x364, 0x02b,
+ 0x080, 0x009, 0x015, 0x3c5, 0x010, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x362, 0x201, 0x002, 0x001, 0x00a, 0x0c1, 0x244,
+ 0x3c5, 0x004, 0x245, 0x3c4, 0x3fb, 0x200, 0x045, 0x364,
+ 0x2cf, 0x010, 0x009, 0x094, 0x364, 0x34a, 0x001, 0x001,
+ 0x009, 0x0c6, 0x33e, 0x044, 0x33f, 0x002, 0x000, 0x088,
+ 0x140, 0x259, 0x009, 0x009, 0x044, 0x346, 0x002, 0x0c0,
+ 0x330, 0x0c0, 0x331, 0x044, 0x012, 0x006, 0x140, 0x330,
+ 0x009, 0x00c, 0x140, 0x331, 0x009, 0x008, 0x347, 0x32e,
+ 0x330, 0x347, 0x32f, 0x331, 0x362, 0x331, 0x000, 0x00f,
+ 0x00c, 0x362, 0x32f, 0x000, 0x00f, 0x007, 0x044, 0x3f3,
+ 0x005, 0x000, 0x057, 0x362, 0x331, 0x000, 0x007, 0x00c,
+ 0x362, 0x32f, 0x000, 0x007, 0x007, 0x044, 0x3e6, 0x005,
+ 0x000, 0x048, 0x044, 0x33f, 0x002, 0x0c6, 0x33e, 0x362,
+ 0x331, 0x000, 0x00f, 0x021, 0x308, 0x330, 0x348, 0x331,
+ 0x336, 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x330, 0x347,
+ 0x336, 0x331, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302,
+ 0x32e, 0x343, 0x32f, 0x336, 0x002, 0x024, 0x044, 0x3e6,
+ 0x005, 0x000, 0x01f, 0x308, 0x32e, 0x348, 0x32f, 0x336,
+ 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x32e, 0x347, 0x336,
+ 0x32f, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302, 0x32e,
+ 0x343, 0x32f, 0x336, 0x002, 0x005, 0x044, 0x3f3, 0x005,
+ 0x347, 0x32e, 0x330, 0x347, 0x32f, 0x331, 0x045, 0x3c5,
+ 0x008, 0x33b, 0x0c3, 0x33d, 0x3c5, 0x004, 0x027, 0x0c7,
+ 0x33e, 0x364, 0x33c, 0x020, 0x001, 0x00a, 0x345, 0x21d,
+ 0x364, 0x347, 0x21d, 0x35a, 0x000, 0x005, 0x345, 0x21d,
+ 0x04d, 0x045, 0x364, 0x04f, 0x03f, 0x001, 0x054, 0x307,
+ 0x04f, 0x384, 0x003, 0x113, 0x009, 0x007, 0x3c5, 0x001,
+ 0x34a, 0x141, 0x37e, 0x307, 0x04f, 0x384, 0x003, 0x112,
+ 0x009, 0x007, 0x3c5, 0x001, 0x34a, 0x141, 0x37d, 0x307,
+ 0x04f, 0x384, 0x00c, 0x11c, 0x009, 0x007, 0x3c5, 0x002,
+ 0x34a, 0x141, 0x386, 0x307, 0x04f, 0x384, 0x00c, 0x118,
+ 0x009, 0x007, 0x3c5, 0x002, 0x34a, 0x141, 0x385, 0x307,
+ 0x04f, 0x384, 0x030, 0x382, 0x030, 0x009, 0x007, 0x3c5,
+ 0x004, 0x34a, 0x141, 0x38e, 0x307, 0x04f, 0x384, 0x030,
+ 0x382, 0x020, 0x009, 0x007, 0x3c5, 0x004, 0x34a, 0x141,
+ 0x38d, 0x347, 0x356, 0x365, 0x0c1, 0x349, 0x044, 0x3c2,
+ 0x005, 0x045, 0x088, 0x189, 0x304, 0x365, 0x001, 0x004,
+ 0x141, 0x376, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x347,
+ 0x21d, 0x35a, 0x000, 0x013, 0x347, 0x365, 0x04e, 0x307,
+ 0x04e, 0x306, 0x356, 0x001, 0x007, 0x3c5, 0x080, 0x027,
+ 0x000, 0x005, 0x3c4, 0x37f, 0x027, 0x045, 0x081, 0x0c4,
+ 0x34f, 0x0c5, 0x372, 0x141, 0x379, 0x141, 0x381, 0x044,
+ 0x000, 0x006, 0x045, 0x088, 0x0c2, 0x34f, 0x0ca, 0x372,
+ 0x141, 0x37a, 0x141, 0x382, 0x044, 0x000, 0x006, 0x045,
+ 0x364, 0x33c, 0x010, 0x001, 0x009, 0x22a, 0x3a4, 0x015,
+ 0x209, 0x384, 0x02a, 0x285, 0x245, 0x365, 0x044, 0x3c2,
+ 0x005, 0x045, 0x0c2, 0x2ce, 0x0c0, 0x304, 0x044, 0x25b,
+ 0x003, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x086, 0x189, 0x395,
+ 0x104, 0x0a2, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003,
+ 0x3c5, 0x100, 0x00d, 0x044, 0x30e, 0x002, 0x347, 0x00e,
+ 0x32e, 0x347, 0x00f, 0x32f, 0x045, 0x307, 0x000, 0x183,
+ 0x247, 0x21b, 0x003, 0x00b, 0x347, 0x21e, 0x008, 0x307,
+ 0x006, 0x327, 0x007, 0x000, 0x009, 0x347, 0x21e, 0x00b,
+ 0x307, 0x009, 0x327, 0x00a, 0x020, 0x1bb, 0x006, 0x347,
+ 0x059, 0x24b, 0x347, 0x248, 0x24a, 0x044, 0x340, 0x001,
+ 0x247, 0x248, 0x054, 0x364, 0x001, 0x080, 0x009, 0x059,
+ 0x347, 0x24a, 0x248, 0x364, 0x001, 0x001, 0x009, 0x171,
+ 0x364, 0x001, 0x002, 0x009, 0x171, 0x364, 0x001, 0x004,
+ 0x009, 0x17c, 0x364, 0x001, 0x008, 0x009, 0x1a4, 0x364,
+ 0x001, 0x010, 0x029, 0x042, 0x008, 0x364, 0x001, 0x020,
+ 0x029, 0x01f, 0x008, 0x364, 0x001, 0x040, 0x009, 0x1b1,
+ 0x364, 0x001, 0x100, 0x009, 0x1ac, 0x327, 0x21c, 0x324,
+ 0x020, 0x029, 0x0ab, 0x006, 0x364, 0x23f, 0x003, 0x001,
+ 0x00c, 0x044, 0x340, 0x001, 0x227, 0x1ba, 0x326, 0x248,
+ 0x054, 0x009, 0x026, 0x307, 0x24b, 0x306, 0x059, 0x384,
+ 0x080, 0x009, 0x01b, 0x055, 0x142, 0x000, 0x047, 0x3c5,
+ 0x020, 0x000, 0x3c5, 0x001, 0x233, 0x000, 0x012, 0x364,
+ 0x233, 0x001, 0x001, 0x00a, 0x3c5, 0x020, 0x000, 0x347,
+ 0x24a, 0x248, 0x000, 0x3e9, 0x044, 0x340, 0x001, 0x247,
+ 0x248, 0x347, 0x059, 0x24b, 0x044, 0x34c, 0x001, 0x009,
+ 0x012, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x364, 0x24b,
+ 0x080, 0x009, 0x008, 0x306, 0x249, 0x19a, 0x001, 0x3cd,
+ 0x054, 0x364, 0x23f, 0x002, 0x001, 0x052, 0x307, 0x248,
+ 0x19e, 0x00a, 0x01c, 0x3c5, 0x084, 0x021, 0x3c4, 0x3fe,
+ 0x026, 0x307, 0x248, 0x306, 0x249, 0x209, 0x00b, 0x022,
+ 0x3b7, 0x210, 0x044, 0x192, 0x000, 0x247, 0x257, 0x080,
+ 0x044, 0x1bd, 0x000, 0x000, 0x015, 0x364, 0x026, 0x001,
+ 0x009, 0x010, 0x3c4, 0x37b, 0x021, 0x364, 0x234, 0x001,
+ 0x001, 0x008, 0x3c5, 0x001, 0x026, 0x044, 0x342, 0x006,
+ 0x307, 0x248, 0x19d, 0x00a, 0x007, 0x3c4, 0x3fd, 0x026,
+ 0x000, 0x00d, 0x364, 0x234, 0x002, 0x001, 0x008, 0x3c5,
+ 0x002, 0x026, 0x044, 0x141, 0x001, 0x327, 0x248, 0x326,
+ 0x249, 0x00b, 0x005, 0x044, 0x27d, 0x001, 0x364, 0x23f,
+ 0x001, 0x001, 0x013, 0x044, 0x34c, 0x001, 0x001, 0x007,
+ 0x2a8, 0x2c4, 0x233, 0x000, 0x00b, 0x327, 0x248, 0x326,
+ 0x249, 0x1ba, 0x3a4, 0x003, 0x001, 0x01f, 0x327, 0x248,
+ 0x1ba, 0x287, 0x190, 0x002, 0x006, 0x3a4, 0x3fb, 0x000,
+ 0x004, 0x3a5, 0x004, 0x3a4, 0x007, 0x307, 0x234, 0x384,
+ 0x3f8, 0x225, 0x344, 0x21f, 0x232, 0x347, 0x248, 0x249,
+ 0x020, 0x022, 0x008, 0x347, 0x248, 0x249, 0x364, 0x33c,
+ 0x020, 0x001, 0x005, 0x044, 0x12d, 0x005, 0x364, 0x233,
+ 0x001, 0x001, 0x314, 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4,
+ 0x000, 0x328, 0x21b, 0x044, 0x32d, 0x001, 0x043, 0x307,
+ 0x21b, 0x009, 0x00e, 0x2c7, 0x005, 0x144, 0x000, 0x364,
+ 0x233, 0x001, 0x001, 0x2fb, 0x020, 0x0a3, 0x006, 0x00b,
+ 0x009, 0x0c0, 0x21b, 0x2c7, 0x00b, 0x148, 0x000, 0x047,
+ 0x0c0, 0x21b, 0x2c7, 0x008, 0x141, 0x000, 0x047, 0x3c4,
+ 0x3df, 0x000, 0x364, 0x000, 0x100, 0x001, 0x2a2, 0x364,
+ 0x233, 0x001, 0x001, 0x005, 0x3c5, 0x020, 0x000, 0x307,
+ 0x003, 0x327, 0x004, 0x041, 0x044, 0x32c, 0x001, 0x043,
+ 0x044, 0x32d, 0x001, 0x040, 0x209, 0x002, 0x083, 0x003,
+ 0x089, 0x188, 0x199, 0x041, 0x229, 0x280, 0x043, 0x390,
+ 0x057, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x3c4,
+ 0x3f3, 0x012, 0x000, 0x2b3, 0x0af, 0x1a7, 0x2a8, 0x2c4,
+ 0x01f, 0x3c5, 0x001, 0x200, 0x344, 0x21f, 0x232, 0x151,
+ 0x2fa, 0x020, 0x247, 0x002, 0x364, 0x03a, 0x040, 0x009,
+ 0x007, 0x3c4, 0x3f7, 0x03a, 0x000, 0x299, 0x3c4, 0x3f0,
+ 0x03a, 0x364, 0x2cf, 0x001, 0x001, 0x01b, 0x364, 0x2cf,
+ 0x002, 0x009, 0x016, 0x142, 0x03a, 0x141, 0x2e6, 0x3c1,
+ 0x000, 0x2e7, 0x32a, 0x03a, 0x2f4, 0x200, 0x001, 0x3fc,
+ 0x143, 0x03a, 0x374, 0x25e, 0x200, 0x009, 0x002, 0x000,
+ 0x276, 0x364, 0x03d, 0x040, 0x009, 0x007, 0x3c4, 0x3f7,
+ 0x03d, 0x000, 0x26c, 0x3c4, 0x3f0, 0x03d, 0x364, 0x33b,
+ 0x001, 0x001, 0x007, 0x044, 0x3a3, 0x003, 0x000, 0x25f,
+ 0x374, 0x25f, 0x200, 0x009, 0x002, 0x000, 0x258, 0x158,
+ 0x025, 0x044, 0x141, 0x001, 0x020, 0x0a3, 0x006, 0x042,
+ 0x3a7, 0x020, 0x1a7, 0x3a5, 0x055, 0x020, 0x1de, 0x007,
+ 0x042, 0x394, 0x3ff, 0x04d, 0x040, 0x020, 0x181, 0x006,
+ 0x042, 0x394, 0x3ff, 0x04c, 0x020, 0x1de, 0x007, 0x364,
+ 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x042, 0x186, 0x003,
+ 0x06b, 0x002, 0x06d, 0x2e4, 0x001, 0x001, 0x059, 0x287,
+ 0x384, 0x00e, 0x190, 0x3a4, 0x370, 0x285, 0x041, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x043, 0x1b3, 0x3c4, 0x3fd, 0x00c,
+ 0x3c4, 0x3fc, 0x018, 0x287, 0x384, 0x070, 0x382, 0x030,
+ 0x001, 0x007, 0x3a4, 0x007, 0x133, 0x009, 0x007, 0x3c4,
+ 0x3ef, 0x021, 0x141, 0x018, 0x142, 0x00c, 0x0a1, 0x1aa,
+ 0x364, 0x026, 0x004, 0x009, 0x004, 0x2c5, 0x02a, 0x2a8,
+ 0x082, 0x044, 0x189, 0x000, 0x2c4, 0x02a, 0x3c4, 0x3f7,
+ 0x00d, 0x3c5, 0x001, 0x00d, 0x3c4, 0x3fe, 0x00d, 0x140,
+ 0x013, 0x009, 0x00b, 0x140, 0x014, 0x009, 0x007, 0x3c5,
+ 0x008, 0x00d, 0x000, 0x014, 0x0c0, 0x012, 0x0c2, 0x012,
+ 0x0c0, 0x012, 0x0c5, 0x012, 0x000, 0x00a, 0x0c0, 0x00d,
+ 0x044, 0x33f, 0x002, 0x3c4, 0x3fd, 0x00c, 0x0a3, 0x020,
+ 0x1de, 0x007, 0x2c7, 0x013, 0x000, 0x3fa, 0x2c7, 0x014,
+ 0x000, 0x3f6, 0x020, 0x181, 0x006, 0x2e4, 0x100, 0x009,
+ 0x042, 0x041, 0x3a4, 0x007, 0x387, 0x006, 0x189, 0x395,
+ 0x321, 0x280, 0x050, 0x0c3, 0x265, 0x262, 0x005, 0x00b,
+ 0x004, 0x0c7, 0x265, 0x043, 0x1b3, 0x2e2, 0x003, 0x009,
+ 0x00b, 0x041, 0x040, 0x084, 0x0b1, 0x044, 0x1b7, 0x000,
+ 0x042, 0x043, 0x2a9, 0x349, 0x265, 0x265, 0x111, 0x00b,
+ 0x3fb, 0x2aa, 0x34a, 0x265, 0x265, 0x348, 0x265, 0x265,
+ 0x041, 0x3a7, 0x030, 0x044, 0x192, 0x000, 0x043, 0x304,
+ 0x265, 0x285, 0x3a7, 0x030, 0x044, 0x1bd, 0x000, 0x000,
+ 0x007, 0x084, 0x0b1, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x000, 0x006, 0x009, 0x002, 0x004, 0x2c7, 0x22c,
+ 0x2f4, 0x100, 0x001, 0x013, 0x044, 0x342, 0x006, 0x364,
+ 0x23f, 0x002, 0x009, 0x005, 0x020, 0x181, 0x006, 0x345,
+ 0x21d, 0x232, 0x020, 0x2ea, 0x002, 0x044, 0x1ef, 0x001,
+ 0x000, 0x3ef, 0x044, 0x1ef, 0x001, 0x091, 0x044, 0x189,
+ 0x000, 0x3b7, 0x210, 0x307, 0x257, 0x020, 0x1bd, 0x000,
+ 0x1ad, 0x1b9, 0x3c4, 0x3cf, 0x232, 0x2c5, 0x232, 0x020,
+ 0x181, 0x006, 0x287, 0x2e4, 0x100, 0x001, 0x016, 0x041,
+ 0x384, 0x001, 0x182, 0x040, 0x3b7, 0x211, 0x044, 0x1a9,
+ 0x000, 0x042, 0x385, 0x3f3, 0x3b7, 0x211, 0x044, 0x1b8,
+ 0x000, 0x043, 0x287, 0x2f4, 0x200, 0x001, 0x016, 0x384,
+ 0x010, 0x190, 0x227, 0x190, 0x285, 0x040, 0x3a7, 0x060,
+ 0x044, 0x1a9, 0x000, 0x042, 0x385, 0x3f3, 0x3a7, 0x060,
+ 0x044, 0x1b8, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x100,
+ 0x009, 0x059, 0x287, 0x384, 0x0ff, 0x264, 0x080, 0x001,
+ 0x004, 0x385, 0x300, 0x1bd, 0x3a4, 0x003, 0x009, 0x004,
+ 0x247, 0x23a, 0x131, 0x009, 0x006, 0x247, 0x23c, 0x003,
+ 0x036, 0x131, 0x009, 0x004, 0x247, 0x23b, 0x131, 0x009,
+ 0x00a, 0x247, 0x23a, 0x003, 0x02a, 0x247, 0x23c, 0x247,
+ 0x23b, 0x327, 0x23a, 0x00b, 0x004, 0x2a8, 0x121, 0x140,
+ 0x23b, 0x00b, 0x006, 0x322, 0x23b, 0x000, 0x004, 0x320,
+ 0x23b, 0x320, 0x23c, 0x322, 0x23e, 0x007, 0x010, 0x347,
+ 0x23a, 0x237, 0x347, 0x23b, 0x239, 0x347, 0x23c, 0x238,
+ 0x044, 0x05b, 0x001, 0x000, 0x038, 0x347, 0x237, 0x23a,
+ 0x347, 0x239, 0x23b, 0x347, 0x238, 0x23c, 0x020, 0x1dc,
+ 0x007, 0x140, 0x33b, 0x009, 0x015, 0x289, 0x002, 0x00c,
+ 0x003, 0x006, 0x327, 0x237, 0x000, 0x008, 0x327, 0x238,
+ 0x000, 0x004, 0x327, 0x239, 0x042, 0x020, 0x1de, 0x007,
+ 0x289, 0x002, 0x00c, 0x003, 0x006, 0x327, 0x345, 0x000,
+ 0x3f5, 0x327, 0x346, 0x000, 0x3f1, 0x327, 0x347, 0x000,
+ 0x3ed, 0x2c7, 0x01a, 0x020, 0x181, 0x006, 0x327, 0x00c,
+ 0x3c5, 0x002, 0x00c, 0x3c5, 0x001, 0x00d, 0x08f, 0x044,
+ 0x189, 0x000, 0x3c4, 0x3fe, 0x00d, 0x2c7, 0x00c, 0x020,
+ 0x181, 0x006, 0x130, 0x001, 0x013, 0x083, 0x044, 0x189,
+ 0x000, 0x3c6, 0x001, 0x02b, 0x083, 0x044, 0x189, 0x000,
+ 0x3c6, 0x001, 0x02b, 0x131, 0x009, 0x3f1, 0x020, 0x181,
+ 0x006, 0x327, 0x00c, 0x3c5, 0x002, 0x00c, 0x387, 0x07c,
+ 0x187, 0x385, 0x08f, 0x244, 0x02a, 0x088, 0x187, 0x107,
+ 0x245, 0x02a, 0x082, 0x044, 0x189, 0x000, 0x081, 0x18a,
+ 0x208, 0x244, 0x02a, 0x2c7, 0x00c, 0x020, 0x181, 0x006,
+ 0x364, 0x026, 0x002, 0x001, 0x007, 0x364, 0x012, 0x040,
+ 0x001, 0x3f8, 0x020, 0x181, 0x006, 0x042, 0x327, 0x00d,
+ 0x000, 0x16e, 0x042, 0x327, 0x012, 0x000, 0x169, 0x042,
+ 0x197, 0x264, 0x001, 0x009, 0x011, 0x264, 0x002, 0x009,
+ 0x015, 0x264, 0x004, 0x009, 0x017, 0x264, 0x010, 0x009,
+ 0x01b, 0x0a0, 0x000, 0x154, 0x325, 0x21d, 0x2c7, 0x054,
+ 0x384, 0x3fe, 0x000, 0x3e7, 0x0c0, 0x054, 0x384, 0x3fd,
+ 0x000, 0x3e1, 0x324, 0x21f, 0x2c7, 0x054, 0x384, 0x3fb,
+ 0x000, 0x3d9, 0x044, 0x048, 0x008, 0x3c6, 0x100, 0x02b,
+ 0x0c0, 0x000, 0x364, 0x000, 0x020, 0x009, 0x3f5, 0x000,
+ 0x3db, 0x042, 0x040, 0x197, 0x001, 0x00d, 0x111, 0x001,
+ 0x028, 0x111, 0x001, 0x029, 0x111, 0x001, 0x02a, 0x000,
+ 0x041, 0x2c7, 0x254, 0x3b7, 0x241, 0x387, 0x100, 0x044,
+ 0x1a9, 0x000, 0x121, 0x044, 0x192, 0x000, 0x384, 0x030,
+ 0x040, 0x132, 0x044, 0x192, 0x000, 0x3b8, 0x300, 0x284,
+ 0x043, 0x1a3, 0x285, 0x395, 0x080, 0x000, 0x01e, 0x2c7,
+ 0x254, 0x000, 0x01f, 0x2c7, 0x255, 0x000, 0x01b, 0x287,
+ 0x040, 0x3b7, 0x241, 0x044, 0x1bd, 0x000, 0x131, 0x044,
+ 0x192, 0x000, 0x3a7, 0x061, 0x1a6, 0x2a8, 0x284, 0x043,
+ 0x3a4, 0x200, 0x285, 0x3b7, 0x240, 0x044, 0x1bd, 0x000,
+ 0x020, 0x181, 0x006, 0x362, 0x391, 0x001, 0x021, 0x181,
+ 0x006, 0x0c1, 0x391, 0x042, 0x0c0, 0x38f, 0x0c0, 0x390,
+ 0x080, 0x051, 0x2c0, 0x38f, 0x3c1, 0x000, 0x390, 0x3c6,
+ 0x0d8, 0x38f, 0x349, 0x38f, 0x38f, 0x34b, 0x390, 0x390,
+ 0x00a, 0x008, 0x3c0, 0x0e5, 0x38f, 0x3c1, 0x000, 0x390,
+ 0x3b7, 0x00b, 0x1a9, 0x3b5, 0x382, 0x222, 0x001, 0x005,
+ 0x101, 0x000, 0x3e0, 0x101, 0x051, 0x326, 0x38f, 0x3a4,
+ 0x0ff, 0x2e2, 0x0ef, 0x009, 0x029, 0x101, 0x051, 0x040,
+ 0x307, 0x38f, 0x197, 0x226, 0x2e2, 0x0be, 0x009, 0x01e,
+ 0x042, 0x101, 0x051, 0x326, 0x390, 0x3a4, 0x0ff, 0x2e2,
+ 0x0ad, 0x009, 0x013, 0x101, 0x050, 0x327, 0x390, 0x1b7,
+ 0x286, 0x262, 0x0de, 0x009, 0x009, 0x080, 0x040, 0x0c0,
+ 0x391, 0x020, 0x181, 0x006, 0x387, 0x0ff, 0x000, 0x3f8,
+ 0x043, 0x0a0, 0x1a7, 0x3a5, 0x045, 0x000, 0x071, 0x345,
+ 0x221, 0x233, 0x28a, 0x00a, 0x00c, 0x3a4, 0x300, 0x3c4,
+ 0x0ff, 0x027, 0x2c5, 0x027, 0x020, 0x181, 0x006, 0x327,
+ 0x059, 0x1a7, 0x042, 0x000, 0x05b, 0x2e4, 0x020, 0x001,
+ 0x012, 0x081, 0x18c, 0x245, 0x233, 0x2e4, 0x010, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x044, 0x11a, 0x001, 0x000,
+ 0x042, 0x041, 0x091, 0x189, 0x2e4, 0x002, 0x001, 0x015,
+ 0x3c5, 0x040, 0x233, 0x245, 0x024, 0x364, 0x026, 0x002,
+ 0x001, 0x01d, 0x3a7, 0x022, 0x397, 0x200, 0x044, 0x1b7,
+ 0x000, 0x000, 0x014, 0x3c4, 0x3bf, 0x233, 0x364, 0x026,
+ 0x002, 0x001, 0x00c, 0x208, 0x244, 0x024, 0x3a7, 0x022,
+ 0x397, 0x200, 0x044, 0x1a9, 0x000, 0x043, 0x081, 0x189,
+ 0x2e4, 0x001, 0x001, 0x009, 0x3a7, 0x020, 0x044, 0x1a9,
+ 0x000, 0x000, 0x008, 0x3a7, 0x020, 0x208, 0x044, 0x1b8,
+ 0x000, 0x020, 0x181, 0x006, 0x043, 0x0a0, 0x041, 0x020,
+ 0x181, 0x006, 0x2e2, 0x00f, 0x001, 0x060, 0x307, 0x2cf,
+ 0x0c0, 0x2d3, 0x120, 0x001, 0x05f, 0x00b, 0x018, 0x0c1,
+ 0x2d3, 0x3c4, 0x00f, 0x2cf, 0x3c5, 0x100, 0x200, 0x000,
+ 0x048, 0x0c0, 0x2d8, 0x3a4, 0x3f7, 0x001, 0x042, 0x0c1,
+ 0x2d8, 0x2c7, 0x2da, 0x000, 0x03c, 0x2e4, 0x008, 0x009,
+ 0x3f2, 0x2e2, 0x004, 0x009, 0x021, 0x364, 0x026, 0x002,
+ 0x001, 0x3cc, 0x364, 0x027, 0x010, 0x001, 0x3c7, 0x140,
+ 0x2f9, 0x009, 0x3c3, 0x140, 0x2d8, 0x001, 0x3bf, 0x327,
+ 0x2da, 0x364, 0x026, 0x002, 0x001, 0x3b8, 0x364, 0x2cf,
+ 0x073, 0x029, 0x1dc, 0x007, 0x3c4, 0x08f, 0x2cf, 0x2e4,
+ 0x003, 0x001, 0x010, 0x287, 0x1b5, 0x2c7, 0x2d4, 0x384,
+ 0x007, 0x183, 0x245, 0x2cf, 0x3c5, 0x040, 0x200, 0x0c0,
+ 0x2f9, 0x020, 0x181, 0x006, 0x30a, 0x2cf, 0x384, 0x020,
+ 0x000, 0x3f2, 0x347, 0x2cf, 0x2f9, 0x3c4, 0x3df, 0x2cf,
+ 0x000, 0x3f1, 0x042, 0x2c7, 0x269, 0x287, 0x384, 0x0ff,
+ 0x1a1, 0x002, 0x00b, 0x00b, 0x014, 0x247, 0x28d, 0x044,
+ 0x2fb, 0x002, 0x000, 0x005, 0x044, 0x125, 0x003, 0x140,
+ 0x269, 0x00b, 0x01b, 0x0a1, 0x044, 0x30f, 0x002, 0x140,
+ 0x269, 0x00b, 0x013, 0x364, 0x012, 0x040, 0x001, 0x3fd,
+ 0x347, 0x00e, 0x266, 0x347, 0x00f, 0x267, 0x397, 0x200,
+ 0x0a1, 0x044, 0x0fb, 0x002, 0x347, 0x28d, 0x268, 0x307,
+ 0x269, 0x197, 0x384, 0x003, 0x390, 0x266, 0x04c, 0x041,
+ 0x020, 0x181, 0x006, 0x042, 0x247, 0x265, 0x041, 0x397,
+ 0x007, 0x189, 0x395, 0x2ca, 0x1bb, 0x3a4, 0x007, 0x280,
+ 0x050, 0x043, 0x041, 0x1b7, 0x3a4, 0x00f, 0x280, 0x043,
+ 0x364, 0x265, 0x100, 0x009, 0x01b, 0x120, 0x00b, 0x008,
+ 0x3a4, 0x0ff, 0x2a8, 0x121, 0x000, 0x004, 0x3a4, 0x0ff,
+ 0x04d, 0x041, 0x044, 0x3f0, 0x002, 0x044, 0x3de, 0x002,
+ 0x044, 0x06e, 0x003, 0x020, 0x181, 0x006, 0x04c, 0x041,
+ 0x000, 0x3fb, 0x2c4, 0x2a1, 0x28e, 0x293, 0x2b2, 0x2d4,
+ 0x2e4, 0x347, 0x22e, 0x22f, 0x2c7, 0x22e, 0x2e4, 0x040,
+ 0x001, 0x01a, 0x3a7, 0x080, 0x044, 0x192, 0x000, 0x384,
+ 0x3c3, 0x327, 0x22e, 0x1b5, 0x3a4, 0x03c, 0x285, 0x3a7,
+ 0x080, 0x044, 0x1bd, 0x000, 0x347, 0x22f, 0x22e, 0x020,
+ 0x181, 0x006, 0x3c5, 0x004, 0x233, 0x364, 0x22e, 0x001,
+ 0x009, 0x005, 0x3c4, 0x3fb, 0x233, 0x364, 0x22e, 0x020,
+ 0x009, 0x005, 0x080, 0x000, 0x003, 0x081, 0x044, 0x254,
+ 0x001, 0x364, 0x22e, 0x001, 0x001, 0x007, 0x0c8, 0x291,
+ 0x044, 0x06e, 0x003, 0x307, 0x22e, 0x20a, 0x384, 0x001,
+ 0x044, 0x045, 0x008, 0x020, 0x181, 0x006, 0x2c7, 0x223,
+ 0x0c0, 0x224, 0x000, 0x01e, 0x309, 0x223, 0x002, 0x01d,
+ 0x003, 0x020, 0x001, 0x030, 0x362, 0x224, 0x001, 0x003,
+ 0x011, 0x112, 0x001, 0x01f, 0x112, 0x001, 0x057, 0x116,
+ 0x001, 0x06a, 0x114, 0x001, 0x055, 0x112, 0x001, 0x060,
+ 0x020, 0x181, 0x006, 0x20e, 0x384, 0x0ff, 0x000, 0x055,
+ 0x041, 0x229, 0x1b1, 0x042, 0x044, 0x1bd, 0x000, 0x000,
+ 0x3f1, 0x344, 0x21f, 0x02a, 0x083, 0x18b, 0x245, 0x02a,
+ 0x000, 0x005, 0x344, 0x21f, 0x029, 0x3b4, 0x3ff, 0x307,
+ 0x224, 0x262, 0x007, 0x00e, 0x3dd, 0x20a, 0x183, 0x247,
+ 0x02d, 0x30a, 0x224, 0x002, 0x006, 0x2c7, 0x02c, 0x000,
+ 0x009, 0x287, 0x189, 0x245, 0x02c, 0x1b5, 0x2c5, 0x02d,
+ 0x307, 0x223, 0x001, 0x00a, 0x3c5, 0x080, 0x02d, 0x3c4,
+ 0x37f, 0x02d, 0x000, 0x030, 0x3c5, 0x040, 0x02d, 0x3c4,
+ 0x3bf, 0x02d, 0x000, 0x028, 0x397, 0x013, 0x000, 0x004,
+ 0x397, 0x2c7, 0x300, 0x224, 0x140, 0x224, 0x009, 0x005,
+ 0x04d, 0x101, 0x0a0, 0x04d, 0x000, 0x016, 0x2c7, 0x2cb,
+ 0x000, 0x3a0, 0x140, 0x224, 0x009, 0x006, 0x2c7, 0x019,
+ 0x000, 0x00a, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x1ab,
+ 0x2c5, 0x018, 0x141, 0x224, 0x000, 0x38c, 0x042, 0x309,
+ 0x223, 0x002, 0x03e, 0x003, 0x043, 0x116, 0x001, 0x049,
+ 0x112, 0x001, 0x00e, 0x114, 0x029, 0x1dd, 0x007, 0x397,
+ 0x200, 0x300, 0x224, 0x04c, 0x041, 0x000, 0x3e5, 0x307,
+ 0x224, 0x262, 0x007, 0x02e, 0x1dd, 0x007, 0x364, 0x026,
+ 0x004, 0x029, 0x1dd, 0x007, 0x20a, 0x187, 0x0a3, 0x1a7,
+ 0x2a8, 0x2c4, 0x02a, 0x245, 0x02a, 0x327, 0x02e, 0x307,
+ 0x224, 0x20a, 0x00a, 0x009, 0x1b9, 0x307, 0x02f, 0x384,
+ 0x00f, 0x185, 0x225, 0x3b4, 0x3ff, 0x000, 0x022, 0x20a,
+ 0x384, 0x0ff, 0x04c, 0x041, 0x000, 0x344, 0x304, 0x21f,
+ 0x22e, 0x044, 0x192, 0x000, 0x040, 0x000, 0x33b, 0x30a,
+ 0x224, 0x002, 0x011, 0x307, 0x00d, 0x3c4, 0x3f7, 0x00d,
+ 0x347, 0x00f, 0x26a, 0x327, 0x00e, 0x247, 0x00d, 0x041,
+ 0x000, 0x39a, 0x327, 0x26a, 0x000, 0x3fb, 0x04e, 0x020,
+ 0x23b, 0x009, 0x020, 0x28e, 0x00a, 0x020, 0x03a, 0x00a,
+ 0x020, 0x0fe, 0x00a, 0x020, 0x3f3, 0x008, 0x020, 0x013,
+ 0x009, 0x020, 0x075, 0x00a, 0x0c1, 0x244, 0x3c5, 0x040,
+ 0x245, 0x3c4, 0x3bf, 0x200, 0x044, 0x172, 0x000, 0x020,
+ 0x0d3, 0x000, 0x020, 0x2c3, 0x008, 0x020, 0x375, 0x00b,
+ 0x0c1, 0x201, 0x045, 0x020, 0x04a, 0x009, 0x020, 0x04e,
+ 0x009, 0x020, 0x256, 0x002, 0x020, 0x17f, 0x009, 0x020,
+ 0x23f, 0x006, 0x020, 0x285, 0x00a, 0x020, 0x209, 0x00b,
+ 0x020, 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x394,
+ 0x00a, 0x020, 0x048, 0x00b, 0x020, 0x1dc, 0x007, 0x020,
+ 0x27f, 0x00a, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007,
+ 0x020, 0x295, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x277,
+ 0x00b, 0x020, 0x2ba, 0x00b, 0x020, 0x2d5, 0x006, 0x020,
+ 0x326, 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007,
+ 0x020, 0x35a, 0x006, 0x020, 0x0a4, 0x00b, 0x020, 0x38e,
+ 0x006, 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020,
+ 0x31e, 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007,
+ 0x020, 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060,
+ 0x007, 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020,
+ 0x2d1, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252,
+ 0x007, 0x020, 0x293, 0x007, 0x020, 0x2d8, 0x00b, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007,
+ 0x020, 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x30e, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007,
+ 0x020, 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5,
+ 0x008, 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287,
+ 0x264, 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5,
+ 0x020, 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004,
+ 0x3a4, 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009,
+ 0x093, 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe,
+ 0x052, 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de,
+ 0x007, 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c,
+ 0x100, 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c,
+ 0x3d5, 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066,
+ 0x004, 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b,
+ 0x262, 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4,
+ 0x3ef, 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009,
+ 0x007, 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200,
+ 0x004, 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001,
+ 0x028, 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344,
+ 0x21f, 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183,
+ 0x19b, 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001,
+ 0x004, 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5,
+ 0x080, 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044,
+ 0x001, 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5,
+ 0x004, 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044,
+ 0x097, 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d,
+ 0x001, 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390,
+ 0x1d7, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000,
+ 0x021, 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000,
+ 0x025, 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000,
+ 0x041, 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000,
+ 0x031, 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007,
+ 0x044, 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072,
+ 0x2c7, 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a,
+ 0x2c7, 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062,
+ 0x2c7, 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a,
+ 0x2c7, 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052,
+ 0x2c7, 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a,
+ 0x307, 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007,
+ 0x006, 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020,
+ 0x1de, 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3,
+ 0x052, 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c,
+ 0x2c5, 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7,
+ 0x3ff, 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf,
+ 0x354, 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222,
+ 0x355, 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a,
+ 0x006, 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357,
+ 0x020, 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4,
+ 0x00f, 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020,
+ 0x131, 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001,
+ 0x025, 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132,
+ 0x182, 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de,
+ 0x007, 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373,
+ 0x000, 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3,
+ 0x322, 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7,
+ 0x327, 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df,
+ 0x307, 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108,
+ 0x04c, 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b,
+ 0x364, 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f,
+ 0x181, 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5,
+ 0x080, 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7,
+ 0x392, 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5,
+ 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364,
+ 0x33c, 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000,
+ 0x018, 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140,
+ 0x04f, 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3,
+ 0x2c5, 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081,
+ 0x189, 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027,
+ 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053,
+ 0x3c4, 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004,
+ 0x020, 0x08d, 0x006, 0x2c7, 0x231, 0x2e4, 0x100, 0x001,
+ 0x045, 0x2e4, 0x001, 0x001, 0x020, 0x3b7, 0x213, 0x084,
+ 0x044, 0x1a9, 0x000, 0x3b7, 0x250, 0x084, 0x044, 0x1a9,
+ 0x000, 0x399, 0x305, 0x3a7, 0x022, 0x044, 0x1a9, 0x000,
+ 0x044, 0x322, 0x001, 0x3c5, 0x002, 0x024, 0x3c5, 0x008,
+ 0x292, 0x000, 0x020, 0x3c4, 0x3fd, 0x024, 0x364, 0x234,
+ 0x002, 0x001, 0x009, 0x3a7, 0x022, 0x399, 0x305, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x213, 0x084, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x250, 0x084, 0x044, 0x1b7, 0x000, 0x3c4, 0x007,
+ 0x292, 0x044, 0x06e, 0x003, 0x307, 0x231, 0x274, 0x200,
+ 0x001, 0x01b, 0x274, 0x010, 0x001, 0x00b, 0x387, 0x020,
+ 0x245, 0x021, 0x185, 0x245, 0x024, 0x000, 0x00b, 0x387,
+ 0x3df, 0x244, 0x021, 0x185, 0x385, 0x0ff, 0x244, 0x024,
+ 0x044, 0x03a, 0x00a, 0x045, 0x327, 0x230, 0x364, 0x233,
+ 0x004, 0x001, 0x003, 0x045, 0x0c2, 0x22a, 0x347, 0x275,
+ 0x270, 0x0c0, 0x27b, 0x264, 0x010, 0x009, 0x00a, 0x0c1,
+ 0x22a, 0x347, 0x274, 0x270, 0x3d7, 0x200, 0x27b, 0x327,
+ 0x230, 0x264, 0x004, 0x001, 0x012, 0x347, 0x277, 0x270,
+ 0x264, 0x010, 0x009, 0x007, 0x347, 0x276, 0x270, 0x384,
+ 0x00f, 0x112, 0x1b7, 0x000, 0x004, 0x3a4, 0x0ff, 0x2c7,
+ 0x225, 0x384, 0x00f, 0x262, 0x003, 0x001, 0x00b, 0x101,
+ 0x3c2, 0x100, 0x270, 0x34a, 0x225, 0x225, 0x000, 0x3f3,
+ 0x044, 0x237, 0x00a, 0x044, 0x35f, 0x009, 0x364, 0x22a,
+ 0x002, 0x001, 0x006, 0x347, 0x270, 0x272, 0x045, 0x347,
+ 0x270, 0x271, 0x045, 0x044, 0x2f6, 0x002, 0x009, 0x015,
+ 0x307, 0x241, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x3d7, 0x200, 0x27b,
+ 0x044, 0x35f, 0x009, 0x347, 0x241, 0x226, 0x0c1, 0x22a,
+ 0x020, 0x032, 0x009, 0x044, 0x2f6, 0x002, 0x009, 0x014,
+ 0x307, 0x243, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x0c0, 0x27b, 0x044,
+ 0x35f, 0x009, 0x347, 0x243, 0x226, 0x0c2, 0x22a, 0x020,
+ 0x032, 0x009, 0x327, 0x226, 0x2c7, 0x225, 0x3c4, 0x0ff,
+ 0x225, 0x1b7, 0x3a4, 0x007, 0x001, 0x00d, 0x34a, 0x225,
+ 0x225, 0x002, 0x008, 0x131, 0x009, 0x3fa, 0x044, 0x237,
+ 0x00a, 0x045, 0x0c1, 0x26f, 0x000, 0x050, 0x327, 0x01d,
+ 0x2e4, 0x380, 0x009, 0x00a, 0x13a, 0x00b, 0x00a, 0x3c5,
+ 0x008, 0x233, 0x000, 0x005, 0x3c4, 0x3f7, 0x233, 0x327,
+ 0x01d, 0x021, 0x2ed, 0x009, 0x364, 0x020, 0x004, 0x009,
+ 0x01a, 0x309, 0x232, 0x00b, 0x009, 0x347, 0x270, 0x271,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x271, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x04c, 0x347, 0x270, 0x271, 0x000,
+ 0x018, 0x309, 0x232, 0x001, 0x009, 0x347, 0x270, 0x272,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x272, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x089, 0x347, 0x270, 0x272, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x364, 0x26f, 0x002, 0x009,
+ 0x005, 0x3c4, 0x3fe, 0x200, 0x364, 0x020, 0x005, 0x029,
+ 0x2e4, 0x002, 0x364, 0x020, 0x010, 0x009, 0x0e9, 0x364,
+ 0x020, 0x100, 0x009, 0x12b, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3f7, 0x233, 0x364, 0x020, 0x001, 0x009, 0x00c, 0x364,
+ 0x020, 0x004, 0x009, 0x05d, 0x000, 0x3d8, 0x3c5, 0x080,
+ 0x020, 0x364, 0x233, 0x006, 0x001, 0x031, 0x081, 0x189,
+ 0x304, 0x232, 0x009, 0x017, 0x347, 0x271, 0x274, 0x080,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x085, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245, 0x232, 0x000,
+ 0x0c8, 0x347, 0x271, 0x276, 0x081, 0x189, 0x208, 0x244,
+ 0x232, 0x307, 0x22b, 0x197, 0x384, 0x007, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x000, 0x005, 0x044, 0x3f3, 0x008,
+ 0x3b7, 0x200, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x203, 0x387, 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020,
+ 0x385, 0x042, 0x384, 0x3ce, 0x247, 0x020, 0x044, 0x044,
+ 0x001, 0x000, 0x383, 0x0a1, 0x1aa, 0x2c5, 0x020, 0x364,
+ 0x233, 0x006, 0x001, 0x039, 0x081, 0x189, 0x304, 0x232,
+ 0x009, 0x017, 0x347, 0x272, 0x275, 0x080, 0x364, 0x233,
+ 0x004, 0x009, 0x003, 0x095, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x081, 0x189, 0x245, 0x232, 0x000, 0x0ce, 0x347,
+ 0x272, 0x277, 0x081, 0x189, 0x208, 0x244, 0x232, 0x307,
+ 0x22b, 0x197, 0x364, 0x233, 0x002, 0x001, 0x007, 0x193,
+ 0x384, 0x007, 0x385, 0x010, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x000, 0x008, 0x347, 0x272, 0x270, 0x044, 0x013,
+ 0x009, 0x081, 0x18a, 0x208, 0x244, 0x232, 0x0a3, 0x387,
+ 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020, 0x0a4, 0x1a7,
+ 0x3a5, 0x008, 0x285, 0x3b7, 0x304, 0x2a8, 0x284, 0x247,
+ 0x020, 0x0a0, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x044,
+ 0x2f6, 0x002, 0x009, 0x00f, 0x364, 0x2cf, 0x080, 0x009,
+ 0x00a, 0x0c3, 0x26f, 0x044, 0x1b6, 0x001, 0x020, 0x2e4,
+ 0x002, 0x044, 0x051, 0x001, 0x000, 0x308, 0x3c4, 0x3f7,
+ 0x233, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x083, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x3d7,
+ 0x200, 0x27b, 0x0c0, 0x271, 0x3b7, 0x300, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x221, 0x091, 0x189, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x201, 0x387, 0x0a6, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x021, 0x020, 0x3c4, 0x3ed, 0x020, 0x3b7,
+ 0x200, 0x387, 0x020, 0x044, 0x020, 0x00a, 0x009, 0x32a,
+ 0x347, 0x271, 0x270, 0x000, 0x055, 0x3a7, 0x071, 0x307,
+ 0x21d, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x233, 0x044,
+ 0x2f6, 0x002, 0x009, 0x007, 0x0c1, 0x2d3, 0x3c5, 0x100,
+ 0x200, 0x081, 0x18a, 0x245, 0x232, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x093, 0x0c2, 0x22a,
+ 0x044, 0x39c, 0x008, 0x0c0, 0x27b, 0x0c0, 0x272, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1, 0x397,
+ 0x205, 0x247, 0x27f, 0x044, 0x1bd, 0x000, 0x3d5, 0x204,
+ 0x020, 0x3c4, 0x2ff, 0x020, 0x0a0, 0x387, 0x021, 0x184,
+ 0x044, 0x020, 0x00a, 0x009, 0x336, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3fe, 0x200, 0x3c4, 0x3fe,
+ 0x26f, 0x344, 0x21f, 0x232, 0x0c0, 0x020, 0x0a3, 0x1ab,
+ 0x2c5, 0x020, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000,
+ 0x044, 0x3b5, 0x000, 0x044, 0x3f0, 0x000, 0x0a9, 0x1ab,
+ 0x09f, 0x18a, 0x208, 0x304, 0x055, 0x225, 0x044, 0x394,
+ 0x00a, 0x307, 0x055, 0x19b, 0x384, 0x003, 0x227, 0x1a3,
+ 0x225, 0x044, 0x0a4, 0x00b, 0x0a7, 0x020, 0x022, 0x008,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x006, 0x387, 0x07f,
+ 0x000, 0x013, 0x399, 0x240, 0x385, 0x07f, 0x306, 0x270,
+ 0x001, 0x06c, 0x399, 0x240, 0x327, 0x270, 0x3a4, 0x380,
+ 0x226, 0x001, 0x015, 0x327, 0x220, 0x324, 0x232, 0x009,
+ 0x019, 0x364, 0x232, 0x001, 0x001, 0x02b, 0x327, 0x01d,
+ 0x307, 0x01b, 0x190, 0x282, 0x003, 0x041, 0x307, 0x01b,
+ 0x191, 0x380, 0x010, 0x282, 0x003, 0x034, 0x000, 0x019,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x00a, 0x327, 0x270,
+ 0x384, 0x07f, 0x3a4, 0x07f, 0x000, 0x008, 0x327, 0x270,
+ 0x3a4, 0x07f, 0x001, 0x005, 0x286, 0x001, 0x02f, 0x307,
+ 0x270, 0x384, 0x07f, 0x386, 0x07f, 0x009, 0x00a, 0x387,
+ 0x080, 0x382, 0x07f, 0x240, 0x270, 0x000, 0x01c, 0x141,
+ 0x270, 0x364, 0x270, 0x07f, 0x009, 0x015, 0x000, 0x013,
+ 0x3c0, 0x040, 0x270, 0x000, 0x00e, 0x307, 0x270, 0x196,
+ 0x384, 0x00f, 0x266, 0x009, 0x001, 0x008, 0x3c0, 0x080,
+ 0x270, 0x044, 0x35f, 0x009, 0x045, 0x309, 0x232, 0x00b,
+ 0x013, 0x347, 0x271, 0x270, 0x374, 0x27b, 0x200, 0x009,
+ 0x005, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x020,
+ 0x0b7, 0x009, 0x327, 0x232, 0x1a2, 0x023, 0x0b7, 0x009,
+ 0x307, 0x270, 0x196, 0x119, 0x021, 0x0b7, 0x009, 0x307,
+ 0x232, 0x181, 0x023, 0x0b7, 0x009, 0x364, 0x270, 0x080,
+ 0x029, 0x0b7, 0x009, 0x083, 0x18c, 0x245, 0x232, 0x3c4,
+ 0x3f7, 0x233, 0x3c4, 0x380, 0x270, 0x3c0, 0x080, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x09c, 0x009, 0x000, 0x000, 0x000, 0x002, 0x000,
+ 0x00a, 0x000, 0x02a, 0x000, 0x0aa, 0x002, 0x0aa, 0x00a,
+ 0x0aa, 0x02a, 0x0aa, 0x06a, 0x0aa, 0x07a, 0x0aa, 0x07e,
+ 0x0aa, 0x07f, 0x0aa, 0x07f, 0x0ba, 0x07f, 0x0be, 0x07f,
+ 0x0fe, 0x07f, 0x0ff, 0x007, 0x000, 0x017, 0x001, 0x027,
+ 0x007, 0x037, 0x00f, 0x047, 0x03f, 0x080, 0x0ff, 0x364,
+ 0x270, 0x07f, 0x307, 0x270, 0x384, 0x07f, 0x009, 0x00d,
+ 0x089, 0x186, 0x306, 0x270, 0x001, 0x007, 0x3c5, 0x001,
+ 0x232, 0x000, 0x005, 0x3c4, 0x3fe, 0x232, 0x347, 0x270,
+ 0x27c, 0x3c4, 0x007, 0x27c, 0x374, 0x27b, 0x200, 0x001,
+ 0x00a, 0x364, 0x233, 0x008, 0x001, 0x005, 0x044, 0x0fe,
+ 0x00a, 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384,
+ 0x3f8, 0x305, 0x27c, 0x044, 0x1bd, 0x000, 0x307, 0x270,
+ 0x192, 0x384, 0x00f, 0x3b9, 0x200, 0x324, 0x233, 0x001,
+ 0x014, 0x229, 0x387, 0x009, 0x189, 0x395, 0x333, 0x280,
+ 0x051, 0x2c7, 0x27d, 0x1c7, 0x27d, 0x101, 0x051, 0x2c5,
+ 0x27d, 0x000, 0x011, 0x0c0, 0x27d, 0x384, 0x00f, 0x001,
+ 0x00b, 0x349, 0x27d, 0x27d, 0x3c5, 0x001, 0x27d, 0x111,
+ 0x000, 0x3f7, 0x0a2, 0x325, 0x27b, 0x307, 0x27d, 0x044,
+ 0x1bd, 0x000, 0x374, 0x27b, 0x200, 0x001, 0x02c, 0x347,
+ 0x270, 0x27a, 0x3c4, 0x07f, 0x27a, 0x0c0, 0x279, 0x387,
+ 0x009, 0x189, 0x395, 0x353, 0x051, 0x041, 0x101, 0x051,
+ 0x2c7, 0x279, 0x101, 0x043, 0x322, 0x27a, 0x00b, 0x004,
+ 0x000, 0x3f4, 0x1c7, 0x279, 0x0a3, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x384, 0x0ff, 0x305, 0x279, 0x044, 0x1bd,
+ 0x000, 0x327, 0x220, 0x324, 0x232, 0x009, 0x022, 0x0c0,
+ 0x27e, 0x307, 0x270, 0x197, 0x00a, 0x004, 0x000, 0x005,
+ 0x3c5, 0x018, 0x27e, 0x0a4, 0x222, 0x2c5, 0x27e, 0x0a4,
+ 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x3e0, 0x3c4,
+ 0x01f, 0x27e, 0x305, 0x27e, 0x044, 0x1bd, 0x000, 0x045,
+ 0x044, 0x1a9, 0x000, 0x083, 0x18c, 0x208, 0x244, 0x232,
+ 0x3c5, 0x001, 0x00c, 0x0a3, 0x325, 0x27b, 0x088, 0x044,
+ 0x1a9, 0x000, 0x387, 0x3f8, 0x044, 0x1b8, 0x000, 0x020,
+ 0x2f6, 0x002, 0x3c7, 0x020, 0x280, 0x3b7, 0x215, 0x044,
+ 0x192, 0x000, 0x247, 0x247, 0x151, 0x280, 0x001, 0x016,
+ 0x0a1, 0x044, 0x182, 0x000, 0x3b7, 0x215, 0x044, 0x192,
+ 0x000, 0x306, 0x247, 0x001, 0x3f1, 0x0a2, 0x364, 0x22b,
+ 0x003, 0x001, 0x003, 0x0a5, 0x020, 0x182, 0x000, 0x1f6,
+ 0x185, 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122,
+ 0x0ff, 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194,
+ 0x0ce, 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020,
+ 0x009, 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9,
+ 0x286, 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x05f,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051,
+ 0x2c7, 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f,
+ 0x040, 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b,
+ 0x005, 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4,
+ 0x00f, 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288,
+ 0x286, 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b,
+ 0x3fc, 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285,
+ 0x307, 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd,
+ 0x000, 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2,
+ 0x12c, 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002,
+ 0x005, 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b,
+ 0x1f0, 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081,
+ 0x138, 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f,
+ 0x080, 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d,
+ 0x1e0, 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b,
+ 0x347, 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044,
+ 0x1bc, 0x00a, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364,
+ 0x233, 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff,
+ 0x242, 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289,
+ 0x397, 0x00a, 0x189, 0x395, 0x0ca, 0x044, 0x2ea, 0x001,
+ 0x009, 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9,
+ 0x121, 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f,
+ 0x282, 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000,
+ 0x3ed, 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233,
+ 0x002, 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff,
+ 0x281, 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366,
+ 0x282, 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282,
+ 0x044, 0x1bc, 0x00a, 0x001, 0x00d, 0x141, 0x28b, 0x362,
+ 0x28b, 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5,
+ 0x362, 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397,
+ 0x00a, 0x189, 0x395, 0x0d8, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005,
+ 0x105, 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f,
+ 0x121, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042,
+ 0x003, 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387,
+ 0x00a, 0x189, 0x395, 0x0d3, 0x300, 0x28a, 0x050, 0x247,
+ 0x28a, 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a,
+ 0x044, 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282,
+ 0x281, 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045,
+ 0x000, 0x301, 0x103, 0x307, 0x006, 0x10e, 0x21e, 0x33e,
+ 0x13c, 0x038, 0x078, 0x1f8, 0x0f0, 0x000, 0x001, 0x003,
+ 0x002, 0x006, 0x187, 0x227, 0x387, 0x00a, 0x189, 0x395,
+ 0x1c8, 0x300, 0x227, 0x111, 0x050, 0x247, 0x24c, 0x1d7,
+ 0x24c, 0x384, 0x0ff, 0x285, 0x044, 0x226, 0x00a, 0x364,
+ 0x22a, 0x001, 0x001, 0x00e, 0x3d5, 0x200, 0x232, 0x040,
+ 0x1aa, 0x285, 0x3b7, 0x220, 0x044, 0x1bd, 0x000, 0x042,
+ 0x364, 0x22a, 0x002, 0x001, 0x022, 0x040, 0x3a7, 0x073,
+ 0x044, 0x192, 0x000, 0x384, 0x31f, 0x044, 0x226, 0x00a,
+ 0x1a4, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000, 0x3a7,
+ 0x070, 0x044, 0x192, 0x000, 0x19a, 0x18a, 0x043, 0x285,
+ 0x3a7, 0x070, 0x044, 0x1bd, 0x000, 0x045, 0x040, 0x387,
+ 0x00a, 0x189, 0x395, 0x1d5, 0x300, 0x278, 0x051, 0x042,
+ 0x045, 0x00a, 0x058, 0x026, 0x015, 0x044, 0x032, 0x0c0,
+ 0x278, 0x0a0, 0x397, 0x00a, 0x189, 0x395, 0x231, 0x280,
+ 0x050, 0x247, 0x228, 0x3c4, 0x00f, 0x228, 0x347, 0x228,
+ 0x229, 0x040, 0x307, 0x278, 0x001, 0x008, 0x340, 0x229,
+ 0x228, 0x111, 0x000, 0x3fa, 0x042, 0x193, 0x0c0, 0x227,
+ 0x347, 0x225, 0x226, 0x141, 0x227, 0x366, 0x227, 0x00e,
+ 0x001, 0x00b, 0x342, 0x228, 0x226, 0x007, 0x3f6, 0x140,
+ 0x226, 0x001, 0x011, 0x121, 0x2e2, 0x006, 0x009, 0x3cc,
+ 0x141, 0x278, 0x366, 0x278, 0x005, 0x009, 0x3c4, 0x0a1,
+ 0x000, 0x006, 0x044, 0x1da, 0x00a, 0x0a0, 0x045, 0x044,
+ 0x333, 0x008, 0x020, 0x181, 0x006, 0x345, 0x21d, 0x232,
+ 0x364, 0x23f, 0x001, 0x029, 0x2e4, 0x002, 0x2c7, 0x234,
+ 0x399, 0x200, 0x245, 0x233, 0x364, 0x234, 0x020, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x083, 0x18c, 0x208, 0x244,
+ 0x036, 0x2e4, 0x008, 0x001, 0x007, 0x345, 0x220, 0x232,
+ 0x000, 0x007, 0x081, 0x18b, 0x208, 0x244, 0x232, 0x307,
+ 0x235, 0x226, 0x3a4, 0x001, 0x001, 0x010, 0x327, 0x234,
+ 0x2e4, 0x001, 0x009, 0x041, 0x041, 0x044, 0x20d, 0x000,
+ 0x044, 0x2f3, 0x000, 0x043, 0x0a3, 0x1ad, 0x3a5, 0x039,
+ 0x326, 0x236, 0x001, 0x02f, 0x3a7, 0x0bf, 0x1a7, 0x3a5,
+ 0x039, 0x326, 0x236, 0x001, 0x026, 0x327, 0x234, 0x307,
+ 0x235, 0x286, 0x384, 0x002, 0x001, 0x01d, 0x2e4, 0x002,
+ 0x009, 0x051, 0x041, 0x0a1, 0x1ad, 0x2c5, 0x024, 0x307,
+ 0x024, 0x19a, 0x002, 0x005, 0x2a8, 0x2c4, 0x024, 0x044,
+ 0x3b5, 0x000, 0x044, 0x0f2, 0x001, 0x044, 0x3f0, 0x000,
+ 0x043, 0x000, 0x067, 0x364, 0x026, 0x001, 0x009, 0x3c6,
+ 0x041, 0x0a0, 0x044, 0x301, 0x000, 0x364, 0x26f, 0x002,
+ 0x021, 0x310, 0x00a, 0x044, 0x290, 0x001, 0x0c1, 0x26f,
+ 0x043, 0x307, 0x020, 0x193, 0x384, 0x007, 0x110, 0x001,
+ 0x00c, 0x364, 0x232, 0x010, 0x009, 0x007, 0x044, 0x044,
+ 0x001, 0x000, 0x3a3, 0x3c4, 0x3fe, 0x26f, 0x3c5, 0x001,
+ 0x200, 0x3c5, 0x010, 0x020, 0x3c4, 0x31f, 0x020, 0x000,
+ 0x395, 0x364, 0x026, 0x002, 0x009, 0x3c5, 0x041, 0x044,
+ 0x003, 0x001, 0x043, 0x307, 0x020, 0x197, 0x384, 0x007,
+ 0x110, 0x001, 0x00f, 0x364, 0x232, 0x020, 0x009, 0x00a,
+ 0x044, 0x051, 0x001, 0x3c5, 0x001, 0x027, 0x000, 0x3ab,
+ 0x3c4, 0x3fc, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x100,
+ 0x020, 0x087, 0x188, 0x208, 0x244, 0x020, 0x000, 0x39b,
+ 0x044, 0x11a, 0x001, 0x307, 0x234, 0x306, 0x235, 0x384,
+ 0x003, 0x021, 0x2e4, 0x002, 0x347, 0x234, 0x235, 0x366,
+ 0x26f, 0x002, 0x021, 0x2e4, 0x002, 0x364, 0x234, 0x003,
+ 0x021, 0x2e4, 0x002, 0x327, 0x21c, 0x324, 0x020, 0x021,
+ 0x385, 0x00a, 0x020, 0x247, 0x002, 0x364, 0x23f, 0x001,
+ 0x021, 0x38e, 0x00a, 0x020, 0x247, 0x002, 0x345, 0x21d,
+ 0x232, 0x020, 0x247, 0x002, 0x2c7, 0x241, 0x307, 0x242,
+ 0x286, 0x001, 0x092, 0x0c1, 0x22a, 0x327, 0x242, 0x326,
+ 0x241, 0x3a4, 0x0ff, 0x001, 0x005, 0x3c4, 0x38f, 0x020,
+ 0x347, 0x241, 0x242, 0x347, 0x241, 0x226, 0x307, 0x21d,
+ 0x304, 0x241, 0x001, 0x00a, 0x0c3, 0x22a, 0x347, 0x241,
+ 0x243, 0x044, 0x39f, 0x002, 0x327, 0x241, 0x3a4, 0x0ff,
+ 0x2c7, 0x225, 0x3c4, 0x300, 0x230, 0x2c5, 0x230, 0x044,
+ 0x099, 0x00b, 0x081, 0x18b, 0x304, 0x226, 0x009, 0x00d,
+ 0x364, 0x232, 0x100, 0x009, 0x008, 0x081, 0x18a, 0x208,
+ 0x044, 0x1b8, 0x000, 0x044, 0x186, 0x001, 0x327, 0x226,
+ 0x003, 0x032, 0x044, 0x2f6, 0x002, 0x001, 0x019, 0x044,
+ 0x237, 0x00a, 0x347, 0x241, 0x226, 0x081, 0x306, 0x22a,
+ 0x384, 0x003, 0x001, 0x005, 0x347, 0x243, 0x226, 0x044,
+ 0x032, 0x009, 0x002, 0x02d, 0x000, 0x031, 0x044, 0x237,
+ 0x00a, 0x009, 0x01c, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x001, 0x023, 0x364, 0x233, 0x001, 0x000,
+ 0x01e, 0x045, 0x044, 0x186, 0x001, 0x3c5, 0x003, 0x22a,
+ 0x347, 0x241, 0x243, 0x000, 0x3c7, 0x3a4, 0x3ff, 0x001,
+ 0x00e, 0x327, 0x21c, 0x324, 0x020, 0x009, 0x006, 0x043,
+ 0x3a7, 0x0ff, 0x041, 0x0c0, 0x270, 0x044, 0x082, 0x00b,
+ 0x3c4, 0x3fc, 0x22a, 0x327, 0x21c, 0x324, 0x020, 0x009,
+ 0x010, 0x307, 0x270, 0x003, 0x005, 0x020, 0x181, 0x006,
+ 0x043, 0x3a7, 0x0ff, 0x041, 0x020, 0x181, 0x006, 0x045,
+ 0x2c7, 0x243, 0x307, 0x250, 0x286, 0x001, 0x3de, 0x0c2,
+ 0x22a, 0x041, 0x044, 0x39f, 0x002, 0x043, 0x2c7, 0x226,
+ 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x0ff, 0x230, 0x1a7,
+ 0x2c5, 0x230, 0x307, 0x21d, 0x304, 0x243, 0x001, 0x013,
+ 0x3c4, 0x0ff, 0x243, 0x347, 0x243, 0x241, 0x347, 0x243,
+ 0x242, 0x3c5, 0x003, 0x22a, 0x044, 0x186, 0x001, 0x000,
+ 0x005, 0x044, 0x191, 0x001, 0x327, 0x243, 0x003, 0x394,
+ 0x000, 0x362, 0x364, 0x232, 0x004, 0x001, 0x008, 0x044,
+ 0x232, 0x000, 0x3c4, 0x3fb, 0x232, 0x364, 0x232, 0x008,
+ 0x001, 0x008, 0x044, 0x326, 0x000, 0x3c4, 0x3f7, 0x232,
+ 0x045, 0x3b7, 0x221, 0x083, 0x189, 0x044, 0x1a9, 0x000,
+ 0x083, 0x020, 0x1b7, 0x000, 0x2c7, 0x24d, 0x364, 0x233,
+ 0x002, 0x001, 0x019, 0x3b9, 0x381, 0x121, 0x081, 0x18a,
+ 0x225, 0x304, 0x24d, 0x001, 0x005, 0x044, 0x1ff, 0x00b,
+ 0x140, 0x24d, 0x00b, 0x008, 0x1a3, 0x044, 0x1ff, 0x00b,
+ 0x000, 0x012, 0x307, 0x220, 0x284, 0x001, 0x00d, 0x3c4,
+ 0x33f, 0x232, 0x1b6, 0x3a4, 0x0c0, 0x2c5, 0x232, 0x020,
+ 0x18a, 0x00b, 0x327, 0x24d, 0x364, 0x020, 0x070, 0x001,
+ 0x00a, 0x399, 0x383, 0x101, 0x326, 0x22b, 0x234, 0x001,
+ 0x02e, 0x3c5, 0x080, 0x233, 0x3d5, 0x200, 0x232, 0x364,
+ 0x026, 0x001, 0x001, 0x00a, 0x3c5, 0x004, 0x232, 0x044,
+ 0x20d, 0x000, 0x307, 0x24d, 0x307, 0x24d, 0x384, 0x007,
+ 0x001, 0x09a, 0x113, 0x001, 0x0a8, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3f8, 0x103, 0x044, 0x1bd, 0x000,
+ 0x3c4, 0x3f7, 0x021, 0x000, 0x0af, 0x0a7, 0x1a7, 0x324,
+ 0x020, 0x001, 0x014, 0x327, 0x24d, 0x326, 0x22b, 0x397,
+ 0x370, 0x364, 0x233, 0x002, 0x001, 0x006, 0x087, 0x18b,
+ 0x385, 0x070, 0x234, 0x001, 0x02b, 0x3c5, 0x100, 0x233,
+ 0x364, 0x026, 0x002, 0x001, 0x008, 0x3c5, 0x008, 0x232,
+ 0x044, 0x3b5, 0x000, 0x307, 0x24d, 0x384, 0x070, 0x001,
+ 0x091, 0x382, 0x030, 0x001, 0x0a6, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x018, 0x044, 0x1bd,
+ 0x000, 0x3c4, 0x3f7, 0x024, 0x000, 0x0a4, 0x347, 0x24d,
+ 0x22b, 0x364, 0x233, 0x006, 0x001, 0x033, 0x307, 0x22b,
+ 0x364, 0x233, 0x002, 0x001, 0x023, 0x364, 0x233, 0x080,
+ 0x001, 0x00a, 0x307, 0x24d, 0x197, 0x384, 0x007, 0x044,
+ 0x3a4, 0x008, 0x364, 0x233, 0x100, 0x001, 0x00c, 0x307,
+ 0x24d, 0x19b, 0x384, 0x007, 0x385, 0x010, 0x044, 0x3a4,
+ 0x008, 0x3c4, 0x27f, 0x233, 0x000, 0x00b, 0x3c5, 0x003,
+ 0x22a, 0x197, 0x044, 0x39c, 0x008, 0x003, 0x298, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x021, 0x181,
+ 0x006, 0x045, 0x3b7, 0x301, 0x087, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7,
+ 0x021, 0x000, 0x36c, 0x3b7, 0x301, 0x044, 0x192, 0x000,
+ 0x384, 0x3f8, 0x105, 0x044, 0x1bd, 0x000, 0x3b7, 0x211,
+ 0x082, 0x044, 0x1a9, 0x000, 0x3c5, 0x008, 0x021, 0x344,
+ 0x21f, 0x021, 0x3b7, 0x250, 0x088, 0x044, 0x1a9, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x347,
+ 0x3b7, 0x301, 0x387, 0x3c7, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x060, 0x083, 0x044, 0x1b7, 0x000, 0x3b7, 0x070, 0x083,
+ 0x18a, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x024, 0x000,
+ 0x36f, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x028, 0x044, 0x1bd, 0x000, 0x3c5, 0x048, 0x024,
+ 0x3a7, 0x060, 0x083, 0x044, 0x1a9, 0x000, 0x3b7, 0x070,
+ 0x083, 0x18a, 0x044, 0x1a9, 0x000, 0x000, 0x351, 0x307,
+ 0x22b, 0x284, 0x2a8, 0x2c4, 0x24d, 0x245, 0x24d, 0x2a8,
+ 0x045, 0x2f4, 0x200, 0x001, 0x008, 0x364, 0x2cf, 0x007,
+ 0x029, 0x1dc, 0x007, 0x287, 0x385, 0x300, 0x208, 0x001,
+ 0x04b, 0x364, 0x33b, 0x010, 0x029, 0x1dc, 0x007, 0x3c5,
+ 0x002, 0x00c, 0x287, 0x384, 0x0ff, 0x305, 0x21d, 0x1b7,
+ 0x2e4, 0x001, 0x001, 0x01d, 0x040, 0x384, 0x007, 0x117,
+ 0x001, 0x00c, 0x3c5, 0x040, 0x02d, 0x081, 0x044, 0x189,
+ 0x000, 0x3c4, 0x3bf, 0x02d, 0x042, 0x247, 0x029, 0x364,
+ 0x018, 0x001, 0x009, 0x005, 0x3c5, 0x010, 0x021, 0x2e4,
+ 0x002, 0x001, 0x017, 0x264, 0x080, 0x001, 0x005, 0x0a4,
+ 0x1a7, 0x285, 0x040, 0x3c5, 0x080, 0x02d, 0x081, 0x044,
+ 0x189, 0x000, 0x3c4, 0x37f, 0x02d, 0x042, 0x247, 0x02a,
+ 0x000, 0x014, 0x38e, 0x3ff, 0x1b8, 0x00a, 0x00a, 0x244,
+ 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4, 0x3fd, 0x00c, 0x120,
+ 0x001, 0x004, 0x244, 0x02a, 0x020, 0x181, 0x006, 0x289,
+ 0x00a, 0x004, 0x2c7, 0x251, 0x22e, 0x364, 0x026, 0x001,
+ 0x021, 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x002, 0x009,
+ 0x01b, 0x3c4, 0x2ff, 0x232, 0x120, 0x001, 0x012, 0x3d5,
+ 0x300, 0x232, 0x044, 0x099, 0x00b, 0x32a, 0x026, 0x00a,
+ 0x008, 0x044, 0x20d, 0x000, 0x044, 0x232, 0x000, 0x020,
+ 0x181, 0x006, 0x042, 0x307, 0x240, 0x20a, 0x040, 0x020,
+ 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x256, 0x22e,
+ 0x364, 0x026, 0x002, 0x021, 0x181, 0x006, 0x041, 0x3a4,
+ 0x01f, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x043, 0x1b7,
+ 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x020, 0x181, 0x006,
+ 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044, 0x340, 0x001,
+ 0x247, 0x249, 0x054, 0x347, 0x249, 0x248, 0x2c7, 0x23f,
+ 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f, 0x001, 0x001,
+ 0x006, 0x245, 0x057, 0x000, 0x005, 0x208, 0x244, 0x057,
+ 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f, 0x002, 0x001,
+ 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001, 0x000, 0x005,
+ 0x208, 0x244, 0x057, 0x020, 0x181, 0x006, 0x2c7, 0x246,
+ 0x3b7, 0x300, 0x044, 0x192, 0x000, 0x384, 0x33f, 0x327,
+ 0x246, 0x3a4, 0x0c0, 0x285, 0x3b7, 0x300, 0x044, 0x1bd,
+ 0x000, 0x3b7, 0x200, 0x044, 0x192, 0x000, 0x0a3, 0x1a9,
+ 0x2a8, 0x284, 0x040, 0x307, 0x246, 0x384, 0x030, 0x264,
+ 0x010, 0x001, 0x012, 0x040, 0x041, 0x3b7, 0x201, 0x044,
+ 0x192, 0x000, 0x384, 0x3e0, 0x385, 0x006, 0x044, 0x1bd,
+ 0x000, 0x043, 0x042, 0x185, 0x2a8, 0x284, 0x227, 0x042,
+ 0x285, 0x3b7, 0x200, 0x044, 0x1bd, 0x000, 0x307, 0x246,
+ 0x384, 0x007, 0x044, 0x375, 0x00b, 0x3a7, 0x073, 0x307,
+ 0x246, 0x184, 0x00a, 0x014, 0x044, 0x192, 0x000, 0x384,
+ 0x3e2, 0x327, 0x246, 0x1b5, 0x3a4, 0x01c, 0x285, 0x3a7,
+ 0x073, 0x044, 0x1bd, 0x000, 0x000, 0x006, 0x081, 0x044,
+ 0x1a9, 0x000, 0x020, 0x181, 0x006, 0x3c5, 0x001, 0x00c,
+ 0x187, 0x0a7, 0x1a7, 0x2a8, 0x324, 0x00c, 0x225, 0x2c7,
+ 0x00c, 0x045, 0x04e, 0x087, 0x011, 0x054, 0x084
+
+}; /* end fm10000_serdes_spico_code_prd2 */
+
+
+
+/* Spico Swap code (production version 2)
+ * source file: serdes.0x2055_0045.swap
+ * Note: if swap code exists, if must be loaded on top of
+ * the sbus master image.
+ */
+
+const uint32_t fm10000_serdes_swap_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_swap_code_size_prd2 = 7842;
+
+const uint16_t fm10000_serdes_swap_code_prd2[] = {
+ 0x01e, 0x09d, 0x002, 0x000, 0x000, 0x008, 0x00f, 0x07b,
+ 0x020, 0x01e, 0x00f, 0x069, 0x01d, 0x029, 0x00b, 0x01d,
+ 0x02c, 0x382, 0x04e, 0x020, 0x23b, 0x009, 0x020, 0x28e,
+ 0x00a, 0x020, 0x03a, 0x00a, 0x020, 0x0fe, 0x00a, 0x020,
+ 0x3f3, 0x008, 0x020, 0x013, 0x009, 0x020, 0x075, 0x00a,
+ 0x0c1, 0x244, 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200,
+ 0x044, 0x172, 0x000, 0x020, 0x0d3, 0x000, 0x020, 0x2c3,
+ 0x008, 0x020, 0x375, 0x00b, 0x0c1, 0x201, 0x045, 0x020,
+ 0x04a, 0x009, 0x020, 0x04e, 0x009, 0x020, 0x256, 0x002,
+ 0x020, 0x17f, 0x009, 0x020, 0x23f, 0x006, 0x020, 0x285,
+ 0x00a, 0x020, 0x209, 0x00b, 0x020, 0x257, 0x006, 0x020,
+ 0x117, 0x008, 0x020, 0x394, 0x00a, 0x020, 0x048, 0x00b,
+ 0x020, 0x1dc, 0x007, 0x020, 0x27f, 0x00a, 0x020, 0x2d2,
+ 0x006, 0x020, 0x1e2, 0x007, 0x020, 0x295, 0x00b, 0x020,
+ 0x3a8, 0x001, 0x020, 0x277, 0x00b, 0x020, 0x2ba, 0x00b,
+ 0x020, 0x2d5, 0x006, 0x020, 0x326, 0x006, 0x020, 0x350,
+ 0x006, 0x020, 0x1dc, 0x007, 0x020, 0x35a, 0x006, 0x020,
+ 0x0a4, 0x00b, 0x020, 0x38e, 0x006, 0x020, 0x011, 0x007,
+ 0x020, 0x016, 0x007, 0x020, 0x31e, 0x007, 0x020, 0x324,
+ 0x007, 0x020, 0x3b6, 0x007, 0x020, 0x02a, 0x007, 0x020,
+ 0x041, 0x007, 0x020, 0x060, 0x007, 0x020, 0x06d, 0x007,
+ 0x020, 0x072, 0x007, 0x020, 0x2d1, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x252, 0x007, 0x020, 0x293, 0x007,
+ 0x020, 0x2d8, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x16f, 0x007, 0x020, 0x185, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x30e, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x077, 0x007, 0x020, 0x0b1, 0x007, 0x020,
+ 0x103, 0x007, 0x020, 0x1c5, 0x008, 0x020, 0x1dc, 0x007,
+ 0x020, 0x168, 0x007, 0x287, 0x264, 0x004, 0x001, 0x049,
+ 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020, 0x33b, 0x18e, 0x00a,
+ 0x013, 0x044, 0x04c, 0x004, 0x3a4, 0x001, 0x001, 0x014,
+ 0x364, 0x052, 0x001, 0x009, 0x093, 0x044, 0x0bc, 0x004,
+ 0x000, 0x08e, 0x3c4, 0x2ff, 0x053, 0x3c4, 0x030, 0x027,
+ 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052, 0x000, 0x081, 0x043,
+ 0x327, 0x33b, 0x020, 0x1de, 0x007, 0x2e4, 0x080, 0x009,
+ 0x00d, 0x3d5, 0x220, 0x33c, 0x100, 0x009, 0x06e, 0x044,
+ 0x3b6, 0x003, 0x000, 0x06c, 0x3d5, 0x120, 0x33c, 0x100,
+ 0x009, 0x022, 0x044, 0x066, 0x004, 0x000, 0x061, 0x384,
+ 0x003, 0x3c4, 0x3df, 0x33b, 0x262, 0x003, 0x001, 0x3d9,
+ 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef, 0x027, 0x2e4, 0x040,
+ 0x009, 0x3d5, 0x100, 0x009, 0x007, 0x044, 0x063, 0x004,
+ 0x000, 0x046, 0x364, 0x200, 0x004, 0x029, 0x1dc, 0x007,
+ 0x0c0, 0x2d8, 0x111, 0x001, 0x028, 0x3c5, 0x002, 0x33c,
+ 0x3c5, 0x001, 0x33b, 0x344, 0x21f, 0x33c, 0x287, 0x19b,
+ 0x247, 0x361, 0x287, 0x183, 0x19b, 0x247, 0x360, 0x0c0,
+ 0x35e, 0x2e4, 0x020, 0x001, 0x004, 0x0c5, 0x35e, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x289, 0x384,
+ 0x010, 0x245, 0x33c, 0x044, 0x001, 0x004, 0x364, 0x33c,
+ 0x002, 0x001, 0x005, 0x3c5, 0x004, 0x200, 0x364, 0x33c,
+ 0x020, 0x009, 0x005, 0x044, 0x097, 0x004, 0x020, 0x181,
+ 0x006, 0x287, 0x19b, 0x11d, 0x001, 0x002, 0x287, 0x1a3,
+ 0x1b3, 0x19b, 0x180, 0x390, 0x1d7, 0x040, 0x199, 0x390,
+ 0x008, 0x040, 0x045, 0x000, 0x021, 0x000, 0x073, 0x000,
+ 0x05f, 0x000, 0x023, 0x000, 0x025, 0x000, 0x033, 0x000,
+ 0x035, 0x000, 0x037, 0x000, 0x041, 0x000, 0x01f, 0x000,
+ 0x021, 0x000, 0x023, 0x000, 0x031, 0x000, 0x082, 0x000,
+ 0x002, 0x020, 0x1dc, 0x007, 0x044, 0x378, 0x003, 0x3c4,
+ 0x3fd, 0x233, 0x000, 0x072, 0x2c7, 0x363, 0x000, 0x06e,
+ 0x2c7, 0x362, 0x000, 0x06a, 0x2c7, 0x341, 0x000, 0x066,
+ 0x2c7, 0x33f, 0x000, 0x062, 0x2c7, 0x340, 0x000, 0x05e,
+ 0x2c7, 0x344, 0x000, 0x05a, 0x2c7, 0x342, 0x000, 0x056,
+ 0x2c7, 0x343, 0x000, 0x052, 0x2c7, 0x341, 0x2c7, 0x33f,
+ 0x2c7, 0x340, 0x000, 0x04a, 0x307, 0x340, 0x300, 0x341,
+ 0x300, 0x33f, 0x282, 0x007, 0x006, 0x2c7, 0x23e, 0x000,
+ 0x03d, 0x043, 0x0a1, 0x020, 0x1de, 0x007, 0x287, 0x384,
+ 0x003, 0x181, 0x3c4, 0x3f3, 0x052, 0x245, 0x052, 0x3a4,
+ 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5, 0x33c, 0x000, 0x026,
+ 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff, 0x352, 0x3c7, 0x3ff,
+ 0x353, 0x1d3, 0x353, 0x0cf, 0x354, 0x1cb, 0x354, 0x3c5,
+ 0x0c0, 0x354, 0x3c9, 0x222, 0x355, 0x3c5, 0x001, 0x33c,
+ 0x0c0, 0x356, 0x1b0, 0x00a, 0x006, 0x081, 0x18a, 0x247,
+ 0x356, 0x1ab, 0x2c7, 0x357, 0x020, 0x181, 0x006, 0x287,
+ 0x001, 0x025, 0x197, 0x3a4, 0x00f, 0x262, 0x003, 0x001,
+ 0x03d, 0x131, 0x001, 0x020, 0x131, 0x001, 0x021, 0x131,
+ 0x001, 0x022, 0x131, 0x001, 0x025, 0x131, 0x001, 0x026,
+ 0x131, 0x001, 0x027, 0x132, 0x182, 0x390, 0x377, 0x280,
+ 0x04c, 0x042, 0x020, 0x1de, 0x007, 0x044, 0x361, 0x003,
+ 0x000, 0x3d4, 0x327, 0x373, 0x000, 0x3f5, 0x327, 0x372,
+ 0x000, 0x3f1, 0x327, 0x2b3, 0x322, 0x2b2, 0x000, 0x3eb,
+ 0x327, 0x374, 0x000, 0x3e7, 0x327, 0x375, 0x000, 0x3e3,
+ 0x327, 0x376, 0x000, 0x3df, 0x307, 0x369, 0x282, 0x272,
+ 0x369, 0x00e, 0x003, 0x108, 0x04c, 0x000, 0x3d4, 0x364,
+ 0x33c, 0x001, 0x001, 0x01b, 0x364, 0x33b, 0x020, 0x009,
+ 0x016, 0x0a0, 0x307, 0x04f, 0x181, 0x00a, 0x003, 0x0a4,
+ 0x100, 0x00b, 0x004, 0x3a5, 0x080, 0x307, 0x050, 0x194,
+ 0x384, 0x040, 0x225, 0x2c7, 0x392, 0x081, 0x189, 0x314,
+ 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x027, 0x364, 0x33c, 0x020, 0x001, 0x007,
+ 0x347, 0x21d, 0x35a, 0x000, 0x018, 0x347, 0x050, 0x368,
+ 0x3b7, 0x201, 0x1a5, 0x140, 0x04f, 0x00b, 0x00b, 0x364,
+ 0x052, 0x020, 0x001, 0x3f3, 0x2c5, 0x33c, 0x000, 0x005,
+ 0x2a8, 0x2c4, 0x33c, 0x081, 0x189, 0x304, 0x053, 0x001,
+ 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7,
+ 0x027, 0x3c5, 0x080, 0x053, 0x3c4, 0x37f, 0x053, 0x364,
+ 0x33b, 0x020, 0x009, 0x00a, 0x364, 0x33c, 0x020, 0x009,
+ 0x005, 0x044, 0x132, 0x004, 0x020, 0x08d, 0x006, 0x2c7,
+ 0x231, 0x2e4, 0x100, 0x001, 0x045, 0x2e4, 0x001, 0x001,
+ 0x020, 0x3b7, 0x213, 0x084, 0x044, 0x1a9, 0x000, 0x3b7,
+ 0x250, 0x084, 0x044, 0x1a9, 0x000, 0x399, 0x305, 0x3a7,
+ 0x022, 0x044, 0x1a9, 0x000, 0x044, 0x322, 0x001, 0x3c5,
+ 0x002, 0x024, 0x3c5, 0x008, 0x292, 0x000, 0x020, 0x3c4,
+ 0x3fd, 0x024, 0x364, 0x234, 0x002, 0x001, 0x009, 0x3a7,
+ 0x022, 0x399, 0x305, 0x044, 0x1b7, 0x000, 0x3b7, 0x213,
+ 0x084, 0x044, 0x1b7, 0x000, 0x3b7, 0x250, 0x084, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x007, 0x292, 0x044, 0x06e, 0x003,
+ 0x307, 0x231, 0x274, 0x200, 0x001, 0x01b, 0x274, 0x010,
+ 0x001, 0x00b, 0x387, 0x020, 0x245, 0x021, 0x185, 0x245,
+ 0x024, 0x000, 0x00b, 0x387, 0x3df, 0x244, 0x021, 0x185,
+ 0x385, 0x0ff, 0x244, 0x024, 0x044, 0x03a, 0x00a, 0x045,
+ 0x327, 0x230, 0x364, 0x233, 0x004, 0x001, 0x003, 0x045,
+ 0x0c2, 0x22a, 0x347, 0x275, 0x270, 0x0c0, 0x27b, 0x264,
+ 0x010, 0x009, 0x00a, 0x0c1, 0x22a, 0x347, 0x274, 0x270,
+ 0x3d7, 0x200, 0x27b, 0x327, 0x230, 0x264, 0x004, 0x001,
+ 0x012, 0x347, 0x277, 0x270, 0x264, 0x010, 0x009, 0x007,
+ 0x347, 0x276, 0x270, 0x384, 0x00f, 0x112, 0x1b7, 0x000,
+ 0x004, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x384, 0x00f, 0x262,
+ 0x003, 0x001, 0x00b, 0x101, 0x3c2, 0x100, 0x270, 0x34a,
+ 0x225, 0x225, 0x000, 0x3f3, 0x044, 0x237, 0x00a, 0x044,
+ 0x35f, 0x009, 0x364, 0x22a, 0x002, 0x001, 0x006, 0x347,
+ 0x270, 0x272, 0x045, 0x347, 0x270, 0x271, 0x045, 0x044,
+ 0x2f6, 0x002, 0x009, 0x015, 0x307, 0x241, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x3d7, 0x200, 0x27b, 0x044, 0x35f, 0x009, 0x347,
+ 0x241, 0x226, 0x0c1, 0x22a, 0x020, 0x032, 0x009, 0x044,
+ 0x2f6, 0x002, 0x009, 0x014, 0x307, 0x243, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x0c0, 0x27b, 0x044, 0x35f, 0x009, 0x347, 0x243,
+ 0x226, 0x0c2, 0x22a, 0x020, 0x032, 0x009, 0x327, 0x226,
+ 0x2c7, 0x225, 0x3c4, 0x0ff, 0x225, 0x1b7, 0x3a4, 0x007,
+ 0x001, 0x00d, 0x34a, 0x225, 0x225, 0x002, 0x008, 0x131,
+ 0x009, 0x3fa, 0x044, 0x237, 0x00a, 0x045, 0x0c1, 0x26f,
+ 0x000, 0x050, 0x327, 0x01d, 0x2e4, 0x380, 0x009, 0x00a,
+ 0x13a, 0x00b, 0x00a, 0x3c5, 0x008, 0x233, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x233, 0x327, 0x01d, 0x021, 0x2ed, 0x009,
+ 0x364, 0x020, 0x004, 0x009, 0x01a, 0x309, 0x232, 0x00b,
+ 0x009, 0x347, 0x270, 0x271, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x271, 0x270, 0x044, 0x270, 0x009, 0x001, 0x04c,
+ 0x347, 0x270, 0x271, 0x000, 0x018, 0x309, 0x232, 0x001,
+ 0x009, 0x347, 0x270, 0x272, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x272, 0x270, 0x044, 0x270, 0x009, 0x001, 0x089,
+ 0x347, 0x270, 0x272, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x364, 0x26f, 0x002, 0x009, 0x005, 0x3c4, 0x3fe, 0x200,
+ 0x364, 0x020, 0x005, 0x029, 0x2e4, 0x002, 0x364, 0x020,
+ 0x010, 0x009, 0x0e9, 0x364, 0x020, 0x100, 0x009, 0x12b,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3f7, 0x233, 0x364, 0x020,
+ 0x001, 0x009, 0x00c, 0x364, 0x020, 0x004, 0x009, 0x05d,
+ 0x000, 0x3d8, 0x3c5, 0x080, 0x020, 0x364, 0x233, 0x006,
+ 0x001, 0x031, 0x081, 0x189, 0x304, 0x232, 0x009, 0x017,
+ 0x347, 0x271, 0x274, 0x080, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x085, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x081,
+ 0x189, 0x245, 0x232, 0x000, 0x0c8, 0x347, 0x271, 0x276,
+ 0x081, 0x189, 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197,
+ 0x384, 0x007, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x000,
+ 0x005, 0x044, 0x3f3, 0x008, 0x3b7, 0x200, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x203, 0x387, 0x3f7, 0x044,
+ 0x1b8, 0x000, 0x307, 0x020, 0x385, 0x042, 0x384, 0x3ce,
+ 0x247, 0x020, 0x044, 0x044, 0x001, 0x000, 0x383, 0x0a1,
+ 0x1aa, 0x2c5, 0x020, 0x364, 0x233, 0x006, 0x001, 0x039,
+ 0x081, 0x189, 0x304, 0x232, 0x009, 0x017, 0x347, 0x272,
+ 0x275, 0x080, 0x364, 0x233, 0x004, 0x009, 0x003, 0x095,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245,
+ 0x232, 0x000, 0x0ce, 0x347, 0x272, 0x277, 0x081, 0x189,
+ 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197, 0x364, 0x233,
+ 0x002, 0x001, 0x007, 0x193, 0x384, 0x007, 0x385, 0x010,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x000, 0x008, 0x347,
+ 0x272, 0x270, 0x044, 0x013, 0x009, 0x081, 0x18a, 0x208,
+ 0x244, 0x232, 0x0a3, 0x387, 0x3f7, 0x044, 0x1b8, 0x000,
+ 0x307, 0x020, 0x0a4, 0x1a7, 0x3a5, 0x008, 0x285, 0x3b7,
+ 0x304, 0x2a8, 0x284, 0x247, 0x020, 0x0a0, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x044, 0x2f6, 0x002, 0x009, 0x00f,
+ 0x364, 0x2cf, 0x080, 0x009, 0x00a, 0x0c3, 0x26f, 0x044,
+ 0x1b6, 0x001, 0x020, 0x2e4, 0x002, 0x044, 0x051, 0x001,
+ 0x000, 0x308, 0x3c4, 0x3f7, 0x233, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x083, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x3d7, 0x200, 0x27b, 0x0c0, 0x271,
+ 0x3b7, 0x300, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x221, 0x091, 0x189, 0x044, 0x1a9, 0x000, 0x3b7, 0x201,
+ 0x387, 0x0a6, 0x044, 0x1bd, 0x000, 0x3c5, 0x021, 0x020,
+ 0x3c4, 0x3ed, 0x020, 0x3b7, 0x200, 0x387, 0x020, 0x044,
+ 0x020, 0x00a, 0x009, 0x32a, 0x347, 0x271, 0x270, 0x000,
+ 0x055, 0x3a7, 0x071, 0x307, 0x21d, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x233, 0x044, 0x2f6, 0x002, 0x009, 0x007,
+ 0x0c1, 0x2d3, 0x3c5, 0x100, 0x200, 0x081, 0x18a, 0x245,
+ 0x232, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x093, 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x0c0,
+ 0x27b, 0x0c0, 0x272, 0x3b7, 0x300, 0x387, 0x020, 0x044,
+ 0x1a9, 0x000, 0x0a1, 0x397, 0x205, 0x247, 0x27f, 0x044,
+ 0x1bd, 0x000, 0x3d5, 0x204, 0x020, 0x3c4, 0x2ff, 0x020,
+ 0x0a0, 0x387, 0x021, 0x184, 0x044, 0x020, 0x00a, 0x009,
+ 0x336, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3fe, 0x200, 0x3c4, 0x3fe, 0x26f, 0x344, 0x21f, 0x232,
+ 0x0c0, 0x020, 0x0a3, 0x1ab, 0x2c5, 0x020, 0x044, 0x20d,
+ 0x000, 0x044, 0x2f3, 0x000, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x0a9, 0x1ab, 0x09f, 0x18a, 0x208, 0x304,
+ 0x055, 0x225, 0x044, 0x394, 0x00a, 0x307, 0x055, 0x19b,
+ 0x384, 0x003, 0x227, 0x1a3, 0x225, 0x044, 0x0a4, 0x00b,
+ 0x0a7, 0x020, 0x022, 0x008, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x006, 0x387, 0x07f, 0x000, 0x013, 0x399, 0x240,
+ 0x385, 0x07f, 0x306, 0x270, 0x001, 0x06c, 0x399, 0x240,
+ 0x327, 0x270, 0x3a4, 0x380, 0x226, 0x001, 0x015, 0x327,
+ 0x220, 0x324, 0x232, 0x009, 0x019, 0x364, 0x232, 0x001,
+ 0x001, 0x02b, 0x327, 0x01d, 0x307, 0x01b, 0x190, 0x282,
+ 0x003, 0x041, 0x307, 0x01b, 0x191, 0x380, 0x010, 0x282,
+ 0x003, 0x034, 0x000, 0x019, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x00a, 0x327, 0x270, 0x384, 0x07f, 0x3a4, 0x07f,
+ 0x000, 0x008, 0x327, 0x270, 0x3a4, 0x07f, 0x001, 0x005,
+ 0x286, 0x001, 0x02f, 0x307, 0x270, 0x384, 0x07f, 0x386,
+ 0x07f, 0x009, 0x00a, 0x387, 0x080, 0x382, 0x07f, 0x240,
+ 0x270, 0x000, 0x01c, 0x141, 0x270, 0x364, 0x270, 0x07f,
+ 0x009, 0x015, 0x000, 0x013, 0x3c0, 0x040, 0x270, 0x000,
+ 0x00e, 0x307, 0x270, 0x196, 0x384, 0x00f, 0x266, 0x009,
+ 0x001, 0x008, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009,
+ 0x045, 0x309, 0x232, 0x00b, 0x013, 0x347, 0x271, 0x270,
+ 0x374, 0x27b, 0x200, 0x009, 0x005, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x020, 0x0b7, 0x009, 0x327, 0x232,
+ 0x1a2, 0x023, 0x0b7, 0x009, 0x307, 0x270, 0x196, 0x119,
+ 0x021, 0x0b7, 0x009, 0x307, 0x232, 0x181, 0x023, 0x0b7,
+ 0x009, 0x364, 0x270, 0x080, 0x029, 0x0b7, 0x009, 0x083,
+ 0x18c, 0x245, 0x232, 0x3c4, 0x3f7, 0x233, 0x3c4, 0x380,
+ 0x270, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x09c, 0x009, 0x000,
+ 0x000, 0x000, 0x002, 0x000, 0x00a, 0x000, 0x02a, 0x000,
+ 0x0aa, 0x002, 0x0aa, 0x00a, 0x0aa, 0x02a, 0x0aa, 0x06a,
+ 0x0aa, 0x07a, 0x0aa, 0x07e, 0x0aa, 0x07f, 0x0aa, 0x07f,
+ 0x0ba, 0x07f, 0x0be, 0x07f, 0x0fe, 0x07f, 0x0ff, 0x007,
+ 0x000, 0x017, 0x001, 0x027, 0x007, 0x037, 0x00f, 0x047,
+ 0x03f, 0x080, 0x0ff, 0x364, 0x270, 0x07f, 0x307, 0x270,
+ 0x384, 0x07f, 0x009, 0x00d, 0x089, 0x186, 0x306, 0x270,
+ 0x001, 0x007, 0x3c5, 0x001, 0x232, 0x000, 0x005, 0x3c4,
+ 0x3fe, 0x232, 0x347, 0x270, 0x27c, 0x3c4, 0x007, 0x27c,
+ 0x374, 0x27b, 0x200, 0x001, 0x00a, 0x364, 0x233, 0x008,
+ 0x001, 0x005, 0x044, 0x0fe, 0x00a, 0x0a3, 0x325, 0x27b,
+ 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x305, 0x27c, 0x044,
+ 0x1bd, 0x000, 0x307, 0x270, 0x192, 0x384, 0x00f, 0x3b9,
+ 0x200, 0x324, 0x233, 0x001, 0x014, 0x229, 0x387, 0x009,
+ 0x189, 0x395, 0x333, 0x280, 0x051, 0x2c7, 0x27d, 0x1c7,
+ 0x27d, 0x101, 0x051, 0x2c5, 0x27d, 0x000, 0x011, 0x0c0,
+ 0x27d, 0x384, 0x00f, 0x001, 0x00b, 0x349, 0x27d, 0x27d,
+ 0x3c5, 0x001, 0x27d, 0x111, 0x000, 0x3f7, 0x0a2, 0x325,
+ 0x27b, 0x307, 0x27d, 0x044, 0x1bd, 0x000, 0x374, 0x27b,
+ 0x200, 0x001, 0x02c, 0x347, 0x270, 0x27a, 0x3c4, 0x07f,
+ 0x27a, 0x0c0, 0x279, 0x387, 0x009, 0x189, 0x395, 0x353,
+ 0x051, 0x041, 0x101, 0x051, 0x2c7, 0x279, 0x101, 0x043,
+ 0x322, 0x27a, 0x00b, 0x004, 0x000, 0x3f4, 0x1c7, 0x279,
+ 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x0ff,
+ 0x305, 0x279, 0x044, 0x1bd, 0x000, 0x327, 0x220, 0x324,
+ 0x232, 0x009, 0x022, 0x0c0, 0x27e, 0x307, 0x270, 0x197,
+ 0x00a, 0x004, 0x000, 0x005, 0x3c5, 0x018, 0x27e, 0x0a4,
+ 0x222, 0x2c5, 0x27e, 0x0a4, 0x325, 0x27b, 0x044, 0x192,
+ 0x000, 0x384, 0x3e0, 0x3c4, 0x01f, 0x27e, 0x305, 0x27e,
+ 0x044, 0x1bd, 0x000, 0x045, 0x044, 0x1a9, 0x000, 0x083,
+ 0x18c, 0x208, 0x244, 0x232, 0x3c5, 0x001, 0x00c, 0x0a3,
+ 0x325, 0x27b, 0x088, 0x044, 0x1a9, 0x000, 0x387, 0x3f8,
+ 0x044, 0x1b8, 0x000, 0x020, 0x2f6, 0x002, 0x3c7, 0x020,
+ 0x280, 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x247, 0x247,
+ 0x151, 0x280, 0x001, 0x016, 0x0a1, 0x044, 0x182, 0x000,
+ 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x306, 0x247, 0x001,
+ 0x3f1, 0x0a2, 0x364, 0x22b, 0x003, 0x001, 0x003, 0x0a5,
+ 0x020, 0x182, 0x000, 0x1f6, 0x185, 0x1f5, 0x129, 0x1f4,
+ 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff, 0x1f6, 0x189, 0x1f5,
+ 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce, 0x123, 0x0f3, 0x0d2,
+ 0x0ff, 0x364, 0x233, 0x020, 0x009, 0x051, 0x307, 0x272,
+ 0x196, 0x384, 0x00f, 0x0c9, 0x286, 0x242, 0x286, 0x397,
+ 0x00a, 0x189, 0x395, 0x05f, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x00a, 0x051, 0x2c7, 0x284, 0x101, 0x051,
+ 0x2c7, 0x288, 0x3a4, 0x07f, 0x040, 0x307, 0x272, 0x384,
+ 0x07f, 0x222, 0x042, 0x00b, 0x005, 0x101, 0x000, 0x3ed,
+ 0x347, 0x284, 0x285, 0x3c4, 0x00f, 0x284, 0x1d3, 0x285,
+ 0x1d6, 0x288, 0x342, 0x288, 0x286, 0x003, 0x008, 0x156,
+ 0x285, 0x151, 0x286, 0x00b, 0x3fc, 0x362, 0x285, 0x003,
+ 0x003, 0x004, 0x0c3, 0x285, 0x307, 0x285, 0x0a1, 0x184,
+ 0x305, 0x284, 0x044, 0x1bd, 0x000, 0x045, 0x3db, 0x071,
+ 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c, 0x176, 0x1bf, 0x00d,
+ 0x007, 0x006, 0x003, 0x002, 0x005, 0x0b3, 0x156, 0x1fa,
+ 0x27f, 0x080, 0x09e, 0x14b, 0x1f0, 0x27f, 0x080, 0x08b,
+ 0x140, 0x1e9, 0x27f, 0x081, 0x138, 0x1e3, 0x27f, 0x00f,
+ 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080, 0x0a3, 0x145, 0x1e7,
+ 0x27f, 0x080, 0x095, 0x13d, 0x1e0, 0x27f, 0x08b, 0x136,
+ 0x1db, 0x27f, 0x0c5, 0x28b, 0x347, 0x271, 0x281, 0x3c4,
+ 0x07f, 0x281, 0x0c5, 0x282, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x0c2, 0x282, 0x044, 0x1bc, 0x00a, 0x347, 0x283,
+ 0x289, 0x307, 0x241, 0x364, 0x233, 0x002, 0x001, 0x004,
+ 0x307, 0x225, 0x384, 0x0ff, 0x242, 0x283, 0x00b, 0x056,
+ 0x0c3, 0x28b, 0x302, 0x289, 0x397, 0x00a, 0x189, 0x395,
+ 0x0ca, 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x004,
+ 0x051, 0x2c7, 0x28c, 0x2a9, 0x121, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003, 0x007,
+ 0x001, 0x005, 0x101, 0x000, 0x3ed, 0x1d5, 0x28c, 0x347,
+ 0x241, 0x281, 0x364, 0x233, 0x002, 0x001, 0x005, 0x347,
+ 0x225, 0x281, 0x3c4, 0x0ff, 0x281, 0x342, 0x289, 0x281,
+ 0x347, 0x28c, 0x282, 0x366, 0x282, 0x00f, 0x009, 0x006,
+ 0x1c0, 0x281, 0x0c1, 0x282, 0x044, 0x1bc, 0x00a, 0x001,
+ 0x00d, 0x141, 0x28b, 0x362, 0x28b, 0x03f, 0x001, 0x00d,
+ 0x151, 0x283, 0x000, 0x3f5, 0x362, 0x28b, 0x00a, 0x003,
+ 0x004, 0x0ca, 0x28b, 0x397, 0x00a, 0x189, 0x395, 0x0d8,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x013, 0x327,
+ 0x24c, 0x131, 0x003, 0x005, 0x105, 0x000, 0x3fc, 0x051,
+ 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121, 0x040, 0x307, 0x271,
+ 0x384, 0x07f, 0x282, 0x042, 0x003, 0x005, 0x101, 0x000,
+ 0x3f0, 0x1d6, 0x28a, 0x387, 0x00a, 0x189, 0x395, 0x0d3,
+ 0x300, 0x28a, 0x050, 0x247, 0x28a, 0x30a, 0x28b, 0x3b7,
+ 0x201, 0x184, 0x305, 0x28a, 0x044, 0x1bd, 0x000, 0x045,
+ 0x0c1, 0x283, 0x342, 0x282, 0x281, 0x003, 0x006, 0x141,
+ 0x283, 0x000, 0x3f9, 0x045, 0x000, 0x301, 0x103, 0x307,
+ 0x006, 0x10e, 0x21e, 0x33e, 0x13c, 0x038, 0x078, 0x1f8,
+ 0x0f0, 0x000, 0x001, 0x003, 0x002, 0x006, 0x187, 0x227,
+ 0x387, 0x00a, 0x189, 0x395, 0x1c8, 0x300, 0x227, 0x111,
+ 0x050, 0x247, 0x24c, 0x1d7, 0x24c, 0x384, 0x0ff, 0x285,
+ 0x044, 0x226, 0x00a, 0x364, 0x22a, 0x001, 0x001, 0x00e,
+ 0x3d5, 0x200, 0x232, 0x040, 0x1aa, 0x285, 0x3b7, 0x220,
+ 0x044, 0x1bd, 0x000, 0x042, 0x364, 0x22a, 0x002, 0x001,
+ 0x022, 0x040, 0x3a7, 0x073, 0x044, 0x192, 0x000, 0x384,
+ 0x31f, 0x044, 0x226, 0x00a, 0x1a4, 0x285, 0x3a7, 0x073,
+ 0x044, 0x1bd, 0x000, 0x3a7, 0x070, 0x044, 0x192, 0x000,
+ 0x19a, 0x18a, 0x043, 0x285, 0x3a7, 0x070, 0x044, 0x1bd,
+ 0x000, 0x045, 0x040, 0x387, 0x00a, 0x189, 0x395, 0x1d5,
+ 0x300, 0x278, 0x051, 0x042, 0x045, 0x00a, 0x058, 0x026,
+ 0x015, 0x044, 0x032, 0x0c0, 0x278, 0x0a0, 0x397, 0x00a,
+ 0x189, 0x395, 0x231, 0x280, 0x050, 0x247, 0x228, 0x3c4,
+ 0x00f, 0x228, 0x347, 0x228, 0x229, 0x040, 0x307, 0x278,
+ 0x001, 0x008, 0x340, 0x229, 0x228, 0x111, 0x000, 0x3fa,
+ 0x042, 0x193, 0x0c0, 0x227, 0x347, 0x225, 0x226, 0x141,
+ 0x227, 0x366, 0x227, 0x00e, 0x001, 0x00b, 0x342, 0x228,
+ 0x226, 0x007, 0x3f6, 0x140, 0x226, 0x001, 0x011, 0x121,
+ 0x2e2, 0x006, 0x009, 0x3cc, 0x141, 0x278, 0x366, 0x278,
+ 0x005, 0x009, 0x3c4, 0x0a1, 0x000, 0x006, 0x044, 0x1da,
+ 0x00a, 0x0a0, 0x045, 0x044, 0x333, 0x008, 0x020, 0x181,
+ 0x006, 0x345, 0x21d, 0x232, 0x364, 0x23f, 0x001, 0x029,
+ 0x2e4, 0x002, 0x2c7, 0x234, 0x399, 0x200, 0x245, 0x233,
+ 0x364, 0x234, 0x020, 0x009, 0x005, 0x208, 0x244, 0x233,
+ 0x083, 0x18c, 0x208, 0x244, 0x036, 0x2e4, 0x008, 0x001,
+ 0x007, 0x345, 0x220, 0x232, 0x000, 0x007, 0x081, 0x18b,
+ 0x208, 0x244, 0x232, 0x307, 0x235, 0x226, 0x3a4, 0x001,
+ 0x001, 0x010, 0x327, 0x234, 0x2e4, 0x001, 0x009, 0x041,
+ 0x041, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x043,
+ 0x0a3, 0x1ad, 0x3a5, 0x039, 0x326, 0x236, 0x001, 0x02f,
+ 0x3a7, 0x0bf, 0x1a7, 0x3a5, 0x039, 0x326, 0x236, 0x001,
+ 0x026, 0x327, 0x234, 0x307, 0x235, 0x286, 0x384, 0x002,
+ 0x001, 0x01d, 0x2e4, 0x002, 0x009, 0x051, 0x041, 0x0a1,
+ 0x1ad, 0x2c5, 0x024, 0x307, 0x024, 0x19a, 0x002, 0x005,
+ 0x2a8, 0x2c4, 0x024, 0x044, 0x3b5, 0x000, 0x044, 0x0f2,
+ 0x001, 0x044, 0x3f0, 0x000, 0x043, 0x000, 0x067, 0x364,
+ 0x026, 0x001, 0x009, 0x3c6, 0x041, 0x0a0, 0x044, 0x301,
+ 0x000, 0x364, 0x26f, 0x002, 0x021, 0x310, 0x00a, 0x044,
+ 0x290, 0x001, 0x0c1, 0x26f, 0x043, 0x307, 0x020, 0x193,
+ 0x384, 0x007, 0x110, 0x001, 0x00c, 0x364, 0x232, 0x010,
+ 0x009, 0x007, 0x044, 0x044, 0x001, 0x000, 0x3a3, 0x3c4,
+ 0x3fe, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x010, 0x020,
+ 0x3c4, 0x31f, 0x020, 0x000, 0x395, 0x364, 0x026, 0x002,
+ 0x009, 0x3c5, 0x041, 0x044, 0x003, 0x001, 0x043, 0x307,
+ 0x020, 0x197, 0x384, 0x007, 0x110, 0x001, 0x00f, 0x364,
+ 0x232, 0x020, 0x009, 0x00a, 0x044, 0x051, 0x001, 0x3c5,
+ 0x001, 0x027, 0x000, 0x3ab, 0x3c4, 0x3fc, 0x26f, 0x3c5,
+ 0x001, 0x200, 0x3c5, 0x100, 0x020, 0x087, 0x188, 0x208,
+ 0x244, 0x020, 0x000, 0x39b, 0x044, 0x11a, 0x001, 0x307,
+ 0x234, 0x306, 0x235, 0x384, 0x003, 0x021, 0x2e4, 0x002,
+ 0x347, 0x234, 0x235, 0x366, 0x26f, 0x002, 0x021, 0x2e4,
+ 0x002, 0x364, 0x234, 0x003, 0x021, 0x2e4, 0x002, 0x327,
+ 0x21c, 0x324, 0x020, 0x021, 0x385, 0x00a, 0x020, 0x247,
+ 0x002, 0x364, 0x23f, 0x001, 0x021, 0x38e, 0x00a, 0x020,
+ 0x247, 0x002, 0x345, 0x21d, 0x232, 0x020, 0x247, 0x002,
+ 0x2c7, 0x241, 0x307, 0x242, 0x286, 0x001, 0x092, 0x0c1,
+ 0x22a, 0x327, 0x242, 0x326, 0x241, 0x3a4, 0x0ff, 0x001,
+ 0x005, 0x3c4, 0x38f, 0x020, 0x347, 0x241, 0x242, 0x347,
+ 0x241, 0x226, 0x307, 0x21d, 0x304, 0x241, 0x001, 0x00a,
+ 0x0c3, 0x22a, 0x347, 0x241, 0x243, 0x044, 0x39f, 0x002,
+ 0x327, 0x241, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x300,
+ 0x230, 0x2c5, 0x230, 0x044, 0x099, 0x00b, 0x081, 0x18b,
+ 0x304, 0x226, 0x009, 0x00d, 0x364, 0x232, 0x100, 0x009,
+ 0x008, 0x081, 0x18a, 0x208, 0x044, 0x1b8, 0x000, 0x044,
+ 0x186, 0x001, 0x327, 0x226, 0x003, 0x032, 0x044, 0x2f6,
+ 0x002, 0x001, 0x019, 0x044, 0x237, 0x00a, 0x347, 0x241,
+ 0x226, 0x081, 0x306, 0x22a, 0x384, 0x003, 0x001, 0x005,
+ 0x347, 0x243, 0x226, 0x044, 0x032, 0x009, 0x002, 0x02d,
+ 0x000, 0x031, 0x044, 0x237, 0x00a, 0x009, 0x01c, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x001, 0x023,
+ 0x364, 0x233, 0x001, 0x000, 0x01e, 0x045, 0x044, 0x186,
+ 0x001, 0x3c5, 0x003, 0x22a, 0x347, 0x241, 0x243, 0x000,
+ 0x3c7, 0x3a4, 0x3ff, 0x001, 0x00e, 0x327, 0x21c, 0x324,
+ 0x020, 0x009, 0x006, 0x043, 0x3a7, 0x0ff, 0x041, 0x0c0,
+ 0x270, 0x044, 0x082, 0x00b, 0x3c4, 0x3fc, 0x22a, 0x327,
+ 0x21c, 0x324, 0x020, 0x009, 0x010, 0x307, 0x270, 0x003,
+ 0x005, 0x020, 0x181, 0x006, 0x043, 0x3a7, 0x0ff, 0x041,
+ 0x020, 0x181, 0x006, 0x045, 0x2c7, 0x243, 0x307, 0x250,
+ 0x286, 0x001, 0x3de, 0x0c2, 0x22a, 0x041, 0x044, 0x39f,
+ 0x002, 0x043, 0x2c7, 0x226, 0x3a4, 0x0ff, 0x2c7, 0x225,
+ 0x3c4, 0x0ff, 0x230, 0x1a7, 0x2c5, 0x230, 0x307, 0x21d,
+ 0x304, 0x243, 0x001, 0x013, 0x3c4, 0x0ff, 0x243, 0x347,
+ 0x243, 0x241, 0x347, 0x243, 0x242, 0x3c5, 0x003, 0x22a,
+ 0x044, 0x186, 0x001, 0x000, 0x005, 0x044, 0x191, 0x001,
+ 0x327, 0x243, 0x003, 0x394, 0x000, 0x362, 0x364, 0x232,
+ 0x004, 0x001, 0x008, 0x044, 0x232, 0x000, 0x3c4, 0x3fb,
+ 0x232, 0x364, 0x232, 0x008, 0x001, 0x008, 0x044, 0x326,
+ 0x000, 0x3c4, 0x3f7, 0x232, 0x045, 0x3b7, 0x221, 0x083,
+ 0x189, 0x044, 0x1a9, 0x000, 0x083, 0x020, 0x1b7, 0x000,
+ 0x2c7, 0x24d, 0x364, 0x233, 0x002, 0x001, 0x019, 0x3b9,
+ 0x381, 0x121, 0x081, 0x18a, 0x225, 0x304, 0x24d, 0x001,
+ 0x005, 0x044, 0x1ff, 0x00b, 0x140, 0x24d, 0x00b, 0x008,
+ 0x1a3, 0x044, 0x1ff, 0x00b, 0x000, 0x012, 0x307, 0x220,
+ 0x284, 0x001, 0x00d, 0x3c4, 0x33f, 0x232, 0x1b6, 0x3a4,
+ 0x0c0, 0x2c5, 0x232, 0x020, 0x18a, 0x00b, 0x327, 0x24d,
+ 0x364, 0x020, 0x070, 0x001, 0x00a, 0x399, 0x383, 0x101,
+ 0x326, 0x22b, 0x234, 0x001, 0x02e, 0x3c5, 0x080, 0x233,
+ 0x3d5, 0x200, 0x232, 0x364, 0x026, 0x001, 0x001, 0x00a,
+ 0x3c5, 0x004, 0x232, 0x044, 0x20d, 0x000, 0x307, 0x24d,
+ 0x307, 0x24d, 0x384, 0x007, 0x001, 0x09a, 0x113, 0x001,
+ 0x0a8, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8,
+ 0x103, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x021, 0x000,
+ 0x0af, 0x0a7, 0x1a7, 0x324, 0x020, 0x001, 0x014, 0x327,
+ 0x24d, 0x326, 0x22b, 0x397, 0x370, 0x364, 0x233, 0x002,
+ 0x001, 0x006, 0x087, 0x18b, 0x385, 0x070, 0x234, 0x001,
+ 0x02b, 0x3c5, 0x100, 0x233, 0x364, 0x026, 0x002, 0x001,
+ 0x008, 0x3c5, 0x008, 0x232, 0x044, 0x3b5, 0x000, 0x307,
+ 0x24d, 0x384, 0x070, 0x001, 0x091, 0x382, 0x030, 0x001,
+ 0x0a6, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x018, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x024,
+ 0x000, 0x0a4, 0x347, 0x24d, 0x22b, 0x364, 0x233, 0x006,
+ 0x001, 0x033, 0x307, 0x22b, 0x364, 0x233, 0x002, 0x001,
+ 0x023, 0x364, 0x233, 0x080, 0x001, 0x00a, 0x307, 0x24d,
+ 0x197, 0x384, 0x007, 0x044, 0x3a4, 0x008, 0x364, 0x233,
+ 0x100, 0x001, 0x00c, 0x307, 0x24d, 0x19b, 0x384, 0x007,
+ 0x385, 0x010, 0x044, 0x3a4, 0x008, 0x3c4, 0x27f, 0x233,
+ 0x000, 0x00b, 0x3c5, 0x003, 0x22a, 0x197, 0x044, 0x39c,
+ 0x008, 0x003, 0x298, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x021, 0x181, 0x006, 0x045, 0x3b7, 0x301,
+ 0x087, 0x044, 0x1b7, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x3f7, 0x021, 0x000, 0x36c, 0x3b7,
+ 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x105, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000,
+ 0x3c5, 0x008, 0x021, 0x344, 0x21f, 0x021, 0x3b7, 0x250,
+ 0x088, 0x044, 0x1a9, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1a9, 0x000, 0x000, 0x347, 0x3b7, 0x301, 0x387, 0x3c7,
+ 0x044, 0x1b8, 0x000, 0x3a7, 0x060, 0x083, 0x044, 0x1b7,
+ 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x024, 0x000, 0x36f, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x028, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x048, 0x024, 0x3a7, 0x060, 0x083, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1a9,
+ 0x000, 0x000, 0x351, 0x307, 0x22b, 0x284, 0x2a8, 0x2c4,
+ 0x24d, 0x245, 0x24d, 0x2a8, 0x045, 0x2f4, 0x200, 0x001,
+ 0x008, 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287,
+ 0x385, 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010,
+ 0x029, 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384,
+ 0x0ff, 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d,
+ 0x040, 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d,
+ 0x042, 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005,
+ 0x3c5, 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264,
+ 0x080, 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5,
+ 0x080, 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f,
+ 0x02d, 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff,
+ 0x1b8, 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021,
+ 0x3c4, 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a,
+ 0x020, 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x251,
+ 0x22e, 0x364, 0x026, 0x001, 0x021, 0x181, 0x006, 0x041,
+ 0x3a4, 0x01f, 0x001, 0x005, 0x044, 0x0d9, 0x001, 0x043,
+ 0x1b7, 0x001, 0x005, 0x044, 0x182, 0x000, 0x020, 0x181,
+ 0x006, 0x2e4, 0x002, 0x009, 0x01b, 0x3c4, 0x2ff, 0x232,
+ 0x120, 0x001, 0x012, 0x3d5, 0x300, 0x232, 0x044, 0x099,
+ 0x00b, 0x32a, 0x026, 0x00a, 0x008, 0x044, 0x20d, 0x000,
+ 0x044, 0x232, 0x000, 0x020, 0x181, 0x006, 0x042, 0x307,
+ 0x240, 0x20a, 0x040, 0x020, 0x181, 0x006, 0x289, 0x00a,
+ 0x004, 0x2c7, 0x256, 0x22e, 0x364, 0x026, 0x002, 0x021,
+ 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005, 0x044,
+ 0x0de, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044, 0x0e2,
+ 0x001, 0x020, 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009,
+ 0x007, 0x044, 0x340, 0x001, 0x247, 0x249, 0x054, 0x347,
+ 0x249, 0x248, 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018,
+ 0x364, 0x23f, 0x001, 0x001, 0x006, 0x245, 0x057, 0x000,
+ 0x005, 0x208, 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0,
+ 0x364, 0x23f, 0x002, 0x001, 0x009, 0x245, 0x057, 0x044,
+ 0x27d, 0x001, 0x000, 0x005, 0x208, 0x244, 0x057, 0x020,
+ 0x181, 0x006, 0x2c7, 0x246, 0x3b7, 0x300, 0x044, 0x192,
+ 0x000, 0x384, 0x33f, 0x327, 0x246, 0x3a4, 0x0c0, 0x285,
+ 0x3b7, 0x300, 0x044, 0x1bd, 0x000, 0x3b7, 0x200, 0x044,
+ 0x192, 0x000, 0x0a3, 0x1a9, 0x2a8, 0x284, 0x040, 0x307,
+ 0x246, 0x384, 0x030, 0x264, 0x010, 0x001, 0x012, 0x040,
+ 0x041, 0x3b7, 0x201, 0x044, 0x192, 0x000, 0x384, 0x3e0,
+ 0x385, 0x006, 0x044, 0x1bd, 0x000, 0x043, 0x042, 0x185,
+ 0x2a8, 0x284, 0x227, 0x042, 0x285, 0x3b7, 0x200, 0x044,
+ 0x1bd, 0x000, 0x307, 0x246, 0x384, 0x007, 0x044, 0x375,
+ 0x00b, 0x3a7, 0x073, 0x307, 0x246, 0x184, 0x00a, 0x014,
+ 0x044, 0x192, 0x000, 0x384, 0x3e2, 0x327, 0x246, 0x1b5,
+ 0x3a4, 0x01c, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000,
+ 0x000, 0x006, 0x081, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x3c5, 0x001, 0x00c, 0x187, 0x0a7, 0x1a7, 0x2a8,
+ 0x324, 0x00c, 0x225, 0x2c7, 0x00c, 0x045, 0x04e, 0x087,
+ 0x011, 0x054, 0x084, 0x020, 0x01e, 0x00f, 0x018, 0x01d,
+ 0x029, 0x00b, 0x01d, 0x02c, 0x331, 0x04e, 0x020, 0x0d3,
+ 0x000, 0x020, 0x167, 0x00b, 0x020, 0x3f6, 0x00a, 0x020,
+ 0x096, 0x00b, 0x020, 0x0d3, 0x000, 0x020, 0x0d3, 0x000,
+ 0x020, 0x00d, 0x00b, 0x020, 0x179, 0x00b, 0x0c1, 0x244,
+ 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200, 0x020, 0x0d3,
+ 0x000, 0x020, 0x2c3, 0x008, 0x04e, 0x04e, 0x045, 0x0c2,
+ 0x201, 0x045, 0x020, 0x3ed, 0x00a, 0x020, 0x256, 0x002,
+ 0x020, 0x256, 0x002, 0x020, 0x256, 0x002, 0x020, 0x23f,
+ 0x006, 0x020, 0x160, 0x00b, 0x020, 0x28d, 0x00b, 0x020,
+ 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007, 0x020,
+ 0x160, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x2d5, 0x006, 0x020, 0x326,
+ 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007, 0x020,
+ 0x35a, 0x006, 0x020, 0x177, 0x00b, 0x020, 0x38e, 0x006,
+ 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020, 0x31e,
+ 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007, 0x020,
+ 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060, 0x007,
+ 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020, 0x2d1,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252, 0x007,
+ 0x020, 0x293, 0x007, 0x020, 0x2fb, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007, 0x020,
+ 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007, 0x020,
+ 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5, 0x008,
+ 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287, 0x264,
+ 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020,
+ 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004, 0x3a4,
+ 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009, 0x093,
+ 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff, 0x053,
+ 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052,
+ 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de, 0x007,
+ 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c, 0x100,
+ 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c, 0x3d5,
+ 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066, 0x004,
+ 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b, 0x262,
+ 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef,
+ 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009, 0x007,
+ 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200, 0x004,
+ 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001, 0x028,
+ 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344, 0x21f,
+ 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183, 0x19b,
+ 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001, 0x004,
+ 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5, 0x080,
+ 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044, 0x001,
+ 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5, 0x004,
+ 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x097,
+ 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d, 0x001,
+ 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390, 0x1d7,
+ 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000, 0x021,
+ 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000, 0x025,
+ 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000, 0x041,
+ 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000, 0x031,
+ 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007, 0x044,
+ 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072, 0x2c7,
+ 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a, 0x2c7,
+ 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062, 0x2c7,
+ 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a, 0x2c7,
+ 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052, 0x2c7,
+ 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a, 0x307,
+ 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007, 0x006,
+ 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020, 0x1de,
+ 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3, 0x052,
+ 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5,
+ 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff,
+ 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf, 0x354,
+ 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222, 0x355,
+ 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a, 0x006,
+ 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357, 0x020,
+ 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4, 0x00f,
+ 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020, 0x131,
+ 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001, 0x025,
+ 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132, 0x182,
+ 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de, 0x007,
+ 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373, 0x000,
+ 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3, 0x322,
+ 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7, 0x327,
+ 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df, 0x307,
+ 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108, 0x04c,
+ 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b, 0x364,
+ 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f, 0x181,
+ 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5, 0x080,
+ 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7, 0x392,
+ 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5, 0x008,
+ 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364, 0x33c,
+ 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000, 0x018,
+ 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140, 0x04f,
+ 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3, 0x2c5,
+ 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081, 0x189,
+ 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000,
+ 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053, 0x3c4,
+ 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a, 0x364,
+ 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004, 0x020,
+ 0x08d, 0x006, 0x364, 0x026, 0x010, 0x009, 0x0e8, 0x0c9,
+ 0x2ce, 0x0c8, 0x30f, 0x362, 0x2d2, 0x002, 0x009, 0x004,
+ 0x0ce, 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x0af, 0x0c4,
+ 0x303, 0x364, 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303,
+ 0x0c1, 0x310, 0x364, 0x30f, 0x001, 0x009, 0x004, 0x152,
+ 0x310, 0x397, 0x2b2, 0x300, 0x30f, 0x04c, 0x2c7, 0x305,
+ 0x397, 0x2aa, 0x300, 0x303, 0x04c, 0x2c0, 0x305, 0x342,
+ 0x310, 0x305, 0x397, 0x2a1, 0x300, 0x303, 0x327, 0x305,
+ 0x04d, 0x044, 0x1bf, 0x003, 0x08c, 0x364, 0x30f, 0x002,
+ 0x001, 0x003, 0x108, 0x364, 0x30f, 0x004, 0x001, 0x003,
+ 0x101, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x364, 0x2cf, 0x002, 0x001, 0x014, 0x309, 0x013, 0x34b,
+ 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b,
+ 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336, 0x014, 0x397,
+ 0x2a1, 0x300, 0x303, 0x04c, 0x320, 0x310, 0x04d, 0x2e4,
+ 0x100, 0x001, 0x004, 0x000, 0x012, 0x044, 0x3f0, 0x002,
+ 0x044, 0x30e, 0x002, 0x140, 0x00f, 0x009, 0x008, 0x307,
+ 0x00e, 0x302, 0x2cd, 0x00a, 0x3e4, 0x0c4, 0x303, 0x364,
+ 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303, 0x397, 0x2a1,
+ 0x300, 0x303, 0x04c, 0x2c7, 0x305, 0x397, 0x2aa, 0x300,
+ 0x303, 0x04c, 0x2c2, 0x305, 0x342, 0x310, 0x305, 0x397,
+ 0x2b2, 0x300, 0x30f, 0x327, 0x305, 0x04d, 0x362, 0x2d2,
+ 0x002, 0x001, 0x00f, 0x364, 0x2cb, 0x002, 0x009, 0x00a,
+ 0x307, 0x30f, 0x118, 0x390, 0x2b2, 0x327, 0x305, 0x04d,
+ 0x141, 0x30f, 0x000, 0x350, 0x044, 0x275, 0x00b, 0x0c0,
+ 0x2a9, 0x0c0, 0x303, 0x362, 0x303, 0x008, 0x00f, 0x017,
+ 0x397, 0x2b2, 0x300, 0x303, 0x04c, 0x2a7, 0x362, 0x303,
+ 0x004, 0x00f, 0x006, 0x2c0, 0x2a9, 0x000, 0x004, 0x2c2,
+ 0x2a9, 0x141, 0x303, 0x000, 0x3e8, 0x30a, 0x2a9, 0x20a,
+ 0x24a, 0x2a9, 0x044, 0x022, 0x009, 0x3c4, 0x3ef, 0x026,
+ 0x045, 0x307, 0x2b2, 0x300, 0x2b3, 0x247, 0x2a1, 0x362,
+ 0x2a1, 0x000, 0x006, 0x004, 0x141, 0x2a1, 0x30a, 0x2a1,
+ 0x300, 0x2aa, 0x247, 0x2a1, 0x307, 0x2b4, 0x300, 0x2b5,
+ 0x247, 0x2a2, 0x362, 0x2a2, 0x000, 0x006, 0x004, 0x141,
+ 0x2a2, 0x30a, 0x2a2, 0x300, 0x2ab, 0x247, 0x2a2, 0x307,
+ 0x2b6, 0x300, 0x2b7, 0x247, 0x2a3, 0x362, 0x2a3, 0x000,
+ 0x006, 0x004, 0x141, 0x2a3, 0x30a, 0x2a3, 0x300, 0x2ac,
+ 0x247, 0x2a3, 0x307, 0x2b8, 0x300, 0x2b9, 0x247, 0x2a4,
+ 0x362, 0x2a4, 0x000, 0x006, 0x004, 0x141, 0x2a4, 0x30a,
+ 0x2a4, 0x300, 0x2ad, 0x247, 0x2a4, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x044, 0x3f0, 0x002, 0x045, 0x0c0,
+ 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x00c, 0x397, 0x2b2,
+ 0x300, 0x30f, 0x0a0, 0x04d, 0x141, 0x30f, 0x000, 0x3f3,
+ 0x045, 0x364, 0x2d4, 0x002, 0x001, 0x003, 0x045, 0x0c7,
+ 0x2ce, 0x347, 0x290, 0x2fb, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003, 0x309,
+ 0x013, 0x34b, 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336,
+ 0x014, 0x307, 0x2ae, 0x300, 0x2ba, 0x114, 0x247, 0x2a5,
+ 0x307, 0x2af, 0x300, 0x2bd, 0x104, 0x247, 0x2a6, 0x3c7,
+ 0x3ff, 0x301, 0x151, 0x2a6, 0x141, 0x2a5, 0x364, 0x2a6,
+ 0x100, 0x001, 0x004, 0x000, 0x01a, 0x364, 0x2a5, 0x100,
+ 0x001, 0x004, 0x000, 0x013, 0x044, 0x014, 0x003, 0x044,
+ 0x30e, 0x002, 0x307, 0x2cd, 0x0c0, 0x336, 0x302, 0x00e,
+ 0x343, 0x00f, 0x336, 0x00a, 0x3df, 0x3ce, 0x3ff, 0x2fe,
+ 0x141, 0x2a6, 0x00b, 0x004, 0x0c0, 0x2a6, 0x151, 0x2a5,
+ 0x00b, 0x004, 0x0c0, 0x2a5, 0x362, 0x2a6, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a6, 0x362, 0x2a5, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a5, 0x044, 0x014, 0x003, 0x309,
+ 0x303, 0x242, 0x290, 0x0c0, 0x304, 0x362, 0x304, 0x003,
+ 0x00f, 0x02c, 0x340, 0x303, 0x290, 0x362, 0x290, 0x000,
+ 0x00f, 0x004, 0x0c0, 0x290, 0x362, 0x290, 0x0ff, 0x006,
+ 0x004, 0x000, 0x01b, 0x044, 0x034, 0x003, 0x044, 0x1cc,
+ 0x003, 0x044, 0x30e, 0x002, 0x0c0, 0x2ff, 0x044, 0x195,
+ 0x003, 0x140, 0x2ff, 0x001, 0x005, 0x347, 0x290, 0x305,
+ 0x141, 0x304, 0x000, 0x3d3, 0x362, 0x301, 0x000, 0x00f,
+ 0x01b, 0x347, 0x305, 0x301, 0x347, 0x2fb, 0x290, 0x3c6,
+ 0x0ff, 0x01a, 0x307, 0x2ae, 0x300, 0x2be, 0x114, 0x247,
+ 0x2a5, 0x307, 0x2af, 0x300, 0x2c1, 0x104, 0x247, 0x2a6,
+ 0x000, 0x36a, 0x307, 0x305, 0x300, 0x301, 0x24a, 0x290,
+ 0x044, 0x034, 0x003, 0x045, 0x0c0, 0x2e4, 0x0c3, 0x2ce,
+ 0x044, 0x1bf, 0x003, 0x0c0, 0x2ea, 0x044, 0x078, 0x009,
+ 0x347, 0x28f, 0x318, 0x347, 0x28e, 0x319, 0x0c0, 0x31b,
+ 0x0c0, 0x31c, 0x3c7, 0x038, 0x317, 0x364, 0x2d4, 0x020,
+ 0x001, 0x005, 0x347, 0x2c6, 0x317, 0x364, 0x2d4, 0x002,
+ 0x001, 0x005, 0x347, 0x290, 0x317, 0x0c0, 0x31e, 0x0cf,
+ 0x324, 0x364, 0x2d4, 0x004, 0x001, 0x008, 0x347, 0x28f,
+ 0x31e, 0x347, 0x28f, 0x324, 0x0c0, 0x31f, 0x0cf, 0x325,
+ 0x364, 0x2d4, 0x008, 0x001, 0x008, 0x347, 0x28e, 0x31f,
+ 0x347, 0x28e, 0x325, 0x364, 0x2d4, 0x010, 0x001, 0x00d,
+ 0x347, 0x291, 0x320, 0x347, 0x291, 0x326, 0x347, 0x291,
+ 0x31a, 0x000, 0x00a, 0x0cd, 0x291, 0x0cd, 0x31a, 0x0c0,
+ 0x320, 0x0cd, 0x326, 0x044, 0x1bf, 0x00a, 0x364, 0x24d,
+ 0x030, 0x009, 0x028, 0x309, 0x2c7, 0x34b, 0x2c8, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x34e, 0x2c8, 0x337, 0x32c,
+ 0x2c7, 0x282, 0x343, 0x337, 0x336, 0x247, 0x332, 0x347,
+ 0x336, 0x333, 0x307, 0x332, 0x347, 0x333, 0x336, 0x300,
+ 0x2c7, 0x341, 0x2c8, 0x336, 0x247, 0x334, 0x347, 0x336,
+ 0x335, 0x0c0, 0x398, 0x364, 0x2df, 0x002, 0x001, 0x006,
+ 0x0c1, 0x39a, 0x000, 0x004, 0x0c4, 0x39a, 0x347, 0x2cd,
+ 0x397, 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x39a, 0x006,
+ 0x076, 0x044, 0x16d, 0x00a, 0x0cf, 0x28f, 0x362, 0x28f,
+ 0x000, 0x007, 0x065, 0x362, 0x28f, 0x00f, 0x00f, 0x009,
+ 0x08f, 0x302, 0x28f, 0x247, 0x28e, 0x000, 0x004, 0x0c0,
+ 0x28e, 0x362, 0x28f, 0x006, 0x00f, 0x004, 0x0cf, 0x28e,
+ 0x0c1, 0x394, 0x362, 0x28e, 0x00a, 0x006, 0x004, 0x0d1,
+ 0x394, 0x0c0, 0x2a9, 0x307, 0x2a9, 0x302, 0x394, 0x006,
+ 0x03b, 0x347, 0x2a9, 0x313, 0x307, 0x2aa, 0x300, 0x2a9,
+ 0x247, 0x2a1, 0x307, 0x2ab, 0x300, 0x2a9, 0x247, 0x2a2,
+ 0x307, 0x2ac, 0x302, 0x2a9, 0x247, 0x2a3, 0x307, 0x2ad,
+ 0x302, 0x2a9, 0x247, 0x2a4, 0x044, 0x12e, 0x00a, 0x347,
+ 0x31d, 0x290, 0x307, 0x290, 0x302, 0x323, 0x007, 0x00a,
+ 0x044, 0x015, 0x00a, 0x340, 0x329, 0x290, 0x000, 0x3f4,
+ 0x307, 0x314, 0x300, 0x315, 0x247, 0x316, 0x141, 0x2a9,
+ 0x000, 0x3c3, 0x151, 0x28f, 0x000, 0x39a, 0x340, 0x317,
+ 0x398, 0x141, 0x399, 0x000, 0x388, 0x347, 0x398, 0x317,
+ 0x362, 0x39a, 0x004, 0x009, 0x007, 0x307, 0x398, 0x191,
+ 0x247, 0x317, 0x044, 0x177, 0x00a, 0x347, 0x317, 0x2e9,
+ 0x347, 0x31b, 0x2a9, 0x347, 0x2a9, 0x313, 0x347, 0x316,
+ 0x2ea, 0x0c0, 0x395, 0x347, 0x31e, 0x28f, 0x307, 0x28f,
+ 0x302, 0x324, 0x007, 0x093, 0x0c3, 0x2ce, 0x347, 0x325,
+ 0x28e, 0x347, 0x326, 0x291, 0x044, 0x3ad, 0x002, 0x0c3,
+ 0x303, 0x044, 0x3d4, 0x009, 0x347, 0x2a5, 0x2e2, 0x364,
+ 0x2d4, 0x008, 0x001, 0x007, 0x364, 0x2d4, 0x010, 0x009,
+ 0x05d, 0x0cf, 0x32e, 0x362, 0x32e, 0x000, 0x007, 0x02d,
+ 0x364, 0x2d4, 0x008, 0x009, 0x00c, 0x347, 0x32e, 0x28e,
+ 0x362, 0x28e, 0x00f, 0x006, 0x004, 0x0cf, 0x28e, 0x364,
+ 0x2d4, 0x010, 0x009, 0x005, 0x347, 0x32e, 0x291, 0x044,
+ 0x3ad, 0x002, 0x0c0, 0x303, 0x044, 0x3d4, 0x009, 0x307,
+ 0x2a5, 0x302, 0x2e2, 0x00e, 0x004, 0x000, 0x006, 0x151,
+ 0x32e, 0x000, 0x3d2, 0x364, 0x2d4, 0x008, 0x001, 0x006,
+ 0x141, 0x291, 0x000, 0x022, 0x307, 0x28e, 0x302, 0x325,
+ 0x006, 0x01c, 0x307, 0x2a5, 0x302, 0x2e2, 0x00e, 0x016,
+ 0x141, 0x28e, 0x307, 0x28e, 0x302, 0x325, 0x009, 0x004,
+ 0x000, 0x00c, 0x044, 0x034, 0x003, 0x0c0, 0x303, 0x044,
+ 0x3d4, 0x009, 0x000, 0x3e2, 0x362, 0x291, 0x00d, 0x006,
+ 0x004, 0x0cd, 0x291, 0x397, 0x39b, 0x300, 0x28f, 0x327,
+ 0x28e, 0x04d, 0x397, 0x3ab, 0x300, 0x28f, 0x327, 0x291,
+ 0x04d, 0x141, 0x28f, 0x000, 0x36b, 0x044, 0x16d, 0x00a,
+ 0x347, 0x2c7, 0x2c2, 0x347, 0x2c8, 0x2c3, 0x0c4, 0x397,
+ 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x2e0, 0x006, 0x00f,
+ 0x349, 0x2c7, 0x2c7, 0x34b, 0x2c8, 0x2c8, 0x349, 0x397,
+ 0x397, 0x141, 0x399, 0x000, 0x3ef, 0x044, 0x1bf, 0x00a,
+ 0x347, 0x31e, 0x28f, 0x307, 0x28f, 0x302, 0x324, 0x007,
+ 0x055, 0x0c3, 0x2ce, 0x397, 0x39b, 0x300, 0x28f, 0x04c,
+ 0x2c7, 0x28e, 0x397, 0x3ab, 0x300, 0x28f, 0x04c, 0x2c7,
+ 0x291, 0x347, 0x317, 0x290, 0x044, 0x3ad, 0x002, 0x0c0,
+ 0x32e, 0x362, 0x32e, 0x010, 0x00f, 0x023, 0x0c1, 0x30a,
+ 0x0c0, 0x330, 0x362, 0x330, 0x004, 0x00f, 0x013, 0x364,
+ 0x026, 0x010, 0x009, 0x00a, 0x362, 0x32e, 0x002, 0x006,
+ 0x005, 0x044, 0x1e6, 0x00a, 0x141, 0x330, 0x000, 0x3ec,
+ 0x3c4, 0x3ef, 0x026, 0x141, 0x32e, 0x000, 0x3dc, 0x0c3,
+ 0x2ce, 0x044, 0x1bf, 0x003, 0x044, 0x30e, 0x002, 0x044,
+ 0x015, 0x00a, 0x307, 0x314, 0x300, 0x315, 0x247, 0x316,
+ 0x141, 0x28f, 0x000, 0x3a9, 0x044, 0x3b3, 0x002, 0x362,
+ 0x316, 0x006, 0x00f, 0x004, 0x0c0, 0x396, 0x044, 0x177,
+ 0x00a, 0x347, 0x316, 0x2ec, 0x347, 0x2c2, 0x2c7, 0x347,
+ 0x2c3, 0x2c8, 0x045, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x207, 0x384, 0x008, 0x247, 0x304, 0x09c, 0x189, 0x395,
+ 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x3c4, 0x300, 0x01a,
+ 0x140, 0x303, 0x009, 0x010, 0x140, 0x304, 0x009, 0x007,
+ 0x3c5, 0x004, 0x01a, 0x000, 0x013, 0x3c5, 0x020, 0x01a,
+ 0x000, 0x00e, 0x140, 0x304, 0x009, 0x007, 0x3c5, 0x080,
+ 0x01a, 0x000, 0x005, 0x3c5, 0x001, 0x01a, 0x044, 0x1ba,
+ 0x003, 0x3d7, 0x200, 0x013, 0x0c0, 0x014, 0x397, 0x2a5,
+ 0x044, 0x1e9, 0x003, 0x045, 0x0c0, 0x303, 0x0c0, 0x330,
+ 0x0c0, 0x331, 0x0c0, 0x32e, 0x0c0, 0x32f, 0x044, 0x3ad,
+ 0x002, 0x0c0, 0x303, 0x307, 0x303, 0x302, 0x2d1, 0x006,
+ 0x0bb, 0x364, 0x303, 0x001, 0x001, 0x016, 0x307, 0x2ae,
+ 0x302, 0x313, 0x114, 0x302, 0x315, 0x247, 0x2a5, 0x307,
+ 0x2af, 0x302, 0x313, 0x104, 0x300, 0x315, 0x247, 0x2a6,
+ 0x000, 0x014, 0x307, 0x2ae, 0x300, 0x2a9, 0x114, 0x302,
+ 0x314, 0x247, 0x2a5, 0x307, 0x2af, 0x300, 0x2a9, 0x104,
+ 0x300, 0x314, 0x247, 0x2a6, 0x362, 0x2a5, 0x000, 0x00f,
+ 0x003, 0x045, 0x362, 0x2a6, 0x0ff, 0x006, 0x003, 0x045,
+ 0x044, 0x3f0, 0x002, 0x362, 0x303, 0x002, 0x00f, 0x02c,
+ 0x083, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x044, 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x334,
+ 0x347, 0x335, 0x336, 0x302, 0x046, 0x343, 0x047, 0x336,
+ 0x00a, 0x00e, 0x307, 0x046, 0x347, 0x047, 0x336, 0x302,
+ 0x332, 0x343, 0x333, 0x336, 0x002, 0x006, 0x347, 0x323,
+ 0x290, 0x045, 0x307, 0x303, 0x384, 0x001, 0x104, 0x18a,
+ 0x385, 0x101, 0x207, 0x0a0, 0x044, 0x18a, 0x002, 0x044,
+ 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x046, 0x347,
+ 0x047, 0x336, 0x302, 0x332, 0x343, 0x333, 0x336, 0x002,
+ 0x006, 0x347, 0x323, 0x290, 0x045, 0x364, 0x303, 0x001,
+ 0x001, 0x00a, 0x340, 0x00e, 0x330, 0x341, 0x00f, 0x331,
+ 0x000, 0x008, 0x340, 0x00e, 0x32e, 0x341, 0x00f, 0x32f,
+ 0x307, 0x2fd, 0x347, 0x2fe, 0x336, 0x327, 0x32e, 0x347,
+ 0x32f, 0x337, 0x320, 0x330, 0x341, 0x331, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x003, 0x045, 0x141, 0x303,
+ 0x000, 0x343, 0x347, 0x290, 0x317, 0x347, 0x28f, 0x318,
+ 0x347, 0x28e, 0x319, 0x347, 0x291, 0x31a, 0x347, 0x2a9,
+ 0x31b, 0x347, 0x313, 0x31c, 0x347, 0x293, 0x396, 0x307,
+ 0x32e, 0x347, 0x32f, 0x336, 0x300, 0x330, 0x341, 0x331,
+ 0x336, 0x247, 0x2fd, 0x347, 0x336, 0x2fe, 0x307, 0x2fd,
+ 0x347, 0x2fe, 0x336, 0x327, 0x397, 0x0c0, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x019, 0x30a, 0x397, 0x327,
+ 0x330, 0x282, 0x00a, 0x004, 0x141, 0x315, 0x30a, 0x397,
+ 0x327, 0x32e, 0x282, 0x00a, 0x004, 0x141, 0x314, 0x3ce,
+ 0x3ff, 0x2fe, 0x000, 0x2ea, 0x045, 0x364, 0x2d4, 0x002,
+ 0x001, 0x00c, 0x347, 0x317, 0x31d, 0x347, 0x317, 0x323,
+ 0x0c1, 0x329, 0x000, 0x031, 0x362, 0x316, 0x000, 0x00e,
+ 0x007, 0x364, 0x2d4, 0x020, 0x001, 0x01f, 0x307, 0x317,
+ 0x114, 0x247, 0x31d, 0x307, 0x317, 0x104, 0x247, 0x323,
+ 0x0c4, 0x329, 0x362, 0x31d, 0x000, 0x00f, 0x004, 0x0c0,
+ 0x31d, 0x362, 0x323, 0x0b8, 0x006, 0x00f, 0x3c7, 0x0b8,
+ 0x323, 0x000, 0x00a, 0x3c7, 0x038, 0x31d, 0x3c7, 0x0b8,
+ 0x323, 0x0c4, 0x329, 0x045, 0x0c0, 0x314, 0x0c0, 0x315,
+ 0x0c0, 0x316, 0x3ce, 0x3ff, 0x2fe, 0x045, 0x347, 0x319,
+ 0x28e, 0x347, 0x318, 0x28f, 0x347, 0x317, 0x290, 0x347,
+ 0x31a, 0x291, 0x347, 0x396, 0x293, 0x347, 0x31b, 0x2bd,
+ 0x347, 0x2bd, 0x2bc, 0x347, 0x2bc, 0x2bb, 0x347, 0x2bb,
+ 0x2ba, 0x347, 0x2ba, 0x2b5, 0x347, 0x2b5, 0x2b4, 0x347,
+ 0x2b4, 0x2b3, 0x347, 0x2b3, 0x2b2, 0x080, 0x302, 0x31c,
+ 0x247, 0x2c1, 0x347, 0x2c1, 0x2c0, 0x347, 0x2c0, 0x2bf,
+ 0x347, 0x2bf, 0x2be, 0x347, 0x2be, 0x2b9, 0x347, 0x2b9,
+ 0x2b8, 0x347, 0x2b8, 0x2b7, 0x347, 0x2b7, 0x2b6, 0x044,
+ 0x3ad, 0x002, 0x044, 0x022, 0x009, 0x045, 0x309, 0x2c7,
+ 0x34b, 0x2c8, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209,
+ 0x34b, 0x336, 0x336, 0x300, 0x2c7, 0x341, 0x2c8, 0x336,
+ 0x247, 0x332, 0x347, 0x336, 0x333, 0x309, 0x2c7, 0x34b,
+ 0x2c8, 0x336, 0x300, 0x332, 0x341, 0x333, 0x336, 0x247,
+ 0x334, 0x347, 0x336, 0x335, 0x045, 0x3a7, 0x022, 0x090,
+ 0x364, 0x2d4, 0x001, 0x001, 0x00b, 0x364, 0x2cf, 0x001,
+ 0x001, 0x005, 0x044, 0x1a9, 0x000, 0x045, 0x044, 0x1b7,
+ 0x000, 0x362, 0x2f6, 0x002, 0x00f, 0x02b, 0x0c0, 0x304,
+ 0x044, 0x25b, 0x003, 0x347, 0x2a5, 0x2e2, 0x001, 0x007,
+ 0x347, 0x2a6, 0x2e1, 0x009, 0x005, 0x141, 0x2ee, 0x045,
+ 0x364, 0x2f6, 0x001, 0x001, 0x037, 0x0c1, 0x304, 0x044,
+ 0x25b, 0x003, 0x362, 0x2a5, 0x0ff, 0x001, 0x007, 0x362,
+ 0x2a6, 0x0ff, 0x009, 0x028, 0x141, 0x2ee, 0x045, 0x307,
+ 0x2bb, 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2bd, 0x300,
+ 0x2af, 0x247, 0x2a6, 0x347, 0x2a5, 0x2e2, 0x347, 0x2a6,
+ 0x2e1, 0x364, 0x2f6, 0x001, 0x001, 0x00e, 0x307, 0x2be,
+ 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2c0, 0x300, 0x2af,
+ 0x247, 0x2a6, 0x044, 0x1bf, 0x003, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x00f, 0x364, 0x2cf, 0x001, 0x001, 0x007, 0x3d7,
+ 0x200, 0x013, 0x0c0, 0x014, 0x3c5, 0x100, 0x00d, 0x0c6,
+ 0x2ce, 0x34e, 0x306, 0x306, 0x0c2, 0x308, 0x362, 0x308,
+ 0x00d, 0x00e, 0x019, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x04c, 0x2c7, 0x303, 0x34e, 0x306, 0x306, 0x364, 0x306,
+ 0x001, 0x009, 0x005, 0x044, 0x2fa, 0x00a, 0x141, 0x308,
+ 0x000, 0x3e6, 0x140, 0x30a, 0x001, 0x05c, 0x0c0, 0x303,
+ 0x0c0, 0x304, 0x362, 0x304, 0x00b, 0x00e, 0x048, 0x397,
+ 0x294, 0x300, 0x304, 0x04c, 0x2c7, 0x305, 0x362, 0x305,
+ 0x000, 0x00f, 0x007, 0x308, 0x305, 0x101, 0x247, 0x305,
+ 0x307, 0x305, 0x302, 0x303, 0x00f, 0x005, 0x347, 0x305,
+ 0x303, 0x362, 0x305, 0x00f, 0x009, 0x025, 0x141, 0x293,
+ 0x0c0, 0x30a, 0x0c4, 0x2f7, 0x0c0, 0x304, 0x362, 0x304,
+ 0x00b, 0x00e, 0x01c, 0x397, 0x294, 0x300, 0x304, 0x04c,
+ 0x2c7, 0x305, 0x397, 0x294, 0x300, 0x304, 0x329, 0x305,
+ 0x320, 0x305, 0x2aa, 0x2aa, 0x04d, 0x141, 0x304, 0x000,
+ 0x3e7, 0x141, 0x304, 0x000, 0x3b7, 0x362, 0x303, 0x00a,
+ 0x00f, 0x008, 0x151, 0x293, 0x0c0, 0x30a, 0x0c4, 0x2f7,
+ 0x044, 0x06e, 0x003, 0x044, 0x096, 0x003, 0x045, 0x307,
+ 0x2a5, 0x347, 0x2e2, 0x2a5, 0x247, 0x2e2, 0x307, 0x2a6,
+ 0x347, 0x2e1, 0x2a6, 0x247, 0x2e1, 0x044, 0x014, 0x003,
+ 0x045, 0x347, 0x303, 0x304, 0x347, 0x304, 0x30e, 0x3ce,
+ 0x3ff, 0x2fe, 0x152, 0x303, 0x0c0, 0x305, 0x362, 0x305,
+ 0x003, 0x00f, 0x0b9, 0x141, 0x303, 0x362, 0x303, 0x3f1,
+ 0x00f, 0x004, 0x000, 0x0ac, 0x362, 0x303, 0x00f, 0x006,
+ 0x007, 0x347, 0x30c, 0x30d, 0x000, 0x0a6, 0x307, 0x308,
+ 0x112, 0x390, 0x294, 0x327, 0x303, 0x04d, 0x044, 0x096,
+ 0x003, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002, 0x00f,
+ 0x00d, 0x08c, 0x189, 0x395, 0x104, 0x328, 0x308, 0x121,
+ 0x3a4, 0x0ff, 0x000, 0x007, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x30e, 0x002, 0x347,
+ 0x00e, 0x32e, 0x347, 0x00f, 0x32f, 0x364, 0x2f6, 0x001,
+ 0x001, 0x024, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x010, 0x08e, 0x189, 0x395, 0x104, 0x328, 0x308,
+ 0x121, 0x3a4, 0x0ff, 0x044, 0x18a, 0x002, 0x000, 0x005,
+ 0x3c6, 0x0ff, 0x01a, 0x044, 0x30e, 0x002, 0x340, 0x00e,
+ 0x32e, 0x341, 0x00f, 0x32f, 0x140, 0x305, 0x001, 0x007,
+ 0x362, 0x303, 0x3f1, 0x009, 0x005, 0x347, 0x32f, 0x30b,
+ 0x362, 0x305, 0x001, 0x009, 0x005, 0x347, 0x32f, 0x30c,
+ 0x362, 0x305, 0x002, 0x009, 0x005, 0x347, 0x32f, 0x30d,
+ 0x362, 0x32f, 0x000, 0x00f, 0x00d, 0x348, 0x32e, 0x32e,
+ 0x348, 0x32f, 0x32f, 0x141, 0x32e, 0x3c1, 0x000, 0x32f,
+ 0x307, 0x32e, 0x347, 0x32f, 0x336, 0x302, 0x2fd, 0x343,
+ 0x2fe, 0x336, 0x002, 0x00b, 0x347, 0x303, 0x304, 0x347,
+ 0x32e, 0x2fd, 0x347, 0x32f, 0x2fe, 0x140, 0x305, 0x009,
+ 0x007, 0x141, 0x2fd, 0x3c1, 0x000, 0x2fe, 0x141, 0x305,
+ 0x000, 0x346, 0x307, 0x30b, 0x306, 0x30d, 0x003, 0x01d,
+ 0x307, 0x30e, 0x111, 0x247, 0x304, 0x362, 0x30c, 0x000,
+ 0x00f, 0x004, 0x142, 0x304, 0x362, 0x304, 0x3f1, 0x00f,
+ 0x005, 0x3c7, 0x3f1, 0x304, 0x362, 0x304, 0x00f, 0x006,
+ 0x004, 0x0cf, 0x304, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x327, 0x304, 0x04d, 0x045, 0x044, 0x290, 0x001, 0x3c4,
+ 0x3fe, 0x200, 0x020, 0x2e0, 0x002, 0x045, 0x1f6, 0x185,
+ 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff,
+ 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce,
+ 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020, 0x009,
+ 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9, 0x286,
+ 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x3f7, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051, 0x2c7,
+ 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b, 0x005,
+ 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4, 0x00f,
+ 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288, 0x286,
+ 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b, 0x3fc,
+ 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285, 0x307,
+ 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd, 0x000,
+ 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c,
+ 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002, 0x005,
+ 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b, 0x1f0,
+ 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081, 0x138,
+ 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080,
+ 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d, 0x1e0,
+ 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b, 0x347,
+ 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044, 0x154,
+ 0x00b, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364, 0x233,
+ 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff, 0x242,
+ 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289, 0x397,
+ 0x00b, 0x189, 0x395, 0x062, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9, 0x121,
+ 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282,
+ 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000, 0x3ed,
+ 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233, 0x002,
+ 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff, 0x281,
+ 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366, 0x282,
+ 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282, 0x044,
+ 0x154, 0x00b, 0x001, 0x00d, 0x141, 0x28b, 0x362, 0x28b,
+ 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5, 0x362,
+ 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397, 0x00b,
+ 0x189, 0x395, 0x070, 0x044, 0x2ea, 0x001, 0x009, 0x004,
+ 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005, 0x105,
+ 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121,
+ 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003,
+ 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387, 0x00b,
+ 0x189, 0x395, 0x06b, 0x300, 0x28a, 0x050, 0x247, 0x28a,
+ 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a, 0x044,
+ 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282, 0x281,
+ 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045, 0x0c0,
+ 0x244, 0x044, 0x172, 0x000, 0x043, 0x047, 0x0a1, 0x1aa,
+ 0x2c5, 0x233, 0x0c0, 0x244, 0x044, 0x172, 0x000, 0x364,
+ 0x233, 0x001, 0x029, 0x0f4, 0x000, 0x047, 0x000, 0x3e9,
+ 0x364, 0x025, 0x004, 0x009, 0x0d4, 0x307, 0x2cf, 0x18a,
+ 0x003, 0x006, 0x002, 0x08c, 0x000, 0x0bd, 0x364, 0x200,
+ 0x100, 0x009, 0x0c0, 0x0c0, 0x2e6, 0x0c0, 0x2e7, 0x08d,
+ 0x18b, 0x385, 0x090, 0x040, 0x083, 0x040, 0x387, 0x080,
+ 0x044, 0x3e3, 0x001, 0x044, 0x346, 0x002, 0x3c5, 0x001,
+ 0x2cf, 0x0c0, 0x2d8, 0x0c0, 0x2f1, 0x3c4, 0x3ef, 0x027,
+ 0x387, 0x085, 0x187, 0x385, 0x080, 0x247, 0x041, 0x3c5,
+ 0x020, 0x00c, 0x0c0, 0x2f6, 0x044, 0x3b3, 0x002, 0x140,
+ 0x2d3, 0x009, 0x026, 0x364, 0x292, 0x008, 0x009, 0x021,
+ 0x044, 0x16d, 0x009, 0x0d0, 0x312, 0x0c0, 0x30a, 0x0c0,
+ 0x2ee, 0x0d0, 0x2f7, 0x044, 0x253, 0x00b, 0x342, 0x293,
+ 0x312, 0x001, 0x00e, 0x0c1, 0x30a, 0x347, 0x293, 0x312,
+ 0x0c8, 0x2f7, 0x044, 0x253, 0x00b, 0x000, 0x3f1, 0x3c4,
+ 0x1ff, 0x2cf, 0x0c0, 0x30a, 0x307, 0x290, 0x183, 0x305,
+ 0x28f, 0x183, 0x305, 0x28e, 0x247, 0x2f2, 0x0cf, 0x2ce,
+ 0x044, 0x113, 0x003, 0x044, 0x33f, 0x002, 0x055, 0x364,
+ 0x2cf, 0x020, 0x009, 0x005, 0x3c4, 0x3bf, 0x200, 0x3c4,
+ 0x3ee, 0x2cf, 0x364, 0x2df, 0x001, 0x009, 0x04a, 0x364,
+ 0x33c, 0x002, 0x009, 0x045, 0x141, 0x2cf, 0x044, 0x346,
+ 0x002, 0x3c5, 0x002, 0x2cf, 0x347, 0x2d5, 0x2f7, 0x0c4,
+ 0x303, 0x044, 0x08a, 0x009, 0x044, 0x333, 0x008, 0x151,
+ 0x2f7, 0x00b, 0x3f6, 0x347, 0x2d5, 0x2f7, 0x044, 0x253,
+ 0x00b, 0x055, 0x3c4, 0x3fc, 0x2cf, 0x364, 0x2cf, 0x040,
+ 0x009, 0x019, 0x3c4, 0x39d, 0x2cf, 0x0ce, 0x2ce, 0x364,
+ 0x2cf, 0x001, 0x009, 0x007, 0x0cf, 0x2ce, 0x3c4, 0x3fe,
+ 0x2cf, 0x364, 0x2cf, 0x010, 0x009, 0x005, 0x3c4, 0x3bf,
+ 0x200, 0x3c4, 0x1ff, 0x2cf, 0x044, 0x33f, 0x002, 0x020,
+ 0x0d3, 0x000, 0x347, 0x2cb, 0x306, 0x044, 0x1e6, 0x00a,
+ 0x044, 0x333, 0x008, 0x151, 0x2f7, 0x009, 0x3f5, 0x045,
+ 0x397, 0x201, 0x0a0, 0x044, 0x18a, 0x002, 0x020, 0x1bf,
+ 0x003, 0x397, 0x2ba, 0x04c, 0x118, 0x04d, 0x109, 0x272,
+ 0x2c1, 0x00b, 0x3fa, 0x045, 0x397, 0x2b2, 0x04c, 0x2c7,
+ 0x303, 0x101, 0x04c, 0x322, 0x303, 0x00b, 0x009, 0x04c,
+ 0x111, 0x04d, 0x101, 0x327, 0x303, 0x04d, 0x101, 0x272,
+ 0x2c1, 0x00b, 0x3ed, 0x045, 0x2f4, 0x200, 0x001, 0x008,
+ 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287, 0x385,
+ 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010, 0x029,
+ 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384, 0x0ff,
+ 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d, 0x040,
+ 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040, 0x02d,
+ 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d, 0x042,
+ 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005, 0x3c5,
+ 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264, 0x080,
+ 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5, 0x080,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f, 0x02d,
+ 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff, 0x1b8,
+ 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4,
+ 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a, 0x020,
+ 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044,
+ 0x340, 0x001, 0x247, 0x249, 0x054, 0x347, 0x249, 0x248,
+ 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f,
+ 0x001, 0x001, 0x006, 0x245, 0x057, 0x000, 0x005, 0x208,
+ 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f,
+ 0x002, 0x001, 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001,
+ 0x000, 0x005, 0x208, 0x244, 0x057, 0x020, 0x181, 0x006,
+ 0x04e, 0x0ce, 0x045, 0x03c, 0x0f5, 0x04e, 0x0fd, 0x07e,
+ 0x00a, 0x0f9
+
+}; /* end fm10000_serdes_swap_code_prd2 */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.h b/drivers/net/fm10k/switch/fm10k_spico_code.h
new file mode 100644
index 0000000..c172638
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SPICO_CODE_H_
+#define _FM10K_SPICO_CODE_H_
+#include <stdint.h>
+
+extern const uint32_t fm10000_sbus_master_code_version_build_id_prd;
+extern const uint32_t fm10000_sbus_master_code_size_prd;
+extern const uint16_t fm10000_sbus_master_code_prd[];
+
+extern const uint32_t fm10000_serdes_spico_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_spico_code_size_prd2;
+extern const uint16_t fm10000_serdes_spico_code_prd2[];
+
+extern const uint32_t fm10000_serdes_swap_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_swap_code_size_prd2;
+extern const uint16_t fm10000_serdes_swap_code_prd2[];
+
+#endif /* _FM10K_SPICO_CODE_H_ */
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 3/5] net/fm10k: add ffu and statistics and config file functions
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
It also supports flowset for a group flow definition.
The config file can configure debug log, port speed,
epl port mapping dpdk port, flowset. All these configuration
will be used by switch management.
Statistics includes epl port, ffu rule, dpdk port, and error.
All these statistics data are read from HW.
Modify switch header file to support getting logical port
and glort and device info.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 3 +
drivers/net/fm10k/switch/fm10k_config.c | 863 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 +++++
drivers/net/fm10k/switch/fm10k_ffu.c | 1253 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_stats.c | 1114 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++++++
drivers/net/fm10k/switch/fm10k_switch.h | 158 +++-
8 files changed, 3848 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index ac2d96f..86a7635 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -89,6 +89,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_config.c b/drivers/net/fm10k/switch/fm10k_config.c
new file mode 100644
index 0000000..e958663
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.c
@@ -0,0 +1,863 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <unistd.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+
+
+#define FM10K_CONFIG_WORD_MAX 10
+#define FM10K_CONFIG_WORD_LEN 20
+#define FM10K_CONFIG_STR_MAX 100
+
+/* used for store the key strings */
+struct fm10k_cfg_key_item {
+ const char *key_str[FM10K_CONFIG_WORD_MAX];
+ uint8_t type;
+};
+
+/* configuration file path */
+static const char *fm10k_config_dpdk_conf_file = "/etc/dpdk_fm10k.conf";
+/* configuration structure */
+static struct fm10k_dpdk_cfg fm10k_config_dpdk_cfg;
+
+/* configuration key strings */
+static struct fm10k_cfg_key_item fm10k_config_key_items[] = {
+ /* enable debug print */
+ { {"debug", "print", "enable"},
+ FM10K_CONFIG_DEBUG_ENABLE},
+ { {"debug", "print", "config"},
+ FM10K_CONFIG_DEBUG_CONFIG},
+ { {"debug", "print", "ffu", "init"},
+ FM10K_CONFIG_DEBUG_FFU_INIT},
+ { {"debug", "print", "ffu", "register"},
+ FM10K_CONFIG_DEBUG_FFU_REG},
+ { {"debug", "print", "ffu", "rule"},
+ FM10K_CONFIG_DEBUG_FFU_RULE},
+ { {"debug", "print", "stats", "port"},
+ FM10K_CONFIG_DEBUG_STATS_PORT},
+ { {"debug", "print", "stats", "queue"},
+ FM10K_CONFIG_DEBUG_STATS_QUEUE},
+ { {"debug", "print", "stats", "rule"},
+ FM10K_CONFIG_DEBUG_STATS_FFU},
+ { {"debug", "print", "stats", "detail"},
+ FM10K_CONFIG_DEBUG_STATS_MORE},
+ { {"debug", "print", "stats", "interval"},
+ FM10K_CONFIG_DEBUG_STATS_INTERVAL},
+ /* general configuration */
+ { {"dpdk", "bind", "pf", "number"},
+ FM10K_CONFIG_BIND_PF_NUMBER},
+ { {"extern", "port", "speed"},
+ FM10K_CONFIG_EXT_PORT_SPEED},
+ /* internal redirect configuration */
+ { {"dpdk", "port", "*", "map", "pf"},
+ FM10K_CONFIG_DPDK_PORT_MAP_PF},
+ { {"extern", "port", "*", "map", "pf"},
+ FM10K_CONFIG_EXT_PORT_MAP_PF},
+ /* external redirect configuration */
+ { {"flowset", "start"},
+ FM10K_CONFIG_FLOWSET_START},
+ { {"flowset", "stop"},
+ FM10K_CONFIG_FLOWSET_STOP},
+ { {"flowset", "enable"},
+ FM10K_CONFIG_FLOWSET_ENABLE},
+ { {"flow", "*", "condition", "source", "extern", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT},
+ { {"flow", "*", "condition", "source", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT},
+ { {"flow", "*", "condition", "vlan"},
+ FM10K_CONFIG_FLOW_COND_VLAN},
+ { {"flow", "*", "action", "forward", "extern", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT},
+ { {"flow", "*", "action", "forward", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT},
+ { {"flow", "*", "action", "forward", "vlan"},
+ FM10K_CONFIG_FLOW_ACT_FW_VALN},
+};
+
+static char default_flowset[10] = "default";
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_2pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PF.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF. ",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_4pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 4
+ },
+
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PFs.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF.",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+
+static int
+fm10k_config_conf_line_parser(char *buff, struct fm10k_cfg_config_item *item)
+{
+ char *p;
+ char *endptr;
+ const char *cfg_delims = { " " };
+ const char *key_delims = { "." };
+ struct fm10k_cfg_key_item *key;
+ char cfgs[3][FM10K_CONFIG_STR_MAX];
+ char cmp_keys[FM10K_CONFIG_WORD_MAX][FM10K_CONFIG_WORD_LEN];
+ long key_param = 0;
+ uint16_t i, j;
+ uint8_t val_type = FM10K_CONFIG_VALUE_NULL;
+
+ for (i = 0; i < 3; i++) {
+ if (i == 0)
+ p = strtok(buff, cfg_delims);
+ else
+ p = strtok(NULL, cfg_delims);
+ if (p == NULL)
+ return -1;
+ strncpy(cfgs[i], p, FM10K_CONFIG_STR_MAX);
+ }
+
+ p = strtok(NULL, cfg_delims);
+ if (p)
+ return -1;
+
+ memset(cmp_keys, 0, sizeof(cmp_keys));
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (i == 0)
+ p = strtok(cfgs[0], key_delims);
+ else
+ p = strtok(NULL, key_delims);
+ if (p == NULL)
+ break;
+ strncpy(cmp_keys[i], p, FM10K_CONFIG_WORD_LEN);
+ }
+
+ if (strcmp(cfgs[1], "int") == 0)
+ val_type = FM10K_CONFIG_VALUE_INT;
+ else if (strcmp(cfgs[1], "string") == 0)
+ val_type = FM10K_CONFIG_VALUE_STR;
+ else
+ return -1;
+
+ for (i = 0;
+ i < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ i++) {
+ key = &fm10k_config_key_items[i];
+ for (j = 0; j < FM10K_CONFIG_WORD_MAX; j++) {
+ if (key->key_str[j] &&
+ strlen(cmp_keys[j]) > 0) {
+ if (strcmp(key->key_str[j], "*") == 0) {
+ key_param =
+ strtol(cmp_keys[j], &endptr, 10);
+ if ((key_param == 0 &&
+ endptr == cmp_keys[j]) ||
+ (endptr != cmp_keys[j] &&
+ strlen(endptr) != 0))
+ break;
+ } else if (strcmp(key->key_str[j],
+ cmp_keys[j]) != 0) {
+ break;
+ }
+ } else if (key->key_str[j] == 0 &&
+ strlen(cmp_keys[j]) == 0) {
+ if (val_type == FM10K_CONFIG_VALUE_STR) {
+ item->val.str =
+ malloc(strlen(cfgs[2]) + 1);
+ if (item->val.str == NULL)
+ return -1;
+ strcpy(item->val.str, cfgs[2]);
+ } else {
+ item->val.int64 =
+ strtol(cfgs[2], NULL, 10);
+ }
+ item->type = key->type;
+ item->key_param = key_param;
+ item->val_type = val_type;
+ return 0;
+ }
+ }
+ }
+ return -1;
+}
+
+static bool
+fm10k_config_blank_line_check(char *line)
+{
+ uint16_t i;
+
+ for (i = 0; i < strlen(line); i++) {
+ if (line[i] == ' ' ||
+ line[i] == '\n' ||
+ line[i] == '\t' || line[i] == 0x0d) /* 0d: CR */
+ continue;
+ else
+ return false;
+ }
+ return true;
+}
+
+static int
+fm10k_config_conf_file_load(void)
+{
+ int i = 0;
+ FILE *fp;
+ char buff[255];
+ struct fm10k_cfg_config_item *item;
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "r");
+ if (fp == NULL)
+ return -1;
+
+ fm10k_config_dpdk_cfg.config_list = malloc
+ (sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+ if (fm10k_config_dpdk_cfg.config_list == NULL)
+ return -1;
+ memset(fm10k_config_dpdk_cfg.config_list, 0,
+ sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+
+ while (fgets(buff, sizeof(buff), fp)) {
+ if (buff[0] == '#' || buff[0] == 0)
+ continue;
+
+ if (fm10k_config_blank_line_check(buff))
+ continue;
+
+ item = &fm10k_config_dpdk_cfg.config_list[i++];
+ if (fm10k_config_conf_line_parser(buff, item) < 0) {
+ FM10K_SW_ERR("Unknown configuration: %s", buff);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+struct fm10k_hw*
+fm10k_config_hw_get(int port_no)
+{
+ int hw_port;
+
+ hw_port = fm10k_config_dpdk_cfg.dpdk_port_map[port_no].map_no[0];
+ return fm10k_config_dpdk_cfg.pf_hw[hw_port];
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_current_get(void)
+{
+ return fm10k_config_dpdk_cfg.current;
+}
+
+void
+fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new)
+{
+ fm10k_config_dpdk_cfg.current = new;
+}
+
+bool
+fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow)
+{
+ return (list == flow);
+}
+
+static void
+fm10k_config_flow_list_init(struct fm10k_cfg_flow *list)
+{
+ list->next = list;
+ list->prev = list;
+}
+
+static void
+fm10k_config_flow_list_add(struct fm10k_cfg_flow *new,
+ struct fm10k_cfg_flow *list)
+{
+ struct fm10k_cfg_flow *next = list->next;
+ next->prev = new;
+ new->next = next;
+ new->prev = list;
+ list->next = new;
+}
+
+void
+fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow)
+{
+ fm10k_config_flow_list_add(flow, flowset->flow_head.prev);
+}
+
+void
+fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow)
+{
+ flow->prev->next = flow->next;
+ flow->next->prev = flow->prev;
+ flow->prev = NULL;
+ flow->next = NULL;
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_get(const char *name)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ if (strcmp(flowset->name, name) == 0)
+ break;
+ flowset = flowset->next;
+ }
+
+ if (flowset == NULL) {
+ flowset = malloc(sizeof(struct fm10k_cfg_flowset));
+ if (flowset == NULL)
+ return NULL;
+ strcpy(flowset->name, name);
+ fm10k_config_flow_list_init(&flowset->flow_head);
+ flowset->next = fm10k_config_dpdk_cfg.flowset_head.next;
+ fm10k_config_dpdk_cfg.flowset_head.next = flowset;
+ }
+ return flowset;
+}
+
+
+static int
+fm10k_cfg_flow_item_set(struct fm10k_cfg_flowset *flowset,
+ uint8_t flow_no, uint8_t type, int64_t val)
+{
+ struct fm10k_cfg_flow *tmp;
+ struct fm10k_cfg_flow *flow = NULL;
+
+ if (flowset == NULL)
+ return -1;
+
+ tmp = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, tmp)) {
+ if (tmp->flow_no == flow_no) {
+ flow = tmp;
+ break;
+ }
+ tmp = tmp->next;
+ }
+ if (flow == NULL) {
+ flow = malloc(sizeof(struct fm10k_cfg_flow));
+ memset(flow, 0, sizeof(struct fm10k_cfg_flow));
+ flow->flow_no = flow_no;
+ fm10k_config_flow_list_add_tail(flowset, flow);
+ }
+ if (flow == NULL)
+ return -1;
+
+ switch (type) {
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ flow->src_port.vlan_id = val;
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flow->fw_port[0].vlan_id == 0)
+ flow->fw_port[0].vlan_id = val;
+ else
+ flow->fw_port[1].vlan_id = val;
+ break;
+ default:
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_transfer(void)
+{
+ int i;
+ int offset;
+ uint32_t tmp, data;
+ struct fm10k_cfg_config_item *item;
+ struct fm10k_cfg_flowset *flowset = NULL;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+
+ switch (item->type) {
+ case FM10K_CONFIG_BIND_PF_NUMBER:
+ fm10k_config_dpdk_cfg.pf_num = item->val.int64;
+ break;
+ case FM10K_CONFIG_EXT_PORT_SPEED:
+ fm10k_config_dpdk_cfg.ext_port_speed = item->val.int64;
+ break;
+ case FM10K_CONFIG_DPDK_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.dpdk_port_map[item->key_param];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+ case FM10K_CONFIG_EXT_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.ext_port_map
+ [item->key_param - 1];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+
+ case FM10K_CONFIG_DEBUG_ENABLE:
+ case FM10K_CONFIG_DEBUG_CONFIG:
+ case FM10K_CONFIG_DEBUG_FFU_INIT:
+ case FM10K_CONFIG_DEBUG_FFU_REG:
+ case FM10K_CONFIG_DEBUG_FFU_RULE:
+ case FM10K_CONFIG_DEBUG_STATS_PORT:
+ case FM10K_CONFIG_DEBUG_STATS_QUEUE:
+ case FM10K_CONFIG_DEBUG_STATS_FFU:
+ case FM10K_CONFIG_DEBUG_STATS_MORE:
+ offset = item->type - FM10K_CONFIG_DEBUG_START;
+ tmp = fm10k_config_dpdk_cfg.debug_cfg;
+ data = 1;
+ if (item->val.int64 != 1)
+ fm10k_config_dpdk_cfg.debug_cfg =
+ tmp & ~(data << offset);
+ else
+ fm10k_config_dpdk_cfg.debug_cfg |=
+ (data << offset);
+ break;
+ case FM10K_CONFIG_DEBUG_STATS_INTERVAL:
+ fm10k_config_dpdk_cfg.stats_interval = item->val.int64;
+ break;
+
+ case FM10K_CONFIG_FLOWSET_START:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ flowset = fm10k_config_flowset_get(item->val.str);
+ if (flowset == NULL)
+ return -1;
+ break;
+ case FM10K_CONFIG_FLOWSET_STOP:
+ flowset = NULL;
+ break;
+ case FM10K_CONFIG_FLOWSET_ENABLE:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ fm10k_config_dpdk_cfg.current =
+ fm10k_config_flowset_get(item->val.str);
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flowset == NULL ||
+ fm10k_cfg_flow_item_set(flowset,
+ item->key_param, item->type,
+ item->val.int64) != 0)
+ return -1;
+ break;
+ default:
+ return -1;
+ }
+ }
+ return 0;
+}
+
+static bool
+fm10k_config_conf_file_exist(void)
+{
+ if (access(fm10k_config_dpdk_conf_file, 0) == 0)
+ return true;
+ return false;
+}
+
+static int
+fm10k_config_conf_file_item_create(FILE *fp, char *buff,
+ struct fm10k_cfg_key_item *key,
+ struct fm10k_cfg_config_item *item)
+{
+ int i;
+
+ if (item->type != key->type)
+ return 0;
+
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (key->key_str[i] == 0)
+ break;
+ if (i == 0) {
+ sprintf(buff + strlen(buff),
+ "%s", key->key_str[i]);
+ } else {
+ if (strcmp(key->key_str[i], "*") == 0)
+ sprintf(buff + strlen(buff),
+ ".%u", item->key_param);
+ else
+ sprintf(buff + strlen(buff),
+ ".%s", key->key_str[i]);
+ }
+ }
+ if (item->val_type == FM10K_CONFIG_VALUE_INT)
+ sprintf(buff + strlen(buff),
+ " int %lld\n", (long long)item->val.int64);
+ else if (item->val_type == FM10K_CONFIG_VALUE_STR)
+ sprintf(buff + strlen(buff), " string %s\n", item->val.str);
+ else
+ return -1;
+ fwrite(buff, strlen(buff), 1, fp);
+ FM10K_SW_TRACE("[write] %s", buff);
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_create(void)
+{
+ uint16_t i, j;
+ struct fm10k_cfg_key_item *key;
+ struct fm10k_cfg_config_item *item;
+ FILE *fp;
+ char buff[255] = "";
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "w");
+ if (fp == NULL)
+ return -1;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+ buff[0] = 0;
+ if (strlen(item->describe) > 0)
+ sprintf(buff, "\n\n%s\n", item->describe);
+ for (j = 0;
+ j < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ j++) {
+ key = &fm10k_config_key_items[j];
+ if (fm10k_config_conf_file_item_create
+ (fp, buff, key, item) != 0) {
+ fclose(fp);
+ return -1;
+ }
+ }
+ }
+ fclose(fp);
+ return 0;
+}
+
+void
+fm10k_config_cfg_flowset_show(void)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d & %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+}
+
+static void
+fm10k_config_cfg_describe(struct fm10k_switch *sw,
+ struct fm10k_device_info *info)
+{
+ uint16_t i;
+ struct fm10k_cfg_flowset *flowset;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K STATIC CONFIG ---\n");
+ FM10K_SW_INFO(" Card Type: %s\n", info->desc);
+ FM10K_SW_INFO(" PF Max : %d\n", fm10k_config_dpdk_cfg.pf_max);
+ FM10K_SW_INFO(" PF Bind : %d\n", fm10k_config_dpdk_cfg.pf_num);
+ FM10K_SW_INFO(" DEBUG : %#x\n", fm10k_config_dpdk_cfg.debug_cfg);
+ FM10K_SW_INFO(" STATS GAP: %d sec\n",
+ fm10k_config_dpdk_cfg.stats_interval);
+ FM10K_SW_INFO(" EXT PORT speed: %d Gbps\n",
+ fm10k_config_dpdk_cfg.ext_port_speed);
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ for (i = 0; i < fm10k_config_dpdk_cfg.ext_port_num; i++) {
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[1]);
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_config_dpdk_cfg.dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[1]);
+ }
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d & %4s PORT %d"
+ " VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+ FM10K_SW_INFO("\n");
+}
+
+int
+fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw)
+{
+ struct fm10k_device_info *info = fm10k_get_device_info(hw);
+
+ fm10k_config_dpdk_cfg.stats_interval = 2;
+ fm10k_config_dpdk_cfg.pf_max = info->num_peps;
+ fm10k_config_dpdk_cfg.ext_port_num = info->num_ext_ports;
+
+ if (!fm10k_config_conf_file_exist()) {
+ if (info->num_epls == 2 && info->num_peps == 2)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_2pep;
+ else if (info->num_epls == 2 && info->num_peps == 4)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_4pep;
+ else
+ return -1;
+
+ fm10k_config_conf_file_create();
+ } else {
+ if (fm10k_config_conf_file_load() < 0)
+ return -1;
+ }
+
+ if (fm10k_config_conf_file_transfer() < 0)
+ return -1;
+ sw->dpdk_cfg = &fm10k_config_dpdk_cfg;
+
+ fm10k_config_cfg_describe(sw, info);
+
+ return 0;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_config.h b/drivers/net/fm10k/switch/fm10k_config.h
new file mode 100644
index 0000000..f1c6ee4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_CONFIG_H_
+#define _FM10K_CONFIG_H_
+
+
+#include <stdint.h>
+#include "fm10k_switch.h"
+
+/* General configuration */
+#define FM10K_CONFIG_TYPE_NULL 0
+#define FM10K_CONFIG_BIND_PF_NUMBER 1
+#define FM10K_CONFIG_EXT_PORT_SPEED 2
+
+/* Internal redirect configuration */
+#define FM10K_CONFIG_DPDK_PORT_MAP_PF 10
+#define FM10K_CONFIG_EXT_PORT_MAP_PF 11
+
+/* Debug configuration */
+#define FM10K_CONFIG_DEBUG_START 50
+#define FM10K_CONFIG_DEBUG_ENABLE (FM10K_CONFIG_DEBUG_START + 0)
+#define FM10K_CONFIG_DEBUG_CONFIG (FM10K_CONFIG_DEBUG_START + 1)
+#define FM10K_CONFIG_DEBUG_FFU_INIT (FM10K_CONFIG_DEBUG_START + 2)
+#define FM10K_CONFIG_DEBUG_FFU_REG (FM10K_CONFIG_DEBUG_START + 3)
+#define FM10K_CONFIG_DEBUG_FFU_RULE (FM10K_CONFIG_DEBUG_START + 4)
+#define FM10K_CONFIG_DEBUG_STATS_PORT (FM10K_CONFIG_DEBUG_START + 5)
+#define FM10K_CONFIG_DEBUG_STATS_QUEUE (FM10K_CONFIG_DEBUG_START + 6)
+#define FM10K_CONFIG_DEBUG_STATS_FFU (FM10K_CONFIG_DEBUG_START + 7)
+#define FM10K_CONFIG_DEBUG_STATS_MORE (FM10K_CONFIG_DEBUG_START + 8)
+#define FM10K_CONFIG_DEBUG_STATS_INTERVAL (FM10K_CONFIG_DEBUG_START + 9)
+
+/* external redirect configuration */
+#define FM10K_CONFIG_FLOWSET_START 80
+#define FM10K_CONFIG_FLOWSET_STOP 81
+#define FM10K_CONFIG_FLOWSET_ENABLE 82
+
+#define FM10K_CONFIG_FLOW_COND_START 100
+#define FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 0)
+#define FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 1)
+#define FM10K_CONFIG_FLOW_COND_VLAN \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+#define FM10K_CONFIG_FLOW_COND_END \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+
+#define FM10K_CONFIG_FLOW_ACT_START 200
+#define FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 0)
+#define FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 1)
+#define FM10K_CONFIG_FLOW_ACT_FW_VALN \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+#define FM10K_CONFIG_FLOW_ACT_END \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+
+#define FM10K_CONFIG_FLOW_NONE_PORT 0
+#define FM10K_CONFIG_FLOW_EXT_PORT 1
+#define FM10K_CONFIG_FLOW_DPDK_PORT 2
+
+#define FM10K_CONFIG_VALUE_NULL 0
+#define FM10K_CONFIG_VALUE_INT 1
+#define FM10K_CONFIG_VALUE_STR 2
+
+/* SWITCH config */
+#define FM10K_CONFIG_PORT_MAP_NULL 0
+#define FM10K_CONFIG_PORT_MAP_PF 1
+#define FM10K_CONFIG_PORT_MAP_PFS 2
+#define FM10K_CONFIG_PORT_MAP_PFSS 3
+
+/* DPDK port */
+#define FM10K_CONFIG_DPDK_NULL 0
+#define FM10K_CONFIG_DPDK_PF 1
+#define FM10K_CONFIG_DPDK_VF 2
+#define FM10K_CONFIG_DPDK_MAX 3
+
+struct fm10k_cfg_port_pf_map {
+ uint8_t type;
+ uint8_t map_no[2];
+};
+
+/* Configuration read from file */
+struct fm10k_cfg_config_item {
+ uint8_t type;
+ uint8_t val_type;
+ uint16_t key_param;
+ const char *describe;
+ union {
+ int64_t int64;
+ char *str;
+ } val;
+};
+
+struct fm10k_hw;
+
+struct fm10k_dpdk_port {
+ uint8_t type;
+ struct fm10k_hw *hw;
+ void *rte_dev;
+ void *flow_list;
+ uint16_t pf_no;
+ uint8_t tx_queue_num;
+ uint8_t rx_queue_num;
+};
+
+
+/* Flow configuration */
+struct fm10k_cfg_port {
+ uint8_t port_type;
+ uint8_t port_no;
+ uint16_t vlan_id;
+};
+
+struct fm10k_cfg_flow {
+ /* set by configuration */
+ struct fm10k_cfg_flow *prev;
+ struct fm10k_cfg_flow *next;
+ uint8_t flow_no; /* only configured flow has this NO. */
+ struct fm10k_cfg_port src_port;
+ struct fm10k_cfg_port fw_port[2];
+ /* set by ffu rule add */
+ uint16_t rule_id;
+};
+
+#define FM10K_CONFIG_FLOWSET_NAME_MAX 256
+struct fm10k_cfg_flowset {
+ char name[FM10K_CONFIG_FLOWSET_NAME_MAX];
+ struct fm10k_cfg_flow flow_head;
+ struct fm10k_cfg_flowset *next;
+};
+
+/* Configuration */
+struct fm10k_dpdk_cfg {
+ uint8_t pf_num; /* configure by conf */
+ uint8_t pf_bind; /* initialize by dpdk */
+ uint8_t pf_max; /* set by card type */
+ uint8_t ext_port_num; /* configure by conf */
+ uint8_t ext_port_speed; /* configure by conf */
+ uint32_t debug_cfg; /* configure by conf */
+ uint32_t stats_interval;/* configure by conf */
+
+ struct fm10k_hw *master_hw; /* initialize by dpdk */
+ struct fm10k_hw *pf_hw[FM10K_SW_PEP_PORTS_MAX]; /* initialize by dpdk */
+
+ /* initialize by dpdk */
+ struct fm10k_dpdk_port ports[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf or default*/
+ struct fm10k_cfg_port_pf_map dpdk_port_map[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf */
+ struct fm10k_cfg_port_pf_map ext_port_map[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_cfg_config_item *config_list; /* configure by conf */
+ struct fm10k_cfg_flowset flowset_head; /* transfer from conf file */
+ struct fm10k_cfg_flowset *current;
+};
+
+static inline bool
+fm10k_config_check_debug(struct fm10k_dpdk_cfg *cfg, uint16_t dbg)
+{
+ return cfg->debug_cfg & 1 << (dbg - FM10K_CONFIG_DEBUG_START) &&
+ cfg->debug_cfg & 1;
+}
+
+struct fm10k_cfg_flowset *fm10k_config_flowset_get(const char *name);
+struct fm10k_cfg_flowset *fm10k_config_flowset_current_get(void);
+void fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new);
+void fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow);
+void fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow);
+bool fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow);
+
+void fm10k_config_cfg_flowset_show(void);
+struct fm10k_hw *fm10k_config_hw_get(int port_no);
+int fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw);
+
+#endif /* _FM10K_CONFIG_H */
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.c b/drivers/net/fm10k/switch/fm10k_ffu.c
new file mode 100644
index 0000000..a9b23aa
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.c
@@ -0,0 +1,1253 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+
+/*
+ * one SLICE for 40bits SEL
+ * SLICE 28 FOR SEL SGLORT(16bits) and VLAN(16bits)
+ * FOR ACT ROUTE
+ * SLICE 29 FOR SEL ETHER_TYPE(16bits)
+ * FOR ACT MIRROR | SET_VLAN
+ * SLICE 30 FOR SEL MPLS_LABEL0(32bits)
+ * SLICE 31 FOR SEL MPLS_LABEL1(32bits)
+ */
+#define FM10K_FFU_SLICE_START 28
+#define FM10K_FFU_SLICE_SGLORT_VID 28
+#define FM10K_FFU_SLICE_ETYPE_VID2 29
+#define FM10K_FFU_SLICE_MPLS_LABEL0 30
+#define FM10K_FFU_SLICE_MPLS_LABEL1 31
+
+#define FM10K_FFU_SLICE_SGLORT_VLAN_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_FFU_SLICE_CFG_START_ACTION | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_ETHER_TYPE_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL0_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL1_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+static uint64_t fm10k_ffu_slice_cfgs[4] = {
+ FM10K_FFU_SLICE_SGLORT_VLAN_CFG,
+ FM10K_FFU_SLICE_ETHER_TYPE_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL0_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL1_CFG
+};
+
+#define FM10K_FFU_INIT_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_INIT)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_RULE_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_RULE)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_CNT_INDEX(x) (FM10K_SW_FFU_CNT_START + (x))
+#define FM10K_FFU_FLOW_START 10
+#define FM10K_FFU_MIRROR_PROFILE 1
+#define FM10K_FFU_SGLORT_TYPE_NULL 0
+#define FM10K_FFU_SGLORT_TYPE_EPL 1
+#define FM10K_FFU_SGLORT_TYPE_PF 2
+#define FM10K_FFU_SGLORT_TYPE_PFS 3
+#define FM10K_FFU_SGLORT_TYPE_DPDK 4
+
+struct fm10k_ffu_rule_data {
+ uint16_t sglort_type;
+ uint16_t cond_sglort;
+ uint16_t cond_vlan;
+ uint16_t act_dglort;
+ uint16_t act_vlan;
+ uint16_t bypass_dglort;
+ uint16_t bypass_vlan;
+};
+
+
+static uint8_t fm10k_ffu_bitmap[FM10K_FFU_RULE_MAX / 8];
+static uint8_t
+fm10k_ffu_rule_get_bit(int id)
+{
+ int num = id / 8;
+ int offset = id % 8;
+
+ return (fm10k_ffu_bitmap[num] >> offset) & 0x1;
+}
+
+static void
+fm10k_ffu_rule_set_bit(int id, uint8_t bit)
+{
+ int num = id / 8;
+ int offset = id % 8;
+ uint8_t tmp = fm10k_ffu_bitmap[num];
+ uint8_t data = 1;
+
+ if (bit == 0)
+ fm10k_ffu_bitmap[num] = tmp & ~(data << offset);
+ else
+ fm10k_ffu_bitmap[num] |= (data << offset);
+}
+
+static int
+fm10k_ffu_rule_alloc(void)
+{
+ int i;
+
+ for (i = FM10K_FFU_FLOW_START; i < FM10K_FFU_RULE_MAX; i++) {
+ if (fm10k_ffu_rule_get_bit(i) != 0)
+ continue;
+ fm10k_ffu_rule_set_bit(i, 1);
+ return i;
+ }
+ return -1;
+}
+
+static void
+fm10k_ffu_rule_free(int id)
+{
+ fm10k_ffu_rule_set_bit(id, 0);
+}
+
+static void
+fm10k_ffu_always_mismatch(struct fm10k_switch *sw,
+ unsigned int slice, unsigned int idx)
+{
+ uint64_t temp64;
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~0ULL) |
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY_TOP, ~0ULL);
+ fm10k_write_switch_reg128(sw, FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64, temp64);
+}
+
+static void
+fm10k_ffu_route_dglort(struct fm10k_switch *sw, unsigned int slice,
+ unsigned int idx, uint16_t sglort, uint16_t dglort)
+{
+ uint64_t temp64, temp64_2;
+
+ /*
+ * Set the key to exact match on the 16 SGLORT bits and
+ * always match everywhere else.
+ */
+ temp64 = FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, sglort);
+ temp64_2 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~sglort & 0xffff);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64_2, temp64);
+
+ /*
+ * Set the corresponding SRAM entry to ROUTE_GLORT to the
+ * corresponding DGLORT.
+ */
+ temp64 = 0x40;
+ temp64 = temp64 << 32 |
+ FM10K_SW_MAKE_REG_FIELD(FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ /*
+ * 11.5.4.2 FFU_SLICE_SRAM[0..31][0..1023]
+ */
+ temp64_2 =
+ FM10K_SW_FFU_CNT_BANK << (35 - 23) |
+ (FM10K_SW_FFU_CNT_START + idx);
+ temp64 |= temp64_2 << 23;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(slice, idx), temp64);
+}
+
+
+static void
+fm10k_ffu_set_dest_glort_mask(struct fm10k_switch *sw,
+ unsigned int idx, uint16_t dglort, uint64_t dport_mask)
+{
+ uint64_t temp64;
+ unsigned int multiple_dports;
+ unsigned int num_dports;
+ unsigned int amplification_factor;
+ unsigned int hashed_entries;
+ unsigned int i, j;
+
+ multiple_dports = (dport_mask & (dport_mask - 1));
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to port ", __func__, dglort);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ if (multiple_dports) {
+ /*
+ * Create a pair of entries and use the hash value to select
+ * among them.
+ */
+ num_dports = 0;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i)) {
+ num_dports++;
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d", i);
+ }
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, "\n");
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ amplification_factor = 16 / num_dports;
+ hashed_entries = num_dports * amplification_factor;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_HASHED);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_COUNT,
+ hashed_entries);
+ fm10k_write_switch_reg64(sw, FM10K_SW_GLORT_RAM(idx), temp64);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ for (j = 0; j < amplification_factor; j++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ (1ULL << i)));
+ sw->glort_dest_table_idx++;
+ }
+ } else {
+ /*
+ * Set the corresponding entry in the DGLORT map RAM to use
+ * strict indexing straight into the DEST_TABLE, then write
+ * the corresponding destination port in the DEST_TABLE.
+ */
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ dport_mask));
+ sw->glort_dest_table_idx++;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d\n", i);
+ }
+}
+
+static struct fm10k_multi_glort {
+ uint8_t lport1;
+ uint8_t lport2;
+ uint16_t vlan1;
+ uint16_t vlan2;
+} fm10k_multi_glorts[FM10K_SW_FFU_RULE_MAX];
+
+static uint32_t
+fm10k_ffu_multi_glort_get(uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (lport1 == fm10k_multi_glorts[i].lport1 &&
+ lport2 == fm10k_multi_glorts[i].lport2 &&
+ vlan1 == fm10k_multi_glorts[i].vlan1 &&
+ vlan2 == fm10k_multi_glorts[i].vlan2) {
+ if (p_new != NULL)
+ *p_new = false;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (fm10k_multi_glorts[i].lport1 == 0 &&
+ fm10k_multi_glorts[i].lport2 == 0 &&
+ fm10k_multi_glorts[i].vlan1 == 0 &&
+ fm10k_multi_glorts[i].vlan2 == 0) {
+ fm10k_multi_glorts[i].lport1 = lport1;
+ fm10k_multi_glorts[i].lport2 = lport2;
+ fm10k_multi_glorts[i].vlan1 = vlan1;
+ fm10k_multi_glorts[i].vlan2 = vlan2;
+ if (p_new != NULL)
+ *p_new = true;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ return 0;
+}
+
+static uint32_t
+fm10k_ffu_set_dest_glort_multi_cast(struct fm10k_switch *sw,
+ uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2)
+{
+ uint32_t dglort;
+ bool is_new = false;
+ uint32_t idx;
+ uint64_t temp64;
+
+ dglort =
+ fm10k_ffu_multi_glort_get(lport1, lport2,
+ vlan1, vlan2, &is_new);
+
+ if (!is_new)
+ return dglort;
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to (%d:%d) (%d:%d)\n",
+ __func__, dglort, lport1, vlan1, lport2, vlan2);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ idx = sw->glort_cam_ram_idx++;
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ /* GLORT_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * DestMask 47:0 RW 0x0
+ * IP_MulticastIndex59:48 RW 0x0
+ * Reserved 63:60 RSV 0x0
+ */
+ temp64 = sw->mcast_dest_table_idx;
+ temp64 = temp64 << 48 | 1 << lport1 | 1 << lport2;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx), temp64);
+ sw->glort_dest_table_idx++;
+
+ /* MCAST_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * PortMask 47:0 RW 0x0
+ * LenTableIdx 61:48 RW 0x0
+ * Reserved 63:62 RSV 00b
+ */
+ temp64 = sw->mcast_len_table_idx;
+ temp64 = 1 << lport1 | 1 << lport2 | temp64 << 48;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE
+ (sw->mcast_dest_table_idx++), temp64);
+
+ /* MCAST_LEN_TABLE
+ * Field Name Bit(s) Type Default
+ * L3_McastIdx 14:0 RW 0x0
+ * L3_Repcnt 26:15 RW 0x0
+ * Reserved 31:27 RSV 0x0
+ */
+ temp64 =
+ sw->mcast_vlan_table_idx | 1 << 15;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE
+ (sw->mcast_len_table_idx++), temp64);
+
+ /* MCAST_VLAN_TABLE
+ * Field Name Bit(s) Type Default
+ * VID 11:0 RW 0x0
+ * DGLORT 27:12 RW 0x0
+ * ReplaceVID 28 RW 0b
+ * ReplaceDGLORT 29 RW 0b
+ * Reserved 31:30 RSV 00b
+ */
+ temp64 = vlan1 |
+ fm10k_switch_pf_glort_get
+ (lport1) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+ temp64 = vlan2 |
+ fm10k_switch_pf_glort_get
+ (lport2) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+
+ return dglort;
+}
+
+static uint64_t
+fm10k_data64_field64_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+
+ if (start == end) {
+ tmp64 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp64 = tmp64 >> (64 - end + start);
+ }
+ return tmp64;
+}
+
+static uint32_t
+fm10k_data64_field32_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+ uint32_t tmp32;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp32 = tmp64 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+static uint32_t
+fm10k_data32_field_get(uint32_t data, int start, int end)
+{
+ uint32_t tmp32 = data;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp32 = tmp32 << (64 - end);
+ tmp32 = tmp32 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+
+static void
+fm10k_glort_register_dump(struct fm10k_switch *sw)
+{
+ uint32_t i;
+ uint64_t data64;
+ uint32_t data32;
+ uint32_t tmp32;
+
+ if (!fm10k_config_check_debug
+ (sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("----- GLORT -----\n");
+
+ for (i = 0; i < sw->glort_cam_ram_idx; i++) {
+ data32 = fm10k_read_switch_reg(sw, FM10K_SW_GLORT_CAM(i));
+ FM10K_SW_INFO("[%02u]GLORT_CAM %#x Key %#x\n", i, data32,
+ fm10k_data32_field_get(data32, 0, 15));
+ data64 = fm10k_read_switch_reg64(sw, FM10K_SW_GLORT_RAM(i));
+ FM10K_SW_INFO(" GLORT_RAM %#llx Strict %u DestIndex %u\n",
+ (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 0, 1),
+ fm10k_data64_field32_get(data64, 2, 13));
+ tmp32 = fm10k_data64_field32_get(data64, 2, 13);
+ data64 =
+ fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" GLORT_DEST_TABLE[%u] %#llx "
+ "IP_MulticastIndex %u\n",
+ tmp32, (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 48, 59));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 59);
+ if (tmp32 == 0)
+ continue;
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_DEST_TABLE[%u] "
+ "%#llx PortMask %#llx"
+ " LenTableIdx %u\n", tmp32,
+ (unsigned long long)data64,
+ (unsigned long long)
+ fm10k_data64_field64_get(data64, 0, 47),
+ fm10k_data64_field32_get(data64, 48, 61));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 61);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_LEN_TABLE[%u] %#x "
+ "L3_McastIdx %u L3_Repcnt %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 14),
+ fm10k_data32_field_get(data32, 15, 26));
+ tmp32 = fm10k_data32_field_get(data32, 0, 14);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32 + 1));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32 + 1, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ }
+}
+
+static void
+fm10k_ffu_register_dump(struct fm10k_switch *sw)
+{
+ int i, j;
+ uint32_t data[8];
+ uint32_t ffu_valid;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("--------------- FFU REGISTERS DUMP -----------------\n");
+
+ fm10k_read_switch_array(sw, FM10K_SW_FFU_MASTER_VALID, data, 2);
+ FM10K_SW_INFO("FFU_MASTER_VALID: %#x %#x\n", data[0], data[1]);
+ ffu_valid = data[0];
+
+ for (i = 0; i < 32; i++) {
+ if ((ffu_valid & (1 << i)) == 0)
+ continue;
+
+ FM10K_SW_INFO("------ SLICE%d ------\n", i);
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_VALID(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_VALID[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CASCADE_ACTION[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ for (j = 0; j < 1; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CFG(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CFG[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ for (j = 0; j < 32; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_TCAM(i, j), data, 4);
+ if ((data[0] != 0 || data[1] != 0 ||
+ data[2] != 0 || data[3] != 0))
+ FM10K_SW_INFO("FFU_SLICE_TCAM[%d][%d]: "
+ "%#x %#x %#x %#x\n",
+ i, j, data[0], data[1],
+ data[2], data[3]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_SRAM[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ }
+}
+
+
+static void
+fm10k_ffu_mirror_set_action(struct fm10k_switch *sw,
+ uint16_t ffu_slice, uint16_t table_idx)
+{
+ uint64_t data64;
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice, table_idx), 0xffffffffff, 0x1);
+ /* SET FFU RX_MIRROR */
+ data64 = 0x60; /* higher than route, not necessary */
+ data64 = data64 << 32 | 0x2 << 21 | 1 << (8 + 4) | 1 << 4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), data64);
+}
+
+static void
+fm10k_ffu_mirror_set_forward(struct fm10k_switch *sw,
+ int profile,
+ uint16_t vlan,
+ uint16_t dest_lport,
+ uint32_t dest_sglort)
+{
+ uint64_t data64;
+
+ /* RX_MIRROR_CFG */
+ fm10k_write_switch_reg(sw, 0xd50000 + 0xd, profile);
+
+ /* FH_MIRROR_PROFILE_TABLE */
+ fm10k_write_switch_reg(sw,
+ 0xd50000 + 0x1 * profile + 0x40, dest_lport);
+
+ /* MOD_MIRROR_PROFILE_TABLE don't work */
+ data64 = vlan;
+ data64 = (dest_sglort & 0xffff) | data64 << 17;
+ fm10k_write_switch_reg64(sw,
+ 0xe80000 + 0x2 * profile + 0x24000, data64);
+}
+
+
+int
+fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_ext_port,
+ uint16_t dest_ext_port,
+ uint16_t vlan)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+ uint16_t mirror_profile = FM10K_FFU_MIRROR_PROFILE;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_ffu_mirror_set_action(sw, ffu_slice, table_idx);
+
+ fm10k_ffu_mirror_set_forward(sw, mirror_profile,
+ vlan, sw->epl_map[dest_ext_port - 1].logical_port,
+ sw->epl_map[dest_ext_port - 1].glort);
+ return 0;
+}
+
+int
+fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_ext_port)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), 0);
+ return 0;
+}
+
+static void
+fm10k_ffu_logical_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t lport, uint16_t vlan_id)
+{
+ uint64_t data;
+
+ /* 11.21.3.9 INGRESS_VID_TABLE[0..4095] */
+ data = fm10k_read_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000);
+ data |= 1 << lport;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000, data);
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "%s lport:%d, vlan_id:%d, reg:%#llx\n",
+ __func__, lport, vlan_id, (unsigned long long)data);
+}
+
+static void
+fm10k_ffu_glort_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t glort, uint16_t vlan_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ if (sw->pep_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->pep_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->epl_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+}
+
+
+static uint16_t
+fm10k_ffu_dpdk_port_glort_get(struct fm10k_switch *sw, int port)
+{
+ int pf;
+ uint16_t glort = 0;
+
+ if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFS ||
+ sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ glort = fm10k_switch_pfs_glort_get
+ (sw->dpdk_cfg->dpdk_port_map[port].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[port].map_no[1]);
+ } else if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf = sw->dpdk_cfg->dpdk_port_map[port].map_no[0];
+ glort = fm10k_switch_pf_glort_get(pf);
+ }
+ return glort;
+}
+
+static void
+fm10k_ffu_rule_enable_single_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint16_t dglort,
+ uint16_t dvlan)
+{
+ uint64_t temp64;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (dvlan) {
+ /* Force updating VLAN tag if present.
+ * Add if absent. 11.5.3.4
+ */
+ temp64 = 3;
+ sram[sram_idx] =
+ temp64 << 16 | dvlan |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 2) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET);
+ fm10k_ffu_glort_port_vlan_set(sw, dglort, dvlan);
+ sram_idx++;
+ }
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+static void
+fm10k_ffu_rule_enable_multi_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint8_t lport1,
+ uint8_t lport2,
+ uint16_t vlan1,
+ uint16_t vlan2)
+{
+ uint64_t temp64;
+ uint32_t dglort;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ dglort =
+ fm10k_ffu_set_dest_glort_multi_cast(sw,
+ lport1, lport2, vlan1, vlan2);
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ fm10k_ffu_logical_port_vlan_set(sw, lport1, vlan1);
+ fm10k_ffu_logical_port_vlan_set(sw, lport2, vlan2);
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+
+void
+fm10k_ffu_flow_enable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ uint16_t sglort = 0;
+ uint16_t svlan = 0;
+ uint16_t dglort = 0;
+ uint16_t dvlan = 0;
+ uint16_t lport1 = 0, lport2 = 0;
+ uint16_t dvlan2 = 0;
+
+ switch (flow->src_port.port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ sglort = fm10k_switch_epl_glort_get(flow->src_port.port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ sglort =
+ fm10k_ffu_dpdk_port_glort_get(sw,
+ flow->src_port.port_no);
+ break;
+ }
+ switch (flow->fw_port[0].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[0].port_no - 1);
+ lport1 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[0].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[0].port_no);
+ lport1 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[0].port_no);
+ break;
+ }
+ svlan = flow->src_port.vlan_id;
+ dvlan = flow->fw_port[0].vlan_id;
+
+ flow->rule_id = fm10k_ffu_rule_alloc();
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Set rule cond sglort:%#x, vlan:%d ==> "
+ "act dglort:%#x, vlan:%d",
+ sglort, svlan, dglort, dvlan);
+
+ if (flow->fw_port[1].port_type) {
+ switch (flow->fw_port[1].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[1].port_no - 1);
+ lport2 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[1].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[1].port_no);
+ lport2 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[1].port_no);
+ break;
+ }
+ dvlan2 = flow->fw_port[1].vlan_id;
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ " (bypass glort:%#x, vlan:%d)\n",
+ dglort, dvlan2);
+
+ fm10k_ffu_rule_enable_multi_cast(sw,
+ flow->rule_id, sglort, svlan,
+ lport1, lport2, dvlan, dvlan2);
+ } else {
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg, "\n");
+ fm10k_ffu_rule_enable_single_cast(sw,
+ flow->rule_id, sglort, svlan, dglort, dvlan);
+ }
+}
+
+
+void
+fm10k_ffu_flow_disable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ int i;
+
+ if (flow->rule_id == 0)
+ return;
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Remove flow %d rule %d\n",
+ flow->flow_no, flow->rule_id);
+
+ for (i = FM10K_FFU_SLICE_START; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_ffu_always_mismatch(sw, i, flow->rule_id);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, flow->rule_id), 0);
+ }
+ fm10k_ffu_rule_free(flow->rule_id);
+}
+
+
+static int
+fm10k_ffu_configured_flowset_enable(struct fm10k_switch *sw)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flow *flow;
+
+ flowset = fm10k_config_flowset_current_get();
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+
+ return 0;
+}
+
+
+void
+fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flowset *cur_fs;
+ struct fm10k_cfg_flow *flow;
+
+ cur_fs = fm10k_config_flowset_current_get();
+ if (strcmp(cur_fs->name, new_name) == 0)
+ return;
+
+ flowset = fm10k_config_flowset_get(new_name);
+ if (flowset == NULL) {
+ FM10K_SW_ERR("Can not find flowset %s!!\n", new_name);
+ return;
+ }
+
+ /* disable current flowset */
+ flow = cur_fs->flow_head.next;
+ while (!fm10k_config_flow_list_end(&cur_fs->flow_head, flow)) {
+ fm10k_ffu_flow_disable(sw, flow);
+ flow = flow->next;
+ }
+
+ /* enable new flowset */
+ fm10k_config_flowset_current_set(flowset);
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+}
+
+
+int
+fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg)
+{
+ int ret = 0;
+ uint64_t data64;
+ uint16_t i, j;
+ uint32_t sglort = 0, dglort = 0;
+ uint16_t table_idx = 0;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_START;
+
+ sw->mcast_dest_table_idx = 1;
+ sw->mcast_len_table_idx = 1;
+ sw->mcast_vlan_table_idx = 1;
+
+ for (i = 0;
+ i < sizeof(fm10k_ffu_slice_cfgs) / sizeof(uint64_t);
+ i++) {
+ for (j = 0; j < FM10K_SW_FFU_NUM_SCENARIOS; j++) {
+ data64 = fm10k_ffu_slice_cfgs[i];
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CFG
+ (FM10K_FFU_SLICE_START + i, j),
+ data64);
+ }
+ FM10K_FFU_INIT_PRINT(cfg, "SET slice %d cfg = %#llx\n",
+ FM10K_FFU_SLICE_START + i,
+ (unsigned long long)data64);
+ }
+
+ for (table_idx = 0;
+ table_idx < FM10K_SW_FFU_SLICE_TCAM_ENTRIES / 2;
+ table_idx++)
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ fm10k_ffu_always_mismatch(sw, i, table_idx);
+
+ /*
+ * Create a TCAM entry to match each SGLORT that might be used, and
+ * set the corresponding SRAM action entries to ROUTE_GLORT to the
+ * corresponding DGLORT (see above table).
+ * SGLORT ROUTE is always the first slice
+ */
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pf_glort_get
+ (cfg->ext_port_map[i].map_no[0]);
+ } else if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ }
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, sglort, dglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ table_idx = FM10K_FFU_EXT_PORT_RULE_EGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, dglort, sglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ }
+
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i),
+ 0xf0f000f);
+ /* Set slice 0 to valid for all scenarios */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_VALID(i),
+ FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS);
+ }
+
+ /* Mark slice 0 as valid and all chunks as invalid */
+ data64 = 0;
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ data64 |= FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(i);
+ fm10k_write_switch_reg64(sw, FM10K_SW_FFU_MASTER_VALID, data64);
+
+ /*
+ * Set up the DGLORT map according to the desired
+ * SGLORT -> { DGLORT, logical_port } map.
+ */
+ for (i = 0; i < sw->info->num_peps; i++) {
+ if (sw->pep_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->pep_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->epl_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->epl_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ uint64_t dport_mask = 0;
+
+ if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ dport_mask =
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[0]].logical_port) |
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[1]].logical_port);
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++, dglort, dport_mask);
+ }
+ }
+
+ /* Ensure the rest of the DGLORT TCAM won't match */
+ for (table_idx = sw->glort_cam_ram_idx;
+ table_idx < FM10K_SW_GLORT_CAM_ENTRIES;
+ table_idx++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_GLORT_CAM(table_idx),
+ FM10K_SW_GLORT_CAM_MATCH_NONE);
+
+ ret = fm10k_ffu_configured_flowset_enable(sw);
+
+ fm10k_ffu_register_dump(sw);
+ fm10k_glort_register_dump(sw);
+ return ret;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.h b/drivers/net/fm10k/switch/fm10k_ffu.h
new file mode 100644
index 0000000..bcd9329
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_FFU_H_
+#define _FM10K_FFU_H_
+
+
+#include <stdint.h>
+
+#define FM10K_FFU_EXT_PORT_RULE_INGRESS(i) ((i) * 2)
+#define FM10K_FFU_EXT_PORT_RULE_EGRESS(i) ((i) * 2 + 1)
+#define FM10K_FFU_RULE_MAX FM10K_SW_FFU_RULE_MAX
+
+struct fm10k_switch;
+struct fm10k_dpdk_cfg;
+struct fm10k_cfg_flow;
+
+int fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg);
+
+int fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_dpdk_port, u16 dest_dpdk_port, uint16_t vlan);
+int fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_dpdk_port);
+
+void fm10k_ffu_flow_enable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flow_disable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name);
+
+#endif /* _FM10K_FFU_H */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
new file mode 100644
index 0000000..563f9ac
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -0,0 +1,1114 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_stats.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+
+
+#define FM10K_MAX_VLAN_COUNTER 63
+#define FM10K_NB_RX_STATS_BANKS 6
+#define FM10K_BINS_PER_RX_STATS_BANK 16
+#define FM10K_WORDS_PER_RX_STATS_COUNTER 4
+
+/* Max expected entries in read stats scatter gather list */
+#define MAX_STATS_SGLIST 128
+#define FM10K_RX_STATS_BASE (0xE00000)
+#define FM10K_MOD_BASE (0xE80000)
+#define FM10K_CM_APPLY_BASE (0xD40000)
+#define FM10K_EPL_BASE (0x0E0000)
+
+#define FM10K_RX_STATS_BANK(index1, index0, word) \
+ ((0x001000) * ((index1) - 0) + \
+ (0x000004) * ((index0) - 0) + \
+ (word) + (0x000000) + \
+ (FM10K_RX_STATS_BASE))
+#define FM10K_MOD_STATS_BANK_FRAME(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x025000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_MOD_STATS_BANK_BYTE(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x026000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_CM_APPLY_DROP_COUNT(index, word) \
+ ((0x000002) * ((index) - 0) + \
+ (word) + (0x000880) + \
+ (FM10K_CM_APPLY_BASE))
+#define FM10K_MAC_OVERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000021) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_JABBER_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000022) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000023) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_RUNT_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000024) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_OVERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000025) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000026) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_CODE_ERROR_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000027) + \
+ (FM10K_EPL_BASE))
+
+/* Rx Bank Definitions */
+#define FM10K_RX_STAT_BANK_TYPE 0
+#define FM10K_RX_STAT_BANK_SIZE 1
+#define FM10K_RX_STAT_BANK_PRI 2
+#define FM10K_RX_STAT_BANK_FWD_1 3
+#define FM10K_RX_STAT_BANK_FWD_2 4
+#define FM10K_RX_STAT_BANK_VLAN 5
+
+/* FM10K_RX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_RX_STAT_NONIP_L2UCAST 0
+#define FM10K_RX_STAT_NONIP_L2MCAST 1
+#define FM10K_RX_STAT_NONIP_L2BCAST 2
+#define FM10K_RX_STAT_IPV4_L2UCAST 3
+#define FM10K_RX_STAT_IPV4_L2MCAST 4
+#define FM10K_RX_STAT_IPV4_L2BCAST 5
+#define FM10K_RX_STAT_IPV6_L2UCAST 6
+#define FM10K_RX_STAT_IPV6_L2MCAST 7
+#define FM10K_RX_STAT_IPV6_L2BCAST 8
+#define FM10K_RX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_RX_STAT_CLASS_BASED_PAUSE 10
+#define FM10K_RX_STAT_FRAMING_ERR 11
+#define FM10K_RX_STAT_FCS_ERR 12
+
+/* FM10K_RX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_RX_STAT_LEN_LT_64 0
+#define FM10K_RX_STAT_LEN_EQ_64 1
+#define FM10K_RX_STAT_LEN_65_127 2
+#define FM10K_RX_STAT_LEN_128_255 3
+#define FM10K_RX_STAT_LEN_256_511 4
+#define FM10K_RX_STAT_LEN_512_1023 5
+#define FM10K_RX_STAT_LEN_1024_1522 6
+#define FM10K_RX_STAT_LEN_1523_2047 7
+#define FM10K_RX_STAT_LEN_2048_4095 8
+#define FM10K_RX_STAT_LEN_4096_8191 9
+#define FM10K_RX_STAT_LEN_8192_10239 10
+#define FM10K_RX_STAT_LEN_GE_10240 11
+
+/* FM10K_RX_STAT_BANK_PRI Bin Definitions */
+#define FM10K_RX_STAT_PRI_0 0
+#define FM10K_RX_STAT_PRI_1 1
+#define FM10K_RX_STAT_PRI_2 2
+#define FM10K_RX_STAT_PRI_3 3
+#define FM10K_RX_STAT_PRI_4 4
+#define FM10K_RX_STAT_PRI_5 5
+#define FM10K_RX_STAT_PRI_6 6
+#define FM10K_RX_STAT_PRI_7 7
+#define FM10K_RX_STAT_PRI_8 8
+#define FM10K_RX_STAT_PRI_9 9
+#define FM10K_RX_STAT_PRI_10 10
+#define FM10K_RX_STAT_PRI_11 11
+#define FM10K_RX_STAT_PRI_12 12
+#define FM10K_RX_STAT_PRI_13 13
+#define FM10K_RX_STAT_PRI_14 14
+#define FM10K_RX_STAT_PRI_15 15
+
+/* FM10K_RX_STAT_BANK_FWD_1 Bin Definitions */
+#define FM10K_RX_STAT_FID_FORWARDED 0
+#define FM10K_RX_STAT_FLOOD_FORWARDED 1
+#define FM10K_RX_STAT_SPECIALLY_HANDLED 2
+#define FM10K_RX_STAT_PARSER_ERROR_DROP 3
+#define FM10K_RX_STAT_ECC_ERROR_DROP 4
+#define FM10K_RX_STAT_TRAPPED 5
+#define FM10K_RX_STAT_PAUSE_DROPS 6
+#define FM10K_RX_STAT_STP_DROPS 7
+#define FM10K_RX_STAT_SECURITY_VIOLATIONS 8
+#define FM10K_RX_STAT_VLAN_TAG_DROPS 9
+#define FM10K_RX_STAT_VLAN_INGRESS_DROPS 10
+#define FM10K_RX_STAT_VLAN_EGRESS_DROPS 11
+#define FM10K_RX_STAT_GLORT_MISS_DROPS 12
+#define FM10K_RX_STAT_FFU_DROPS 13
+#define FM10K_RX_STAT_TRIGGER_DROPS 14
+#define FM10K_RX_STAT_OVERFLOW4 15
+
+/* FM10K_RX_STAT_BANK_FWD_2 Bin Definitions */
+#define FM10K_RX_STAT_POLICER_DROPS 0
+#define FM10K_RX_STAT_TTL_DROPS 1
+#define FM10K_RX_STAT_CM_PRIV_DROPS 2
+#define FM10K_RX_STAT_CM_SMP0_DROPS 3
+#define FM10K_RX_STAT_CM_SMP1_DROPS 4
+#define FM10K_RX_STAT_CM_RX_HOG_0_DROPS 5
+#define FM10K_RX_STAT_CM_RX_HOG_1_DROPS 6
+#define FM10K_RX_STAT_CM_TX_HOG_0_DROPS 7
+#define FM10K_RX_STAT_CM_TX_HOG_1_DROPS 8
+/* Bin 9 reserved */
+#define FM10K_RX_STAT_TRIGGER_REDIRECTS 10
+#define FM10K_RX_STAT_FLOOD_CONTROL_DROPS 11
+#define FM10K_RX_STAT_GLORT_FORWARDED 12
+#define FM10K_RX_STAT_LOOPBACK_SUPPRESS 13
+#define FM10K_RX_STAT_OTHERS 14
+#define FM10K_RX_STAT_OVERFLOW5 15
+
+/* FM10K_RX_STAT_BANK_VLAN Bin definitions */
+#define FM10K_RX_STAT_VLAN_UCAST 0
+#define FM10K_RX_STAT_VLAN_MCAST 1
+#define FM10K_RX_STAT_VLAN_BCAST 2
+
+/* Tx Bank Definitions */
+#define FM10K_TX_STAT_BANK_TYPE 0
+#define FM10K_TX_STAT_BANK_SIZE 1
+
+/* FM10K_TX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_TX_STAT_L2UCAST 0
+#define FM10K_TX_STAT_L2MCAST 1
+#define FM10K_TX_STAT_L2BCAST 2
+#define FM10K_TX_STAT_ERR_SENT 3
+#define FM10K_TX_STAT_TIMEOUT_DROP 4
+#define FM10K_TX_STAT_ERR_DROP 5
+#define FM10K_TX_STAT_ECC_DROP 6
+#define FM10K_TX_STAT_LOOPBACK_DROP 7
+#define FM10K_TX_STAT_TTL1_DROP 8
+#define FM10K_TX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_TX_STAT_CLASS_BASED_PAUSE 10
+
+/* FM10K_TX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_TX_STAT_LEN_LT_64 0
+#define FM10K_TX_STAT_LEN_EQ_64 1
+#define FM10K_TX_STAT_LEN_65_127 2
+#define FM10K_TX_STAT_LEN_128_255 3
+#define FM10K_TX_STAT_LEN_256_511 4
+#define FM10K_TX_STAT_LEN_512_1023 5
+#define FM10K_TX_STAT_LEN_1024_1522 6
+#define FM10K_TX_STAT_LEN_1523_2047 7
+#define FM10K_TX_STAT_LEN_2048_4095 8
+#define FM10K_TX_STAT_LEN_4096_8191 9
+#define FM10K_TX_STAT_LEN_8192_10239 10
+#define FM10K_TX_STAT_LEN_GE_10240 11
+
+
+/* This structure is used to build a table of all rx / tx counters */
+struct fm10k_port_count_mapping {
+ /* Bank in which the counter is located */
+ uint32_t bank;
+
+ /* Bin in which the counter is located */
+ uint32_t bin;
+
+ /*
+ * Offset (in bytes) of a the frame counter in
+ * the fm10k_counters structure
+ */
+ uint32_t frame_offset;
+
+ /*
+ * Offset (in bytes) of a the byte counter in
+ * the fm10k_counters structure
+ */
+ uint32_t byte_offset;
+};
+
+
+/*
+ * Table of all RX port counters in the RX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping rx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FRAMING_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FCS_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors),
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors_octets)},
+
+ /* Size Bank */
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_64_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_octets)},
+
+ /* priority Bank */
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_0,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[0]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[0])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_1,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[1]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[1])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_2,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[2]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[2])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_3,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[3]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[3])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_4,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[4]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[4])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_5,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[5]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[5])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_6,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[6]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[6])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_7,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[7]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[7])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_8,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[8]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[8])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_9,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[9]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[9])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_10,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[10]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[10])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_11,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[11]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[11])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_12,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[12]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[12])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_13,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[13]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[13])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_14,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[14]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[14])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_15,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[15]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[15])},
+
+ /* Forwarding Bank */
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FID_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FLOOD_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SPECIALLY_HANDLED,
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_pkts),
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PARSER_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_ECC_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parity_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parity_error_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRAPPED,
+ offsetof(struct fm10k_port_counters, cnt_trapped_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trapped_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PAUSE_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_STP_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SECURITY_VIOLATIONS,
+ offsetof(struct fm10k_port_counters, cnt_security_violation_pkts),
+ offsetof(struct fm10k_port_counters, cnt_security_violation_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_TAG_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_INGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_EGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_GLORT_MISS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FFU_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRIGGER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_POLICER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TTL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_PRIV_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TRIGGER_REDIRECTS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_FLOOD_CONTROL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_GLORT_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_LOOPBACK_SUPPRESS,
+ offsetof(struct fm10k_port_counters, cnt_loopback_drops_pkts),
+ offsetof(struct fm10k_port_counters, cnt_loopback_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_OTHERS,
+ offsetof(struct fm10k_port_counters, cnt_other_pkts),
+ offsetof(struct fm10k_port_counters, cnt_other_octets)},
+
+}; /* end rxPortCntMapTable */
+
+
+/*
+ * Table of all TX port counters in the TX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping tx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_SENT,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TIMEOUT_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ECC_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_LOOPBACK_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TTL1_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_octets)},
+
+ /* Size Bank */
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_64_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_octets)},
+
+}; /* end tx_port_cnt_map_table */
+
+
+/*
+ * Get a 32bit EPL counter by adding it to the scatter gather
+ * list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+static inline void
+fm10k_get_epl_port_stat32(struct fm10k_scatter_gather_entry *sg_list,
+ struct fm10k_port_counters *counters,
+ int *sg_list_cnt, int epl, int lane)
+{
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_oversized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_JABBER_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_jabber_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_UNDERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_undersized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_RUNT_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_fragment_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERRUN_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_over_run_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_CODE_ERROR_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_code_errors;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+}
+
+
+static void
+fm10k_read_scatter_gather(struct fm10k_switch *sw,
+ int n_entries,
+ struct fm10k_scatter_gather_entry *sg_list)
+{
+ int i;
+
+ for (i = 0; i < n_entries; i++) {
+ uint32_t addr = sg_list[i].addr;
+ uint32_t count = sg_list[i].count;
+ uint32_t *data = sg_list[i].data;
+
+ fm10k_read_switch_array(sw, addr, data, count);
+ }
+}
+
+
+static int
+fm10k_get_port_counters(struct fm10k_switch *sw,
+ struct fm10k_ext_port *ext_port,
+ struct fm10k_port_counters *counters)
+{
+ int phys_port;
+ int epl;
+ int lane;
+ struct timeval ts;
+ bool valid_ip_stats = true;
+ uint32_t i;
+ struct fm10k_scatter_gather_entry sg_list[MAX_STATS_SGLIST];
+ int sg_list_cnt = 0;
+ /* Temporary bin array to retrieve 128bit
+ * port counters (frame + bytes).
+ */
+ uint32_t cnt_rx_port_stats_bank
+ [FM10K_NB_RX_STATS_BANKS]
+ [FM10K_BINS_PER_RX_STATS_BANK]
+ [FM10K_WORDS_PER_RX_STATS_COUNTER];
+ /*
+ * Fill the counters structure with 0 to assure
+ * all fields not explicitly set below, which will
+ * be the fields not supported by the FM10000,
+ * will be 0 on return.
+ */
+ memset(counters, 0, sizeof(*counters));
+
+ phys_port = ext_port->portno;
+ epl = ext_port->eplno;
+ lane = ext_port->first_lane;
+
+ counters->cnt_version = FM10K_STATS_VERSION;
+
+ /*
+ * Reading counters for each RX bank
+ *
+ * Because the RX_STATS_BANK register contains
+ * both, frame count and byte count in a 128bit
+ * register, we first store the data in a temporary
+ * array.
+ * 1. Fill Scatter Gather to retrieve each bin
+ * counter from the rx bank and store in a temp
+ * array.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 128bit read of an rx counter to the scatter gather
+ * list.
+ * NOTE: Read values will be stored in the temporary
+ * array switchExt->cnt_rx_port_stats_bank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [out] - Array to store the
+ * 128bit values sgList
+ * [out] - scatter gather array to hold the reads sgListCnt
+ * [out] - number of entry in the sgList
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_RX_STATS_BANK(bank, (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data = cnt_rx_port_stats_bank[bank][bin];
+ sg_list[sg_list_cnt].count = 4;
+ sg_list_cnt++;
+}
+
+ /*
+ * 2. Fill in the scatter gather for tx bank
+ * counters. They will directly be stored
+ * in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 64bit read of a tx counter to the scatter gather list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+ uint32_t bank = tx_port_cnt_map_table[i].bank;
+ uint32_t bin = tx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = tx_port_cnt_map_table[i].frame_offset;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_MOD_STATS_BANK_FRAME(bank,
+ (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)(((uint8_t *)counters) + (frame_offset));
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ }
+
+ /*
+ * 3. Fill in the scatter gather for TX CM DROP
+ * and for EPL counters. They will directly be
+ * stored in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ sg_list[sg_list_cnt].addr =
+ FM10K_CM_APPLY_DROP_COUNT(phys_port, 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_cm_drop_pkts;
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ /* EPL Counters */
+ fm10k_get_epl_port_stat32(sg_list, counters, &sg_list_cnt, epl, lane);
+
+ /*
+ * 4. Execute scatter gather read.
+ */
+ if (sg_list_cnt >= MAX_STATS_SGLIST) {
+ /*
+ * Pretty static. Mainly to warn if something new added,
+ * but the array size is not adjust accordingly
+ */
+ FM10K_SW_ERR("Scatter list array %d for port %d overflow.\n",
+ sg_list_cnt, phys_port);
+ return -1;
+ }
+
+ /*
+ * Taking lock to protect temporary structures used to
+ * store 128b counters
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ /* now get the stats in one shot, optimized for fibm */
+ fm10k_read_scatter_gather(sw, sg_list_cnt, sg_list);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ counters->timestamp = ts.tv_sec * 1000000 + ts.tv_usec;
+
+ /*
+ * 5. Retrieve the frame/byte counts from 128bit
+ * registers stored in temporary array and set
+ * the proper fm_portCounter structure members.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Update the counter variable related to a 128bit HW counter
+ * NOTE: Read values will be retrieved from the temporary
+ * array switchExt->cnt_rx_PortStatsBank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [in] - Array to read the
+ * 128bit values from
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = rx_port_cnt_map_table[i].frame_offset;
+ uint32_t byte_offset = rx_port_cnt_map_table[i].byte_offset;
+
+ *((uint64_t *)(((uint8_t *)counters) + frame_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][1]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][0])));
+
+ *((uint64_t *)(((uint8_t *)counters) + byte_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][3]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][2])));
+ }
+
+ /*
+ * 6. Set some counters that are not available
+ * in HW but can be computed from two or more
+ * HW counters.
+ */
+ /*
+ * If IP parsing is enabled then the IP stats will be read and will be
+ * valid. Else all traffic, whether IP or not, is counted as _nonip.
+ */
+ if (valid_ip_stats) {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip +
+ counters->cnt_rx_ucst_pkts_ipv4 +
+ counters->cnt_rx_ucst_pkts_ipv6;
+
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip +
+ counters->cnt_rx_mcst_pkts_ipv4 +
+ counters->cnt_rx_mcst_pkts_ipv6;
+
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip +
+ counters->cnt_rx_bcst_pkts_ipv4 +
+ counters->cnt_rx_bcst_pkts_ipv6;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_octets_ipv4 =
+ counters->cnt_rx_ucst_octets_ipv4 +
+ counters->cnt_rx_mcst_octets_ipv4 +
+ counters->cnt_rx_bcst_octets_ipv4;
+
+ counters->cnt_rx_octets_ipv6 =
+ counters->cnt_rx_ucst_octets_ipv6 +
+ counters->cnt_rx_mcst_octets_ipv6 +
+ counters->cnt_rx_bcst_octets_ipv6;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip +
+ counters->cnt_rx_octets_ipv4 +
+ counters->cnt_rx_octets_ipv6;
+ } else {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip;
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip;
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip;
+ }
+
+ counters->cnt_trigger_drop_redir_pkts =
+ counters->cnt_trigger_redir_pkts +
+ counters->cnt_trigger_drop_pkts;
+
+ /* Emulate Tx _octets counter using the sum of all size bins. */
+ counters->cnt_tx_octets += counters->cnt_tx_minto63_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_64_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_65to127_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_128to255_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_256to511_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_512to1023_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1024to1522_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1523to2047_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_2048to4095_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_4096to8191_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_8192to10239_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_10240tomax_octets;
+
+ return 0;
+} /* end fm10k_get_port_counters */
+
+
+struct fm10k_stats_data {
+ uint64_t rx_pkts;
+ uint64_t tx_pkts;
+ uint64_t rx_drop;
+};
+
+
+static uint64_t
+fm10k_stats_ffu_count_get(struct fm10k_switch *sw, int table_id)
+{
+ uint64_t data;
+
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x4000 * FM10K_SW_FFU_CNT_BANK +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x8000);
+ } else {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x800 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x10000);
+ }
+ return data;
+}
+
+static void
+fm10k_stats_epl_ffu_print(struct fm10k_switch *sw, int epl)
+{
+ int table_id;
+ uint64_t data;
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_INGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] ingress %-12llu\n",
+ table_id, (unsigned long long)(data));
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_EGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] egress %-12llu\n",
+ table_id, (unsigned long long)(data));
+}
+
+#define FM10K_STATS_RULE_NUM_MAX 100
+static uint16_t fm10k_stats_rule_list[FM10K_STATS_RULE_NUM_MAX];
+void
+fm10k_stats_rule_count_reg(uint16_t rule_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0) {
+ fm10k_stats_rule_list[i] = rule_id;
+ break;
+ }
+ }
+}
+
+void
+fm10k_stats_ffu_count_print(struct fm10k_switch *sw)
+{
+ int i, rule_id;
+ uint64_t data;
+ static uint64_t ffu_count[FM10K_STATS_RULE_NUM_MAX];
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0)
+ continue;
+
+ rule_id = fm10k_stats_rule_list[i];
+ data = fm10k_stats_ffu_count_get(sw, rule_id);
+ FM10K_SW_INFO("FFU[%d] count %-12llu\n",
+ rule_id,
+ (unsigned long long)
+ (data - ffu_count[rule_id]));
+ ffu_count[rule_id] = data;
+ }
+}
+
+void
+fm10k_stats_epl_port_print(struct fm10k_switch *sw)
+{
+ int i, lport;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_stats_data data;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ lport = sw->epl_map[i].logical_port;
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ data.rx_pkts =
+ counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts =
+ counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ FM10K_SW_INFO("EPL port %-2d lport %-5d tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, lport,
+ (unsigned long long)
+ (data.tx_pkts - last_data[i].tx_pkts),
+ (unsigned long long)
+ (data.rx_pkts - last_data[i].rx_pkts),
+ (unsigned long long)
+ (data.rx_drop - last_data[i].rx_drop));
+ last_data[i] = data;
+
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU))
+ fm10k_stats_epl_ffu_print(sw, i);
+ }
+}
+
+
+static void
+fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
+{
+ unsigned int i;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ uint64_t *pdata;
+
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ rx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d rx bank %d idx %d pkt %llu\n",
+ lport,
+ rx_port_cnt_map_table[i].bank,
+ rx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ tx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d tx bank %d idx %d pkt %llu\n",
+ lport,
+ tx_port_cnt_map_table[i].bank,
+ tx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+}
+
+void
+fm10k_stats_port_bank_print(struct fm10k_switch *sw)
+{
+ int i;
+
+ for (i = 0;
+ i < FM10K_SW_EPLS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->epl_map[i].logical_port);
+ }
+ for (i = 0;
+ i < FM10K_SW_PEPS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->pep_map[i].logical_port);
+ }
+}
+
+
+void *
+fm10k_switch_process_stats(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ usec_delay(5000000);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_ENABLE))
+ return NULL;
+
+ if (!sw->dpdk_cfg->stats_interval)
+ return NULL;
+
+ while (sw->detaching == 0) {
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_PORT)) {
+ FM10K_SW_INFO("--- port statistic ---\n");
+ fm10k_stats_epl_port_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU)) {
+ FM10K_SW_INFO("--- ffu statistic ---\n");
+ fm10k_stats_ffu_count_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_MORE)) {
+ FM10K_SW_INFO("--- detail statistic ---\n");
+ fm10k_stats_port_bank_print(sw);
+ }
+ usec_delay(1000000 * sw->dpdk_cfg->stats_interval);
+ }
+ return NULL;
+}
+
diff --git a/drivers/net/fm10k/switch/fm10k_stats.h b/drivers/net/fm10k/switch/fm10k_stats.h
new file mode 100644
index 0000000..0bc84e4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.h
@@ -0,0 +1,257 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#ifndef _fm10k_stats_h_
+#define _fm10k_stats_h_
+
+
+#include <stdint.h>
+
+
+#define FM10K_STATS_VERSION (1 << 4)
+
+
+struct fm10k_scatter_gather_entry {
+ uint32_t addr;
+ uint32_t count;
+ uint32_t *data;
+
+};
+
+
+struct fm10k_port_counters {
+ uint64_t cnt_version;
+ uint64_t cnt_rx_ucst_pkts;
+ uint64_t cnt_rx_ucst_pkts_nonip;
+ uint64_t cnt_rx_ucst_pkts_ipv4;
+ uint64_t cnt_rx_ucst_pkts_ipv6;
+ uint64_t cnt_rx_bcst_pkts;
+ uint64_t cnt_rx_bcst_pkts_nonip;
+ uint64_t cnt_rx_bcst_pkts_ipv4;
+ uint64_t cnt_rx_bcst_pkts_ipv6;
+ uint64_t cnt_rx_mcst_pkts;
+ uint64_t cnt_rx_mcst_pkts_nonip;
+ uint64_t cnt_rx_mcst_pkts_ipv4;
+ uint64_t cnt_rx_mcst_pkts_ipv6;
+ uint64_t cnt_rx_pause_pkts;
+ uint64_t cnt_rx_cbpause_pkts;
+ uint64_t cnt_rx_fcs_errors;
+ uint64_t cnt_rx_symbol_errors;
+ uint64_t cnt_rx_framesize_errors;
+ uint64_t cnt_rx_framing_error_pkts;
+ uint64_t cnt_rx_minto63_pkts;
+ uint64_t cnt_rx_64_pkts;
+ uint64_t cnt_rx_65to127_pkts;
+ uint64_t cnt_rx_128to255_pkts;
+ uint64_t cnt_rx_256to511_pkts;
+ uint64_t cnt_rx_512to1023_pkts;
+ uint64_t cnt_rx_1024to1522_pkts;
+ uint64_t cnt_rx_1523to2047_pkts;
+ uint64_t cnt_rx_2048to4095_pkts;
+ uint64_t cnt_rx_4096to8191_pkts;
+ uint64_t cnt_rx_8192to10239_pkts;
+ uint64_t cnt_rx_10240tomax_pkts;
+ uint64_t cnt_rx_minto63_octets;
+ uint64_t cnt_rx_64_octets;
+ uint64_t cnt_rx_65to127_octets;
+ uint64_t cnt_rx_128to255_octets;
+ uint64_t cnt_rx_256to511_octets;
+ uint64_t cnt_rx_512to1023_octets;
+ uint64_t cnt_rx_1024to1522_octets;
+ uint64_t cnt_rx_1523to2047_octets;
+ uint64_t cnt_rx_2048to4095_octets;
+ uint64_t cnt_rx_4096to8191_octets;
+ uint64_t cnt_rx_8192to10239_octets;
+ uint64_t cnt_rx_10240tomax_octets;
+ uint64_t cnt_rx_octets_nonip;
+ uint64_t cnt_rx_octets_ipv4;
+ uint64_t cnt_rx_octets_ipv6;
+ uint64_t cnt_rx_ucst_octets_nonip;
+ uint64_t cnt_rx_ucst_octets_ipv4;
+ uint64_t cnt_rx_ucst_octets_ipv6;
+ uint64_t cnt_rx_bcst_octets_nonip;
+ uint64_t cnt_rx_bcst_octets_ipv4;
+ uint64_t cnt_rx_bcst_octets_ipv6;
+ uint64_t cnt_rx_mcst_octets_nonip;
+ uint64_t cnt_rx_mcst_octets_ipv4;
+ uint64_t cnt_rx_mcst_octets_ipv6;
+ uint64_t cnt_rx_pause_octets;
+ uint64_t cnt_rx_cbpause_octets;
+ uint64_t cnt_rx_fcs_errors_octets;
+ uint64_t cnt_rx_framing_error_octets;
+ uint64_t cnt_rx_good_octets;
+ uint64_t cnt_rx_bad_octets;
+ uint64_t cnt_rx_priority_pkts[16];
+ uint64_t cnt_rx_invalid_priority_pkts;
+ uint64_t cnt_rx_priority_octets[16];
+ uint64_t cnt_rx_invalid_priority_octets;
+ uint64_t cnt_fid_forwarded_pkts;
+ uint64_t cnt_flood_forwarded_pkts;
+ uint64_t cnt_glort_switched_pkts;
+ uint64_t cnt_glort_routed_pkts;
+ uint64_t cnt_specially_handled_pkts;
+ uint64_t cnt_parse_err_drop_pkts;
+ uint64_t cnt_parity_error_pkts;
+ uint64_t cnt_trapped_pkts;
+ uint64_t cnt_pause_drop_pkts;
+ uint64_t cnt_stp_drop_pkts;
+ uint64_t cnt_stp_ingress_drops_pkts;
+ uint64_t cnt_stp_egress_drops_pkts;
+ uint64_t cnt_reserved_trap_pkts;
+ uint64_t cnt_security_violation_pkts;
+ uint64_t cnt_vlan_tag_drop_pkts;
+ uint64_t cnt_vlan_ingressbv_pkts;
+ uint64_t cnt_vlan_egressbv_pkts;
+ uint64_t cnt_loopback_drops_pkts;
+ uint64_t cnt_glort_miss_drop_pkts;
+ uint64_t cnt_ffu_drop_pkts;
+ uint64_t cnt_invalid_drop_pkts;
+ uint64_t cnt_policer_drop_pkts;
+ uint64_t cnt_ttl_drop_pkts;
+ uint64_t cnt_global_wmdrop_pkts;
+ uint64_t cnt_rx_mpdrop_pkts;
+ uint64_t cnt_rx_hogdrop_pkts;
+ uint64_t cnt_tx_hogdrop_pkts;
+ uint64_t cnt_other_pkts;
+ uint64_t cnt_flood_control_drop_pkts;
+ uint64_t cnt_cmpriv_drop_pkts;
+ uint64_t cnt_smp0_drop_pkts;
+ uint64_t cnt_smp1_drop_pkts;
+ uint64_t cnt_rx_hog0_drop_pkts;
+ uint64_t cnt_rx_hog1_drop_pkts;
+ uint64_t cnt_tx_hog0_drop_pkts;
+ uint64_t cnt_tx_hog1_drop_pkts;
+ uint64_t cnt_rate_limit0_drop_pkts;
+ uint64_t cnt_rate_limit1_drop_pkts;
+ uint64_t cnt_bad_smp_drop_pkts;
+ uint64_t cnt_trigger_drop_redir_pkts;
+ uint64_t cnt_trigger_drop_pkts;
+ uint64_t cnt_trigger_redir_pkts;
+ uint64_t cnt_glort_forwarded_pkts;
+ uint64_t cnt_trigger_mirrored_pkts;
+ uint64_t cnt_broadcast_drop_pkts;
+ uint64_t cnt_dlf_drop_pkts;
+ uint64_t cnt_rx_cm_drop_pkts;
+ uint64_t cnt_fid_forwarded_octets;
+ uint64_t cnt_flood_forwarded_octets;
+ uint64_t cnt_specially_handled_octets;
+ uint64_t cnt_parse_err_drop_octets;
+ uint64_t cnt_parity_error_octets;
+ uint64_t cnt_trapped_octets;
+ uint64_t cnt_pause_drop_octets;
+ uint64_t cnt_stp_drop_octets;
+ uint64_t cnt_security_violation_octets;
+ uint64_t cnt_vlan_tag_drop_octets;
+ uint64_t cnt_vlan_ingressbv_octets;
+ uint64_t cnt_vlan_egressbv_octets;
+ uint64_t cnt_loopback_drop_octets;
+ uint64_t cnt_glort_miss_drop_octets;
+ uint64_t cnt_ffu_drop_octets;
+ uint64_t cnt_policer_drop_octets;
+ uint64_t cnt_ttl_drop_octets;
+ uint64_t cnt_other_octets;
+ uint64_t cnt_flood_control_drop_octets;
+ uint64_t cnt_cmpriv_drop_octets;
+ uint64_t cnt_smp0_drop_octets;
+ uint64_t cnt_smp1_drop_octets;
+ uint64_t cnt_rx_hog0_drop_octets;
+ uint64_t cnt_rx_hog1_drop_octets;
+ uint64_t cnt_tx_hog0_drop_octets;
+ uint64_t cnt_tx_hog1_drop_octets;
+ uint64_t cnt_trigger_drop_octets;
+ uint64_t cnt_trigger_redir_octets;
+ uint64_t cnt_glort_forwarded_octets;
+ uint64_t cnt_tx_ucst_pkts;
+ uint64_t cnt_tx_bcst_pkts;
+ uint64_t cnt_tx_mcst_pkts;
+ uint64_t cnt_tx_ucst_pkts_nonip;
+ uint64_t cnt_tx_bcst_pkts_nonip;
+ uint64_t cnt_tx_mcst_pkts_nonip;
+ uint64_t cnt_tx_ucst_pkts_ip;
+ uint64_t cnt_tx_bcst_pkts_ip;
+ uint64_t cnt_tx_mcst_pkts_ip;
+ uint64_t cnt_tx_pause_pkts;
+ uint64_t cnt_tx_cbpause_pkts;
+ uint64_t cnt_tx_fcs_err_drop_pkts;
+ uint64_t cnt_tx_framing_error_pkts;
+ uint64_t cnt_tx_error_sent_pkts;
+ uint64_t cnt_tx_error_drop_pkts;
+ uint64_t cnt_tx_timeout_pkts;
+ uint64_t cnt_tx_outofmem_err_pkts;
+ uint64_t cnt_tx_unrepair_ecc_pkts;
+ uint64_t cnt_tx_loopback_pkts;
+ uint64_t cnt_tx_ttl_drop_pkts;
+ uint64_t cnt_tx_minto63_pkts;
+ uint64_t cnt_tx_64_pkts;
+ uint64_t cnt_tx_65to127_pkts;
+ uint64_t cnt_tx_128to255_pkts;
+ uint64_t cnt_tx_256to511_pkts;
+ uint64_t cnt_tx_512to1023_pkts;
+ uint64_t cnt_tx_1024to1522_pkts;
+ uint64_t cnt_tx_1523to2047_pkts;
+ uint64_t cnt_tx_2048to4095_pkts;
+ uint64_t cnt_tx_4096to8191_pkts;
+ uint64_t cnt_tx_8192to10239_pkts;
+ uint64_t cnt_tx_10240tomax_pkts;
+ uint64_t cnt_tx_minto63_octets;
+ uint64_t cnt_tx_64_octets;
+ uint64_t cnt_tx_65to127_octets;
+ uint64_t cnt_tx_128to255_octets;
+ uint64_t cnt_tx_256to511_octets;
+ uint64_t cnt_tx_512to1023_octets;
+ uint64_t cnt_tx_1024to1522_octets;
+ uint64_t cnt_tx_1523to2047_octets;
+ uint64_t cnt_tx_2048to4095_octets;
+ uint64_t cnt_tx_4096to8191_octets;
+ uint64_t cnt_tx_8192to10239_octets;
+ uint64_t cnt_tx_10240tomax_octets;
+ uint64_t cnt_tx_ucst_octets_nonip;
+ uint64_t cnt_tx_bcst_octets_nonip;
+ uint64_t cnt_tx_mcst_octets_nonip;
+ uint64_t cnt_tx_ucst_octets_ip;
+ uint64_t cnt_tx_bcst_octets_ip;
+ uint64_t cnt_tx_mcst_octets_ip;
+ uint64_t cnt_tx_ucst_octets;
+ uint64_t cnt_tx_mcst_octets;
+ uint64_t cnt_tx_bcst_octets;
+ uint64_t cnt_tx_fcs_err_drop_octets;
+ uint64_t cnt_tx_octets;
+ uint64_t cnt_tx_error_octets;
+ uint64_t cnt_tx_framing_error_octets;
+ uint64_t cnt_tx_pause_octets;
+ uint64_t cnt_tx_cbpause_octets;
+ uint64_t cnt_tx_fcs_errored_octets;
+ uint64_t cnt_tx_error_sent_octets;
+ uint64_t cnt_tx_timeout_octets;
+ uint64_t cnt_tx_outofmem_err_octets;
+ uint64_t cnt_tx_unrepair_ecc_octets;
+ uint64_t cnt_tx_loopback_octets;
+ uint64_t cnt_tx_ttl_drop_octets;
+ uint64_t cnt_tx_priority_octets[16];
+ uint64_t cnt_under_run_pkts;
+ uint64_t cnt_over_run_pkts;
+ uint64_t cnt_rx_fragment_pkts;
+ uint64_t cnt_rx_undersized_pkts;
+ uint64_t cnt_rx_jabber_pkts;
+ uint64_t cnt_corrupted_pkts;
+ uint64_t cnt_code_errors;
+ uint64_t cnt_rx_oversized_pkts;
+ uint64_t cnt_tx_fcs_errored_pkts;
+ uint64_t cnt_stats_drop_count_tx;
+ uint64_t cnt_stats_drop_count_rx;
+ uint64_t cnt_tx_mirror_pkts;
+ uint64_t cnt_tx_mirror_octets;
+ uint64_t cnt_tx_cmdrop_pkts;
+ uint64_t timestamp;
+};
+
+void fm10k_stats_rule_count_reg(uint16_t rule_id);
+void *fm10k_switch_process_stats(void *ctx_);
+
+void fm10k_stats_epl_port_print(struct fm10k_switch *sw);
+void fm10k_stats_dpdk_port_print(struct fm10k_switch *sw);
+void fm10k_stats_ffu_count_print(struct fm10k_switch *sw);
+void fm10k_stats_port_bank_print(struct fm10k_switch *sw);
+
+#endif /* _fm10k_stats_h */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
index 2423dbf..b4b363c 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.h
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -76,6 +76,37 @@
#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+/*
+ * GLORT MAP
+ */
+#define FM10K_SW_EPLA_GLORT 0x10
+#define FM10K_SW_EPLB_GLORT 0x20
+#define FM10K_SW_PEP01_GLORT 0x30
+#define FM10K_SW_PEP23_GLORT 0x40
+#define FM10K_SW_PEP45_GLORT 0x50
+#define FM10K_SW_PEP67_GLORT 0x60
+
+/*
+ * logical port number
+ */
+#define FM10K_SW_EPLA_LOGICAL_PORT 1
+#define FM10K_SW_EPLB_LOGICAL_PORT 2
+
+#define FM10K_SW_PEP01_LOGICAL_PORT 3
+#define FM10K_SW_PEP23_LOGICAL_PORT 4
+#define FM10K_SW_PEP45_LOGICAL_PORT 5
+#define FM10K_SW_PEP67_LOGICAL_PORT 6
+
+/*
+ * physical port number
+ */
+#define FM10K_SW_EPLA_PHYSICAL_PORT 0
+#define FM10K_SW_EPLB_PHYSICAL_PORT 4
+
+#define FM10K_SW_PEP01_PHYSICAL_PORT 36
+#define FM10K_SW_PEP23_PHYSICAL_PORT 40
+#define FM10K_SW_PEP45_PHYSICAL_PORT 44
+#define FM10K_SW_PEP67_PHYSICAL_PORT 48
#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
@@ -111,6 +142,7 @@
#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
/*
* SWITCH
*/
@@ -125,6 +157,15 @@ struct fm10k_device_info {
uint8_t num_peps;
};
+static struct fm10k_device_info fm10k_device_table[] = {
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4,
+ "Silicom PE3100G2DQiR-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4,
+ "Silicom PE3100G2DQiRL-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4,
+ "Silicom PE3100G2DQiRM-QX4/QS4/QL4", 2, 100, 2, 4 },
+};
+
struct fm10k_i2c;
struct fm10k_sbus;
@@ -287,6 +328,114 @@ struct fm10k_switch {
fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
}
+
+static struct fm10k_sw_port_map fm10k_pep_port_map[FM10K_SW_PEPS_SUPPORTED] = {
+ {
+ FM10K_SW_PEP01_GLORT,
+ FM10K_SW_PEP01_LOGICAL_PORT,
+ FM10K_SW_PEP01_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP23_GLORT,
+ FM10K_SW_PEP23_LOGICAL_PORT,
+ FM10K_SW_PEP23_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP45_GLORT,
+ FM10K_SW_PEP45_LOGICAL_PORT,
+ FM10K_SW_PEP45_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP67_GLORT,
+ FM10K_SW_PEP67_LOGICAL_PORT,
+ FM10K_SW_PEP67_PHYSICAL_PORT
+ },
+};
+
+static struct fm10k_sw_port_map fm10k_epl_port_map[FM10K_SW_EPLS_SUPPORTED] = {
+ {
+ FM10K_SW_EPLA_GLORT,
+ FM10K_SW_EPLA_LOGICAL_PORT,
+ FM10K_SW_EPLA_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_EPLB_GLORT,
+ FM10K_SW_EPLB_LOGICAL_PORT,
+ FM10K_SW_EPLB_PHYSICAL_PORT
+ },
+};
+
+static inline uint32_t
+fm10k_switch_pf_logical_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_epl_logical_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_vf_glort_get(uint8_t vf_no)
+{
+ return FM10K_SW_VF_GLORT_START + vf_no;
+}
+
+static inline uint32_t
+fm10k_switch_pf_glort_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_epl_glort_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2)
+{
+ uint8_t idx;
+ if (pf1 > pf2)
+ idx = (pf2 & 0xf) | (pf1 << 4 & 0xf0);
+ else
+ idx = (pf1 & 0xf) | (pf2 << 4 & 0xf0);
+ return FM10K_SW_PFS_GLORT_START + idx;
+}
+
+
+static inline struct fm10k_device_info*
+fm10k_get_device_info(struct fm10k_hw *hw)
+{
+ unsigned int i;
+ struct fm10k_device_info *info;
+ uint16_t pci_vendor = hw->vendor_id;
+ uint16_t pci_device = hw->device_id;
+ uint16_t pci_subvendor = hw->subsystem_vendor_id;
+ uint16_t pci_subdevice = hw->subsystem_device_id;
+
+ if (pci_vendor != FM10K_SW_VENDOR_ID_INTEL ||
+ pci_device != FM10K_SW_DEV_ID_FM10K)
+ return (NULL);
+
+ for (i = 0;
+ i < sizeof(fm10k_device_table) /
+ sizeof(fm10k_device_table[0]);
+ i++) {
+ info = &fm10k_device_table[i];
+ if (pci_subvendor == info->subvendor &&
+ pci_subdevice == info->subdevice) {
+ return info;
+ }
+ }
+
+ return NULL;
+}
+
+
#define fm10k_udelay usec_delay
typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
@@ -312,15 +461,6 @@ struct fm10k_flow_list *
void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
-uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
-uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
-uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
-uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
-uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
-uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
-uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
- uint16_t vlan1, uint16_t vlan2, bool *p_new);
-
int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 4/5] net/fm10k: add flow interface and switch management
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
` (2 preceding siblings ...)
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add flow interface to support offload flow into HW.
It supports parse vlan and parse mpls, all these
data will be transffered to ffu data.
Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
It create 3 threads. One for interrupt handler, one for
LED controller, one for statistics.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 2 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_stats.c | 128 ++
drivers/net/fm10k/switch/fm10k_switch.c | 2376 +++++++++++++++++++++++++++++++
5 files changed, 3404 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 86a7635..a9d6b5c 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -92,6 +92,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_flow.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_switch.c
endif
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_flow.c b/drivers/net/fm10k/switch/fm10k_flow.c
new file mode 100644
index 0000000..353f021
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.c
@@ -0,0 +1,872 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <sys/queue.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_eth_ctrl.h>
+#include <rte_tailq.h>
+#include <rte_flow_driver.h>
+
+#include "fm10k_flow.h"
+#include "fm10k_switch.h"
+#include "fm10k_ffu.h"
+#include "fm10k_config.h"
+
+
+static int fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static struct rte_flow *fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static int fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error);
+static int fm10k_flow_flush(struct rte_eth_dev *dev,
+ struct rte_flow_error *error);
+static int fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error);
+
+const struct rte_flow_ops fm10k_flow_ops = {
+ .validate = fm10k_flow_validate,
+ .create = fm10k_flow_create,
+ .destroy = fm10k_flow_destroy,
+ .flush = fm10k_flow_flush,
+};
+
+union fm10k_filter_t cons_filter;
+enum rte_filter_type fm10k_cons_filter_type = RTE_ETH_FILTER_NONE;
+
+/**
+ * MPLS filter configuration.
+ */
+enum fm10k_mpls_type {
+ FM10K_MPLS_TYPE_UNI,
+ FM10K_MPLS_TYPE_MULTI,
+};
+
+enum fm10k_mpls_action {
+ FM10K_MPLS_ACTION_DROP,
+ FM10K_MPLS_ACTION_QUEUE,
+};
+
+struct fm10k_mpls_filter_conf {
+ enum fm10k_mpls_type mpls_type; /**< mandatory for MPLS */
+ uint32_t mpls_header; /**< MPLS header */
+ uint32_t mpls_header_mask; /**< MPLS header mask */
+ enum fm10k_mpls_action mpls_action;
+ uint16_t queue;
+ uint8_t ffu_id;
+ uint8_t ffu_prio;
+};
+
+/**
+ * VLAN filter configuration.
+ */
+struct fm10k_vlan_filter_conf {
+ int ffu_id;
+ uint8_t ffu_prio;
+ uint8_t is_ingress;
+ uint8_t port;
+ uint8_t in_ext_port;
+ uint8_t out_ext_port;
+ uint16_t in_vlan;
+ uint16_t out_vlan;
+};
+
+
+union fm10k_filter_t {
+ struct fm10k_mpls_filter_conf mpls_filter;
+ struct fm10k_vlan_filter_conf vlan_filter;
+};
+
+typedef int (*parse_filter_t)(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+struct fm10k_valid_pattern {
+ enum rte_flow_item_type *items;
+ parse_filter_t parse_filter;
+};
+
+static enum rte_flow_item_type pattern_mpls_1[] = {
+ RTE_FLOW_ITEM_TYPE_ETH,
+ RTE_FLOW_ITEM_TYPE_MPLS,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_1[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_2[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static int fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+static struct fm10k_valid_pattern fm10k_supported_patterns[] = {
+ /* MPLS */
+ { pattern_mpls_1, fm10k_flow_parse_mpls_filter },
+ /* VLAN */
+ { pattern_vlan_1, fm10k_flow_parse_vlan_filter },
+ /* VLAN */
+ { pattern_vlan_2, fm10k_flow_parse_vlan_filter },
+};
+
+static const struct rte_flow_action *
+fm10k_next_item_of_action(const struct rte_flow_action *actions,
+ uint32_t *index)
+{
+ static const struct rte_flow_action *act;
+
+ act = actions + *index;
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID) {
+ (*index)++;
+ act = actions + *index;
+ }
+ return act;
+}
+
+/* Find the first VOID or non-VOID item pointer */
+static const struct rte_flow_item *
+fm10k_find_first_item(const struct rte_flow_item *item, bool is_void)
+{
+ bool is_find;
+
+ while (item->type != RTE_FLOW_ITEM_TYPE_END) {
+ if (is_void)
+ is_find = item->type == RTE_FLOW_ITEM_TYPE_VOID;
+ else
+ is_find = item->type != RTE_FLOW_ITEM_TYPE_VOID;
+ if (is_find)
+ break;
+ item++;
+ }
+ return item;
+}
+
+/* Skip all VOID items of the pattern */
+static void
+fm10k_pattern_skip_void_item(struct rte_flow_item *items,
+ const struct rte_flow_item *pattern)
+{
+ uint32_t cpy_count = 0;
+ const struct rte_flow_item *pb = pattern, *pe = pattern;
+
+ for (;;) {
+ /* Find a non-void item first */
+ pb = fm10k_find_first_item(pb, false);
+ if (pb->type == RTE_FLOW_ITEM_TYPE_END) {
+ pe = pb;
+ break;
+ }
+
+ /* Find a void item */
+ pe = fm10k_find_first_item(pb + 1, true);
+
+ cpy_count = pe - pb;
+ rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+
+ items += cpy_count;
+
+ if (pe->type == RTE_FLOW_ITEM_TYPE_END) {
+ pb = pe;
+ break;
+ }
+
+ pb = pe + 1;
+ }
+ /* Copy the END item. */
+ rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+}
+
+/* Check if the pattern matches a supported item type array */
+static bool
+fm10k_match_pattern(enum rte_flow_item_type *item_array,
+ struct rte_flow_item *pattern)
+{
+ struct rte_flow_item *item = pattern;
+
+ while ((*item_array == item->type) &&
+ (*item_array != RTE_FLOW_ITEM_TYPE_END)) {
+ item_array++;
+ item++;
+ }
+
+ return (*item_array == RTE_FLOW_ITEM_TYPE_END &&
+ item->type == RTE_FLOW_ITEM_TYPE_END);
+}
+
+/* Find if there's parse filter function matched */
+static parse_filter_t
+fm10k_find_parse_filter_func(struct rte_flow_item *pattern, uint32_t *idx)
+{
+ parse_filter_t parse_filter = NULL;
+ uint8_t i = *idx;
+
+ for (; i < RTE_DIM(fm10k_supported_patterns); i++) {
+ if (fm10k_match_pattern(fm10k_supported_patterns[i].items,
+ pattern)) {
+ parse_filter = fm10k_supported_patterns[i].parse_filter;
+ break;
+ }
+ }
+
+ *idx = ++i;
+
+ return parse_filter;
+}
+
+/* Parse attributes */
+static int
+fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ /* Not supported */
+ if (attr->group) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+ attr, "Not support group.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+/*
+ * MPLS
+ */
+/* 1. Last in item should be NULL as range is not supported.
+ * 2. Supported filter types: MPLS label.
+ * 3. Mask of fields which need to be matched should be
+ * filled with 1.
+ * 4. Mask of fields which needn't to be matched should be
+ * filled with 0.
+ */
+static int
+fm10k_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_mpls *mpls_spec;
+ const struct rte_flow_item_mpls *mpls_mask;
+ const struct rte_flow_item_eth *eth_spec;
+ enum rte_flow_item_type item_type;
+ const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
+ uint32_t label_be = 0;
+ uint32_t be_mask = 0;
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_ETH:
+ if (!item->spec || item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid ETH item");
+ return -rte_errno;
+ }
+ eth_spec =
+ (const struct rte_flow_item_eth *)item->spec;
+
+ if (rte_be_to_cpu_16(eth_spec->type) == 0x8847) {
+ filter->mpls_type = FM10K_MPLS_TYPE_UNI;
+ } else if (rte_be_to_cpu_16(eth_spec->type) == 0x8848) {
+ filter->mpls_type = FM10K_MPLS_TYPE_MULTI;
+ } else {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Invalid ETH item");
+ return -rte_errno;
+ }
+ break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ mpls_spec =
+ (const struct rte_flow_item_mpls *)item->spec;
+ mpls_mask =
+ (const struct rte_flow_item_mpls *)item->mask;
+
+ if (!mpls_spec || !mpls_mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS item");
+ return -rte_errno;
+ }
+
+ if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS label mask");
+ return -rte_errno;
+ }
+ rte_memcpy(((uint8_t *)&label_be + 1),
+ mpls_spec->label_tc_s, 3);
+ rte_memcpy(((uint8_t *)&be_mask + 1),
+ mpls_mask->label_tc_s, 3);
+ filter->mpls_header =
+ rte_be_to_cpu_32(label_be) >> 4;
+ filter->mpls_header_mask =
+ rte_be_to_cpu_32(be_mask) >> 4;
+
+ fm10k_cons_filter_type = RTE_ETH_FILTER_TUNNEL;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* MPLS action only supports QUEUE or DROP. */
+static int
+fm10k_flow_parse_mpls_action(const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ const struct rte_flow_action_queue *act_q;
+ uint32_t index = 0;
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_QUEUE &&
+ act->type != RTE_FLOW_ACTION_TYPE_DROP) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ if (act->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->mpls_action = FM10K_MPLS_ACTION_QUEUE;
+ filter->queue = act_q->index;
+ } else {
+ filter->mpls_action = FM10K_MPLS_ACTION_DROP;
+ }
+
+ /* Check if the next non-void item is END */
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ struct fm10k_mpls_filter_conf *mpls_filter =
+ &filter->mpls_filter;
+ int ret;
+
+ ret = fm10k_flow_parse_mpls_pattern(dev, pattern,
+ error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_mpls_action(actions, error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+
+/*
+ * VLAN
+ */
+static int
+fm10k_flow_parse_vlan_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_vlan *vlan_spec;
+ const struct rte_flow_item_phy_port *pp_spec;
+ enum rte_flow_item_type item_type;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan pattern ffu id %d", filter->ffu_id);
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_VLAN:
+ vlan_spec =
+ (const struct rte_flow_item_vlan *)item->spec;
+
+ if (!vlan_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid VLAN item");
+ return -rte_errno;
+ }
+ break;
+
+ case RTE_FLOW_ITEM_TYPE_PHY_PORT:
+ pp_spec =
+ (const struct rte_flow_item_phy_port *)item->spec;
+ if (!pp_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid PHY PORT item");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_flow_parse_vlan_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ uint32_t index = 0;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan action name %s ffu id %d",
+ dev->device->name, filter->ffu_id);
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK &&
+ act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+ if (act->type == RTE_FLOW_ACTION_TYPE_END)
+ return 0;
+
+ index++;
+ /* Check if the next non-void item is END */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ int ret;
+ struct fm10k_vlan_filter_conf *vlan_filter =
+ &filter->vlan_filter;
+
+ ret = fm10k_flow_parse_vlan_pattern(dev, pattern,
+ error, vlan_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_vlan_action(dev, actions, error, vlan_filter);
+ if (ret)
+ return ret;
+
+ if (attr->ingress)
+ vlan_filter->is_ingress = 1;
+ else if (attr->egress)
+ vlan_filter->is_ingress = 0;
+ vlan_filter->ffu_prio = attr->priority;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+/*
+ *
+ */
+static int
+fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct rte_flow_item *items; /* internal pattern w/o VOID items */
+ parse_filter_t parse_filter;
+ uint32_t item_num = 0; /* non-void item number of pattern*/
+ uint32_t i = 0;
+ bool flag = false;
+ int ret = -1;
+
+ if (!pattern) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "NULL pattern.");
+ return -rte_errno;
+ }
+
+ if (!actions) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_NUM,
+ NULL, "NULL action.");
+ return -rte_errno;
+ }
+
+ if (!attr) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR,
+ NULL, "NULL attribute.");
+ return -rte_errno;
+ }
+
+ /* Get the non-void item number of pattern */
+ while ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_END) {
+ if ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_VOID)
+ item_num++;
+ i++;
+ }
+ item_num++;
+
+ items = rte_zmalloc("fm10k_pattern",
+ item_num * sizeof(struct rte_flow_item), 0);
+ if (!items) {
+ rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "No memory for PMD internal items.");
+ return -ENOMEM;
+ }
+
+ fm10k_pattern_skip_void_item(items, pattern);
+
+ i = 0;
+ do {
+ parse_filter = fm10k_find_parse_filter_func(items, &i);
+ if (!parse_filter && !flag) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ pattern, "Unsupported pattern");
+ rte_free(items);
+ return -rte_errno;
+ }
+ if (parse_filter)
+ ret = parse_filter(dev, attr, items, actions,
+ error, &cons_filter);
+ flag = true;
+ } while ((ret < 0) && (i < RTE_DIM(fm10k_supported_patterns)));
+
+ rte_free(items);
+
+ return ret;
+}
+
+static struct fm10k_cfg_flow *
+fm10k_flow_cfg_transfer(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ int i;
+ u8 port_id;
+ int set_port_num = 0, set_vlan_num = 0;
+ u16 fw_port_id = 0, bp_port_id = 0;
+ u16 filter_vlan_id = 0, fw_vlan_id = 0, bp_vlan_id = 0;
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_cfg_flow *cf;
+
+ cf = rte_zmalloc("fm10k_rule", sizeof(struct fm10k_cfg_flow), 0);
+ if (!cf) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return NULL;
+ }
+ memset(cf, 0, sizeof(struct fm10k_cfg_flow));
+
+ port_id = fm10k_switch_dpdk_port_no_get(hw);
+ for (i = 0; i < 4; i++) {
+ if (pattern[i].type == RTE_FLOW_ITEM_TYPE_VLAN) {
+ filter_vlan_id =
+ rte_be_to_cpu_16
+ (((const struct rte_flow_item_vlan *)
+ pattern[i].spec)->tci);
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_PHY_PORT) {
+ if (set_port_num)
+ bp_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ else
+ fw_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ set_port_num++;
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_END) {
+ break;
+ }
+ }
+
+ for (i = 0; i < 3; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_MARK) {
+ if (set_vlan_num)
+ bp_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ else
+ fw_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ set_vlan_num++;
+ } else if (actions[i].type == RTE_FLOW_ACTION_TYPE_END) {
+ break;
+ }
+ }
+
+ if (attr->ingress && !attr->egress) {
+ /* this port is DPDK port and it is destination port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = fw_port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && attr->egress) {
+ /* this port is DPDK port and it is source port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && !attr->egress) {
+ /* two ports are external port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else {
+ /* two ports are DPDK port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ }
+
+ if (set_port_num == 2 && set_vlan_num == 2) {
+ cf->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[1].port_no = bp_port_id;
+ cf->fw_port[1].vlan_id = bp_vlan_id;
+ }
+
+ return cf;
+}
+
+
+static struct rte_flow *
+fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ struct fm10k_switch *sw = fm10k_switch_get();
+ struct fm10k_cfg_flow *cf;
+ int ret;
+
+ flow = rte_zmalloc("fm10k_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return flow;
+ }
+
+ ret = fm10k_flow_validate(dev, attr, pattern, actions, error);
+ if (ret < 0)
+ return NULL;
+
+ cf = fm10k_flow_cfg_transfer(dev, attr, pattern, actions, error);
+ if (!cf)
+ goto free_flow;
+
+ flow->rule = cf;
+ fm10k_ffu_flow_enable(sw, cf);
+ fm10k_config_flow_list_add_tail(fm10k_config_flowset_current_get(), cf);
+
+ TAILQ_INSERT_TAIL((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw), flow, node);
+ return flow;
+
+free_flow:
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to create flow.");
+ rte_free(flow);
+ return NULL;
+}
+
+static int
+fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret = 0;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (flow->rule) {
+ fm10k_config_flow_list_delete
+ ((struct fm10k_cfg_flow *)flow->rule);
+ fm10k_ffu_flow_disable(sw,
+ (struct fm10k_cfg_flow *)flow->rule);
+ }
+
+ if (!ret) {
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow->rule);
+ rte_free(flow);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to destroy flow.");
+ }
+ return ret;
+}
+
+
+static int
+fm10k_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ void *temp;
+ int ret = 0;
+ struct fm10k_cfg_flow *cf;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ /* Delete flows in flow list. */
+ TAILQ_FOREACH_SAFE(flow,
+ (struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ node, temp) {
+ cf = flow->rule;
+ if (cf) {
+ fm10k_config_flow_list_delete(cf);
+ fm10k_ffu_flow_disable(sw, cf);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "No such rule in flow.");
+ }
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+void
+fm10k_flow_list_init(void *flow_list)
+{
+ TAILQ_INIT((struct fm10k_flow_list *)flow_list);
+}
+
+/* Flow operations */
+const struct rte_flow_ops *
+fm10k_flow_ops_get(void)
+{
+ return &fm10k_flow_ops;
+}
+
+
diff --git a/drivers/net/fm10k/switch/fm10k_flow.h b/drivers/net/fm10k/switch/fm10k_flow.h
new file mode 100644
index 0000000..c538544
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_FLOW_H_
+#define _FM10K_SW_FLOW_H_
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+/*
+ * Struct to store flow created.
+ */
+struct rte_flow {
+ TAILQ_ENTRY(rte_flow) node;
+ enum rte_filter_type filter_type;
+ void *rule;
+};
+
+TAILQ_HEAD(fm10k_flow_list, rte_flow);
+
+#endif /* _FM10K_SW_FLOW_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
index 563f9ac..b90f190 100644
--- a/drivers/net/fm10k/switch/fm10k_stats.c
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -1012,6 +1012,133 @@ struct fm10k_stats_data {
}
}
+void
+fm10k_stats_dpdk_port_print(struct fm10k_switch *sw)
+{
+ int i, j, lport, pf_no;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_PEPS_MAX];
+ static struct fm10k_stats_data last_queue_data[FM10K_SW_PEPS_MAX][4];
+ struct fm10k_stats_data data, mydata;
+ char pf_ports[10];
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].hw == NULL)
+ continue;
+ if (sw->dpdk_cfg->ports[i].type != FM10K_CONFIG_DPDK_PF)
+ continue;
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ memset(&mydata, 0, sizeof(mydata));
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ for (j = 0; j < 2; j++) {
+ pf_no =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[j];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters
+ (sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ }
+ } else if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ } else {
+ continue;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ sprintf(pf_ports, "%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0]);
+ else
+ sprintf(pf_ports, "%d/%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1]);
+
+ FM10K_SW_INFO("DPDK port %-2d pf %-5s tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, pf_ports,
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_QUEUE))
+ continue;
+ memset(&mydata, 0, sizeof(mydata));
+ for (j = 0;
+ j < sw->dpdk_cfg->ports[i].tx_queue_num;
+ j++) {
+ struct fm10k_hw *hw = sw->dpdk_cfg->ports[i].hw;
+ uint16_t queue_id = j, pf_no;
+
+ fm10k_switch_dpdk_hw_queue_map(hw, queue_id,
+ sw->dpdk_cfg->ports[i].tx_queue_num,
+ &hw, &queue_id);
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ data.tx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPTC(queue_id));
+ data.rx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPRC(queue_id));
+ data.rx_drop =
+ FM10K_READ_REG(hw, FM10K_QPRDC(queue_id));
+ mydata.tx_pkts += data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop;
+ FM10K_SW_INFO(" queue %d(%d:%d) tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n", j,
+ pf_no, queue_id,
+ (unsigned long long)(data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts),
+ (unsigned long long)(data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts),
+ (unsigned long long)(data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop));
+ last_queue_data[pf_no][queue_id] = data;
+ }
+ FM10K_SW_INFO(" total tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+ }
+}
static void
fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
@@ -1096,6 +1223,7 @@ struct fm10k_stats_data {
FM10K_CONFIG_DEBUG_STATS_PORT)) {
FM10K_SW_INFO("--- port statistic ---\n");
fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
}
if (fm10k_config_check_debug(sw->dpdk_cfg,
FM10K_CONFIG_DEBUG_STATS_FFU)) {
diff --git a/drivers/net/fm10k/switch/fm10k_switch.c b/drivers/net/fm10k/switch/fm10k_switch.c
new file mode 100644
index 0000000..39a2b3c
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.c
@@ -0,0 +1,2376 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_ethdev.h>
+#include <rte_ethdev_pci.h>
+
+#include <rte_malloc.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_serdes.h"
+#include "fm10k_i2c.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+#include "fm10k_config.h"
+
+
+static struct fm10k_switch fm10k_sw;
+
+#define FM10K_AM_TIMEOUT 16384
+#define FM10K_COMP_PPM_SCALE 1000000
+
+#define FM10K_LED_POLL_INTERVAL_MS 500
+#define FM10K_LED_BLINKS_PER_SECOND 2
+
+
+/*
+ * use epl as external port map, only support QUAD_ON
+ */
+struct fm10k_sched_prog {
+ uint8_t idle;
+ uint8_t phys; /* physical port */
+ uint8_t log; /* logical port */
+ uint8_t quad; /* now, only support QUAD_ON */
+/* convert entry to idle if EPL in quad port mode */
+#define FM10K_SW_QUAD_OFF 0
+/* always use quad port mode */
+#define FM10K_SW_QUAD_ON 1
+/* use quad port mode if link speed is 40/100G */
+#define FM10K_SW_QUAD_40_100 2
+};
+
+static struct fm10k_sched_prog
+ fm10k_sched_prog[FM10K_SW_PEPS_SUPPORTED + FM10K_SW_EPLS_SUPPORTED + 1];
+
+
+/*
+ * Note that for physical port numbers, the initial values used for
+ * the EPL entries assume EPL[A] is EPL[0] and EPL[B] is EPL[1].
+ * fm10k_switch_determine_epls() will update these physical port
+ * numbers based on the actual A and B indices.
+ */
+static void
+fm10k_switch_set_sched_prog(void)
+{
+ int i;
+ int start = 0;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_sched_prog[i].idle = 0;
+ fm10k_sched_prog[i].phys = fm10k_epl_port_map[i].physical_port;
+ fm10k_sched_prog[i].log = fm10k_epl_port_map[i].logical_port;
+ fm10k_sched_prog[i].quad = FM10K_SW_QUAD_ON;
+ }
+ start += FM10K_SW_EPLS_SUPPORTED;
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ fm10k_sched_prog[start + i].idle = 0;
+ fm10k_sched_prog[start + i].phys =
+ fm10k_pep_port_map[i].physical_port;
+ fm10k_sched_prog[start + i].log =
+ fm10k_pep_port_map[i].logical_port;
+ fm10k_sched_prog[start + i].quad = FM10K_SW_QUAD_40_100;
+ }
+ start += FM10K_SW_PEPS_SUPPORTED;
+ fm10k_sched_prog[start].idle = 1;
+}
+
+
+static void
+fm10k_switch_determine_epls(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t phys;
+
+ sw->epla_no = 0;
+ sw->eplb_no = 6;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ sw->epla_no = 1;
+ sw->eplb_no = 6;
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE325G2DSIR):
+ sw->epla_no = 1;
+ sw->eplb_no = 7;
+ break;
+ }
+
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ if (fm10k_sched_prog[i].idle)
+ continue;
+
+ phys = fm10k_sched_prog[i].phys;
+ if (phys <= 3) {
+ /* Substitute actual epla phys port number */
+ fm10k_sched_prog[i].phys = sw->epla_no * 4 + phys;
+ } else if (phys >= 4 && phys <= 7) {
+ /* Substitute actual eplb phys port number */
+ fm10k_sched_prog[i].phys = sw->eplb_no * 4 + (phys - 4);
+ }
+ }
+}
+
+static void
+fm10k_hw_eicr_disable_source(struct fm10k_switch *sw, unsigned int source)
+{
+ unsigned int shift;
+
+ switch (source) {
+ case FM10K_SW_EICR_PCA_FAULT:
+ shift = FM10K_SW_EICR_PCA_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_THI_FAULT:
+ shift = FM10K_SW_EICR_THI_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_FUM_FAULT:
+ shift = FM10K_SW_EICR_FUM_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAILBOX:
+ shift = FM10K_SW_EICR_MAILBOX_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_READY:
+ shift = FM10K_SW_EICR_SWITCH_READY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_NREADY:
+ shift = FM10K_SW_EICR_SWITCH_NREADY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_INT:
+ shift = FM10K_SW_EICR_SWITCH_INT_SHIFT;
+ break;
+ case FM10K_SW_EICR_SRAM_ERROR:
+ shift = FM10K_SW_EICR_SRAM_ERROR_SHIFT;
+ break;
+ case FM10K_SW_EICR_VFLR:
+ shift = FM10K_SW_EICR_VFLR_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAX_HOLD_TIME:
+ shift = FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT;
+ break;
+ default:
+ return;
+ }
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_DISABLE << (shift * 2));
+ fm10k_write_flush(sw);
+}
+
+
+unsigned int
+fm10k_switch_eplidx_to_eplno(struct fm10k_switch *sw, unsigned int eplidx)
+{
+ return eplidx ? sw->eplb_no : sw->epla_no;
+}
+
+
+static uint16_t
+fm10k_switch_ppm_to_tx_clk_compensation_timeout(uint32_t pcs, uint32_t num_ppm)
+{
+ unsigned int scale;
+ unsigned int ppm;
+ unsigned int timeout;
+ unsigned int am_ppm;
+
+ if (num_ppm == 0 || pcs == FM10K_SW_EPL_PCS_SEL_DISABLE) {
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER)
+ return (FM10K_AM_TIMEOUT / 2);
+ else
+ return (0);
+ }
+
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER) {
+ am_ppm = 1000000 / FM10K_AM_TIMEOUT;
+ scale = FM10K_COMP_PPM_SCALE / 2;
+ } else {
+ am_ppm = 0;
+ scale = FM10K_COMP_PPM_SCALE;
+ }
+
+ ppm = num_ppm + am_ppm;
+ timeout = scale / ppm;
+
+ if (timeout >= 0xffff)
+ return 0xffff;
+ else
+ return timeout;
+}
+
+
+static int
+fm10k_switch_configure_epls(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ext_ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_device_info *cfg = sw->info;
+ u32 mac_cfg[FM10K_SW_MAC_CFG_ARRAY_SIZE];
+ u32 data, pcs, qpl;
+ u32 pcstmp;
+ unsigned int i, j;
+ unsigned int dic_enable;
+ unsigned int anti_bubble_wm;
+ unsigned int rate_fifo_wm;
+ unsigned int rate_fifo_slow_inc, rate_fifo_fast_inc;
+ int error;
+ u16 timeout;
+
+ cfg = fm10k_get_device_info(hw);
+ if (cfg == NULL)
+ return -1;
+
+ /*
+ * Assumptions:
+ * - All external interfaces are the same speed
+ * - 1G/10G ports are packed into the minimum number of EPLs
+ * - quad-mode EPLs use lane 0 as master
+ * - The lowest numbered PEPs are used
+ * - PEPs are always used in x8 mode.
+ */
+ switch (cfg->ext_port_speed) {
+ case 10:
+ pcs = FM10K_SW_EPL_PCS_SEL_10GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 1;
+ anti_bubble_wm = 5;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 12;
+ rate_fifo_fast_inc = 13;
+ break;
+ case 25:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 0;
+ anti_bubble_wm = 6;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 32;
+ rate_fifo_fast_inc = 33;
+ break;
+ case 40:
+ pcs = FM10K_SW_EPL_PCS_SEL_40GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 5;
+ rate_fifo_slow_inc = 51;
+ rate_fifo_fast_inc = 52;
+ break;
+ case 100:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 129;
+ rate_fifo_fast_inc = 130;
+ break;
+ default:
+ error = -1;
+ goto done;
+ }
+
+ /*
+ * EPL_CFG_A
+ *
+ * Mark all used lanes as active and all unused lanes as
+ * inactive. Adjust timeout and skew tolerance values for EPLs that
+ * are used.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ data = fm10k_read_switch_reg(sw, FM10K_SW_EPL_CFG_A(i));
+ data &= ~FM10K_SW_EPL_CFG_A_ACTIVE_QUAD;
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < ext_ports->num_ports; j++) {
+ port = &ext_ports->ports[j];
+ if (port->eplno == i)
+ data |= port->is_quad ?
+ FM10K_SW_EPL_CFG_A_ACTIVE_QUAD :
+ FM10K_SW_EPL_CFG_A_ACTIVE
+ (port->first_lane);
+ }
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_TIMEOUT, 19, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_SKEW_TOLERANCE, 38, data);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_A(i), data);
+ }
+
+ /*
+ * EPL_CFG_B
+ *
+ * Disable all unused lanes and configure all used ones
+ * appropriately. For EPLs used in quad port mode, the master lane
+ * is the only one that gets configured.
+ */
+ data = 0;
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++) {
+ if (j < ext_ports->ports_per_epl)
+ pcstmp = pcs;
+ else
+ pcstmp = FM10K_SW_EPL_PCS_SEL_DISABLE;
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j, pcstmp, j, j);
+ }
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE, qpl);
+ } else {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++)
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j,
+ FM10K_SW_EPL_PCS_SEL_DISABLE, j, j);
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE,
+ FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_B(i), data);
+ }
+
+ /*
+ * MAC_CFG, LINK_RULES and PCS_ML_BASER_CFG
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_read_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ /* dic enable */
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_IDLE_ENABLE_DIC,
+ dic_enable, mac_cfg);
+
+ /* ifg */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_IDLE_MIN_IFG_BYTES,
+ 12, mac_cfg);
+
+ /* tx pad size: (64 bytes + 8 preamble) / 4 */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_MIN_COLUMNS,
+ 18, mac_cfg);
+
+ /* tx clock compensation */
+ timeout =
+ fm10k_switch_ppm_to_tx_clk_compensation_timeout(pcs,
+ 100);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE,
+ (timeout > 0), mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT,
+ timeout, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_PREAMBLE_MODE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MIN_FRAME_LENGTH,
+ 64, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MAX_FRAME_LENGTH,
+ FM10K_SW_PACKET_SIZE_MAX, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_RX_IGNORE_IFG_ERRORS,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FCS_MODE,
+ FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_IEEE_1588_ENABLE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_DRAIN_MODE,
+ FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIMEOUT,
+ 100, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIME_SCALE,
+ 3, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIMEOUT,
+ 180, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIME_SCALE,
+ 1, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIMEOUT,
+ 20, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIME_SCALE,
+ 1, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LPI_AUTOMATIC,
+ 1, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LP_IDLE_REQUEST,
+ 0, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FAULT_MODE,
+ FM10K_SW_TX_MAC_FAULT_MODE_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_ANTI_BUBBLE_WATERMARK,
+ anti_bubble_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_WATERMARK,
+ rate_fifo_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_FAST_INC,
+ rate_fifo_fast_inc, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_SLOW_INC,
+ rate_fifo_slow_inc, mac_cfg);
+
+ fm10k_write_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j));
+
+ /* link-up debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_UP,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_UP,
+ 30, data);
+
+ /* link-down debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_DOWN,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_DOWN,
+ 5, data);
+
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_HEARTBEAT_TIME_SCALE,
+ cfg->ext_port_speed == 10 ? 4 : 0, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j), data);
+ }
+
+ if (cfg->ext_port_speed != 10)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PCS_ML_BASER_CFG(i), 0x00003fff);
+ }
+
+
+ /*
+ * LANE_CFG, LANE_SERDES_CFG, LANE_ENERGY_DETECT_CFG,
+ * LANE_SIGNAL_DETECT_CFG, and EPL_FIFO_ERROR_STATUS
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_CFG(i, j), 0);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_CFG(i, j), 0x1106);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j));
+
+ data |=
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY;
+
+ data &=
+ ~FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j), data);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j));
+
+ data &=
+ ~(FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY);
+
+ data |=
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j), data);
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+ }
+
+ error = fm10k_epl_serdes_reset_and_load_all(sw);
+ if (error)
+ goto done;
+
+ /*
+ * EPL_FIFO_ERROR_STATUS LINK_IP
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IP(i, j), FM10K_SW_MASK32(31, 0));
+ }
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+done:
+ return (error);
+}
+
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+
+int
+fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ /* source port is external port number */
+ if (src_port < 0 || src_port == dest_port)
+ return -1;
+
+ return fm10k_ffu_mirror_set(sw, src_port, dest_port, vlan);
+}
+
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (src_port < 0)
+ return -1;
+
+ return fm10k_ffu_mirror_reset(sw, src_port);
+}
+
+
+typedef struct {
+ u8 lport;
+ u8 has_ftag;
+} fm10k_sw_lport;
+
+static int
+fm10k_switch_init(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ u32 data;
+ unsigned int num_lports = sw->info->num_peps + FM10K_SW_EPLS_SUPPORTED;
+ fm10k_sw_lport all_lports[num_lports];
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ unsigned int is_quad;
+ unsigned int table_idx;
+ int error;
+ u32 watermark;
+ u64 data64, data64_2;
+
+ /*
+ * Build list of all logical ports that might appear in the
+ * scheduler program. Note that for any particular card
+ * configuration, not all of these logical ports may be used.
+ */
+ table_idx = 0;
+ for (i = 0; i < sw->info->num_peps; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->pep_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 1;
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->epl_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 0;
+ }
+
+ if (table_idx != num_lports) {
+ FM10K_SW_ERR("fm10k switch lport table construction error");
+ return -1;
+ }
+
+ /*
+ * Reset the switch to get to the default state
+ */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+
+ usec_delay(100);
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ usec_delay(1000);
+
+ /* Clear memories */
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL);
+ fm10k_write_flush(sw);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_EPL);
+ fm10k_write_flush(sw);
+ usec_delay(800);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL, 0);
+ fm10k_write_flush(sw);
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~(FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET);
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ /* ensure switch reset is deasserted for at least 100ns */
+ usec_delay(1);
+
+ sw->epl_sbus = fm10k_sbus_attach(sw, "EPL", FM10K_SW_SBUS_EPL_CFG);
+ if (sw->epl_sbus == NULL) {
+ error = -1;
+ goto done;
+ }
+
+ /* Clear non-BIST accessible pause state memories */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 0), 0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 1), 0);
+ }
+
+ /* Initialize RXQ_MCAST list */
+ for (i = 0; i < FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(i),
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE, i) |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES));
+ }
+ /* Initialize TXQ list */
+ for (i = 0; i < FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_HEAD_PERQ_HEAD, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL0_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL0_PERQ_TAIL, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL1_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL1_PERQ_TAIL, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_TXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES));
+ }
+ /* Initialize free segment list */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_SSCHED_RX_PERPORT(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SSCHED_RX_PERPORT_NEXT, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_FREELIST_INIT_ENTRIES -
+ FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_LOGICAL_PORTS_MAX));
+ }
+ /* Disable switch scan chain */
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCAN_DATA_IN,
+ FM10K_SW_SCAN_DATA_IN_UPDATE_NODES |
+ FM10K_SW_SCAN_DATA_IN_PASSTHRU);
+
+ error = fm10k_switch_configure_epls(hw, sw);
+ if (error)
+ goto done;
+
+ /*
+ * for now configure store-and-forward between PEPs and external
+ * ports regardless of relative speeds
+ */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SAF_MATRIX(all_lports[i].lport),
+ FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS);
+ }
+
+ /* Disable MTU violation trap */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SYS_CFG_1);
+ data &= ~FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS;
+ fm10k_write_switch_reg(sw, FM10K_SW_SYS_CFG_1, data);
+
+ /* Disable ingress VLAN filtering and learning */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_3(all_lports[i].lport), 0);
+ }
+ /*
+ * Make all ports members of every VLAN, and configure every VLAN to
+ * use MST instance 0.
+ */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ fm10k_write_switch_reg128(sw, FM10K_SW_INGRESS_VID_TABLE(i),
+ FM10K_SW_INGRESS_VID_TABLE_REFLECT, data64);
+ }
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_EGRESS_VID_TABLE_ENTRIES; i++)
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_EGRESS_VID_TABLE(i), 0, data64);
+
+ // Init MOD_VLAN_TAG_VID1_MAP
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ data64 = i;
+ data64 = data64 << 48;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * i + 0x20000, data64);
+ }
+
+ /* Configure MST instance 0 to forward for all ports */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_MST_TABLE_FORWARDING
+ (all_lports[i].lport);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_EGRESS_MST_TABLE(0), data64);
+
+ data64 = 0;
+ data64_2 = 0;
+ for (i = 0; i < num_lports; i++) {
+ if (all_lports[i].lport <
+ FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) {
+ data64 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ } else {
+ data64_2 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ }
+ }
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(0, 0), data64);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(1, 0), data64_2);
+
+ for (i = 0; i < num_lports; i++) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(0) |
+ FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport), data64);
+
+ /*
+ * Configure tags for f-tagged lports
+ */
+ if (all_lports[i].has_ftag) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport),
+ data64);
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_MOD_PER_PORT_CFG_2_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport),
+ data64);
+ }
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3;
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport), data64);
+ }
+
+ /*
+ * Assign default SGLORTs for the EPL ports in the PARSER config.
+ * This isn't necessary for the PEPs as for those, as all frames
+ * ingressing from PEPs are tagged with an SGLORT that is derived
+ * from a per-tx-queue setting.
+ *
+ * The PORT_CFG_ISL register offset is determined by logical port
+ * number and the register contents determined by the corresponding
+ * SGLORT.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_ISL
+ (fm10k_sw.epl_map[i].logical_port),
+ FM10K_SW_MAKE_REG_FIELD
+ (PORT_CFG_ISL_SGLORT,
+ fm10k_sw.epl_map[i].glort));
+ }
+
+ /*
+ * FFU counter, 11.10.3.5 POLICER_CFG[0..3]
+ */
+ fm10k_write_switch_reg64(sw,
+ 0xE40000 + 0x2 * FM10K_SW_FFU_CNT_BANK + 0x11000,
+ FM10K_SW_POLICER_LAST);
+
+ /*
+ * 11.10.3.1 POLICER_CFG_4K[0..1][0..4095]
+ * 11.10.3.2 POLICER_CFG_512[0..1][0..511]
+ */
+ for (i = 0; i < FM10K_SW_FFU_CNT_MAX; i++) {
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x2000 * FM10K_SW_FFU_CNT_BANK +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x0);
+ } else {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x400 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x4000);
+ }
+ }
+
+ /*
+ * FFU
+ *
+ * The FFU is programmed to match on SGLORT and to set the DGLORT to
+ * a unique value corresponding to the given SGLORT. One TCAM entry
+ * is required per host port per PEP and one per external interface.
+ * Only slice 0 is used.
+ */
+ if (fm10k_ffu_init(sw, sw->dpdk_cfg) < 0)
+ return -1;
+
+ /*
+ * Program the segment scheduler (see tables at top)
+ *
+ * The TX and RX schedules are the same. Page 0 is used.
+ */
+ if (cfg->ext_port_speed == 40 || cfg->ext_port_speed == 100)
+ is_quad = 1;
+ else
+ is_quad = 0;
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ /*
+ * In addition to explicit idle cycles, non-quad port
+ * entries are converted to idle cycles if the interfaces
+ * are configured in quad-port mode.
+ */
+ if (fm10k_sched_prog[i].idle ||
+ (!fm10k_sched_prog[i].quad && is_quad))
+ data = FM10K_SW_SCHED_SCHEDULE_IDLE;
+ else
+ data = FM10K_SW_SCHED_SCHEDULE_ENTRY
+ (fm10k_sched_prog[i].phys,
+ fm10k_sched_prog[i].log,
+ (fm10k_sched_prog[i].quad == FM10K_SW_QUAD_40_100) ?
+ is_quad : fm10k_sched_prog[i].quad);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RX_SCHEDULE(0, i), data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TX_SCHEDULE(0, i), data);
+ }
+
+ fm10k_write_switch_reg(sw, FM10K_SW_SCHED_SCHEDULE_CTRL,
+ FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_RX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1) |
+ FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_TX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1));
+
+ /* Per 5.7.10.4 */
+ watermark = FM10K_SW_MEM_POOL_SEGS_MAX -
+ ((sw->info->num_peps +
+ FM10K_SW_EPLS_SUPPORTED * FM10K_SW_EPL_LANES) *
+ FM10K_SW_HOWMANY(FM10K_SW_PACKET_SIZE_MAX,
+ FM10K_SW_MEM_POOL_SEG_SIZE, FM10K_SW_MEM_POOL_SEG_SIZE)) -
+ FM10K_SW_MEM_POOL_SEGS_RSVD;
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_WM,
+ FM10K_SW_MAKE_REG_FIELD(CM_GLOBAL_WM_WATERMARK, watermark));
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_CFG,
+ FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN |
+ FM10K_SW_MAKE_REG_FIELD
+ (CM_GLOBAL_CFG_NUM_SWEEPER_PORTS,
+ FM10K_SW_LOGICAL_PORTS_MAX));
+
+ /* Configure stats counters */
+ data = FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS;
+ data64 =
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 |
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8;
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_RX_STATS_CFG(all_lports[i].lport),
+ data |
+ ((all_lports[i].has_ftag) ?
+ FM10K_SW_MAKE_REG_FIELD
+ (RX_STATS_CFG_PER_FRAME_ADJUSTMENT,
+ FM10K_SW_FTAG_SIZE) :
+ 0));
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_STATS_CFG(all_lports[i].lport),
+ data64);
+ }
+
+ /* Transition switch to ready */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+
+ done:
+ return (error);
+}
+
+
+static void
+fm10k_switch_leds_init(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t addr;
+ uint32_t data;
+ int max;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ /* MODE1 - put into sleep mode to ensure it is brought out
+ * of sleep without violating the oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x00);
+
+ /* Wait for oscillator to stabilize after coming out of sleep */
+ fm10k_udelay(500);
+
+ /* MODE2 - group control is blinking, open drain outputs,
+ * OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x01, 0x20);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x02, 0xff);
+
+ /* PWM1 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x03, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x13, max - 1);
+
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ addr = 0x6a;
+ for (i = 0; i < 2; i++) {
+ /* MODE1 - put into sleep mode to ensure it is
+ * brought out of sleep without violating the
+ * oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x00);
+
+ /* MODE2 - group control is blinking, open drain
+ * outputs, OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x20);
+
+ /* Wait for oscillator to stabilize
+ * after coming out of sleep
+ */
+ fm10k_udelay(500);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x02, 0xff);
+
+ /* PWM3 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x05, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, addr, 0x13, max - 1);
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x88);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * PCA9538 initialization
+ * only differences from defaults are noted
+ */
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x65;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0xf0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x66;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ /* set LEC_CFG */
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+
+ /* port from rdifd, LED */
+ fm10k_gpio_output_set(sw, 4, 0);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to operate LEDs for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+
+static unsigned int
+fm10k_switch_pca9635_led_bits(uint8_t led_flags)
+{
+ unsigned int bits;
+
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE)
+ bits = 0x3; /* group blink */
+ else
+ bits = 0x1; /* full on */
+ } else {
+ bits = 0; /* off */
+ }
+ return (bits);
+}
+
+static void
+fm10k_switch_process_leds(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_ports *ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ unsigned int num_ports = ports->num_ports;
+ uint32_t data;
+ uint8_t update_port[num_ports];
+ uint8_t led_flags = 0, read;
+ uint8_t addr;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ if (sw->master_hw->sw_addr == NULL) {
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ return;
+ }
+
+ for (i = 0; i < num_ports; i++) {
+ port = &ports->ports[i];
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_EPL_LED_STATUS(port->eplno));
+ led_flags =
+ ((data & FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP
+ (port->first_lane)) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_UP : 0) |
+ ((data &
+ (FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING
+ (port->first_lane) |
+ FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING
+ (port->first_lane))) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE : 0);
+ update_port[i] = (led_flags != port->last_led_flags);
+ port->last_led_flags = led_flags;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ if (!(update_port[0] || update_port[1])) {
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (ports->ports[0].last_led_flags) |
+ (fm10k_switch_pca9635_led_bits
+ (ports->ports[1].last_led_flags) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs operating
+ * as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (update_port[i] == 0)
+ continue;
+
+ addr = (i == 0) ? 0x6a : 0x69;
+
+ port = &ports->ports[i];
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (port->last_led_flags);
+
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ led_flags = 0;
+ addr = 0x62;
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ case 25:
+ led_flags |= 0x6 << (4 * i); /* 100G */
+ break;
+ case 40:
+ case 10:
+ led_flags |= 0x4 << (4 * i); /* 40G */
+ break;
+ default:
+ led_flags = 0;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ }
+
+ if (update_port[0] || update_port[1]) {
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ if (update_port[0])
+ led_flags |= read & 0xf0;
+ else
+ led_flags |= read & 0xf;
+ fm10k_i2c_write16(sw->i2c, addr, 0x1, led_flags);
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = (i == 0) ? 0x62 : 0x66;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ led_flags = 0x08; /* 100G */
+ break;
+ case 40:
+ led_flags = 0x02; /* 40G */
+ break;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x1, led_flags);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+static void *
+fm10k_switch_leds_update(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ while (sw->detaching == 0) {
+ fm10k_switch_process_leds(ctx);
+ usec_delay(FM10K_LED_POLL_INTERVAL_MS * 1000);
+ }
+ return NULL;
+}
+
+static void
+fm10k_switch_leds_off(struct fm10k_switch *sw, struct fm10k_ext_ports *ports)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ uint8_t led_flags;
+ uint8_t addr;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits(0) |
+ (fm10k_switch_pca9635_led_bits(0) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ addr = 0x6a;
+ led_flags = fm10k_switch_pca9635_led_bits(0);
+ for (i = 0; i < 2; i++) {
+ port = &ports->ports[i];
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+
+static void *
+fm10k_switch_process_intr(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_ext_ports *ports;
+ uint64_t gid;
+ uint64_t reenable_mask;
+ uint16_t epl_mask;
+
+ while (sw->detaching == 0) {
+ sem_wait(&sw->intr_tq);
+ /*
+ * Mask off all global interrupt detect interrupts
+ * toward PCIe to ensure that the PEP sees the
+ * interrupt condition clear so that all subsequent
+ * events will be recognized. The same result could
+ * be achieved by masking off all block-internal
+ * interrupt sources in all blocks prior to handling
+ * any global interrupt detect interrupts, but the
+ * approach taken here makes the interrupt processing
+ * easier to decompose and increases the likelihood that
+ * more work will be coalesced into fewer interrupts.
+ */
+ FM10K_SW_TRACE("masking off PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ /*
+ * sw->ext_ports will be set to NULL during detach
+ * to indicate that we are to no longer process EPL
+ * interrupts.
+ */
+ ports = sw->ext_ports;
+ /*
+ * sw->sm_mailbox_enabled will be set to 0 during
+ * detach to indicate that we are to no longer process
+ * PEP interrupts.
+ */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ FM10K_SW_INTERRUPT_MASK_PCIE_ALL);
+ gid = fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ reenable_mask = 0;
+ if (ports) {
+ epl_mask = fm10k_ext_ports_epl_intrs(ports, gid);
+ reenable_mask |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (INTERRUPT_MASK_PCIE_EPL, epl_mask);
+ }
+
+ if (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C)
+ fm10k_i2c_intr(sw->i2c);
+
+ reenable_mask |= FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+
+ FM10K_SW_TRACE("re-enabling PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ ~reenable_mask);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ }
+ return NULL;
+}
+
+void
+fm10k_switch_intr(struct fm10k_hw *hw)
+{
+ if (hw)
+ sem_post(&fm10k_sw.intr_tq);
+}
+
+
+static void
+fm10k_switch_enable_interrupts(struct fm10k_switch *sw)
+{
+ uint64_t data64;
+ unsigned int i;
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno));
+
+ /* enable interrupts from all EPLs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_EPL,
+ ~sw->ext_ports->epl_mask, data64);
+
+ /* enable interrupts from all non-master PEPs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_PCIE,
+ ~sw->pep_mask, data64);
+
+ /* enable I2C interrupt */
+ data64 &= ~FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno), data64);
+
+ /* enable outbound mailbox interrupts from all non-master PEPs */
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (sw->pep_mask & (1 << i))
+ fm10k_write_switch_reg(sw, FM10K_SW_PCIE_GLOBAL(i, IM),
+ ~FM10K_SW_IM_MAILBOX);
+ }
+ FM10K_WRITE_REG(sw->master_hw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_FIELD(SWITCH_INT, ENABLE));
+}
+
+
+static void
+fm10k_switch_detach(struct fm10k_hw *hw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_switch *sw = &fm10k_sw;
+
+ if (hw == NULL || sw == NULL)
+ return;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->detaching = 1;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (sw->ext_ports) {
+ ports = sw->ext_ports;
+ /*
+ * Ensure that any further switch interrupts will not
+ * process sw->ext_ports before detaching them.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->ext_ports = NULL;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_switch_leds_off(sw, ports);
+ fm10k_ext_ports_detach(ports);
+ }
+
+ if (sw->epl_sbus)
+ fm10k_sbus_detach(sw->epl_sbus);
+
+ /*
+ * Detach i2c after ext_ports so ext_ports can use i2c to disable
+ * phys.
+ */
+ if (sw->i2c)
+ fm10k_i2c_detach(sw->i2c);
+
+ fm10k_hw_eicr_disable_source(sw, FM10K_SW_EICR_SWITCH_INT);
+
+ pthread_join(sw->intr_task, NULL);
+ pthread_join(sw->led_task, NULL);
+ pthread_join(sw->stats_task, NULL);
+}
+
+
+static unsigned int
+fm10k_switch_get_fabric_clock(struct fm10k_switch *sw)
+{
+ uint32_t pll_fabric;
+ unsigned int freq;
+
+ pll_fabric = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_LOCK);
+
+ switch (FM10K_SW_REG_FIELD(pll_fabric, PLL_FABRIC_FREQSEL)) {
+ case FM10K_SW_PLL_FABRIC_FREQSEL_CTRL: {
+ uint32_t ctrl;
+ unsigned int refdiv, fbdiv4, fbdiv255;
+
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_CTRL);
+ refdiv = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_REFDIV);
+ fbdiv4 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV4);
+ fbdiv255 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV255);
+
+ freq = (15625 * 4 * fbdiv255 * (1 + fbdiv4)) / (refdiv * 100);
+ break;
+ }
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F600:
+ freq = 600;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F500:
+ freq = 500;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F400:
+ freq = 400;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F300:
+ freq = 300;
+ break;
+ default:
+ freq = 0;
+ break;
+ }
+
+ return freq;
+}
+
+static unsigned int
+fm10k_switch_get_sku(struct fm10k_switch *sw)
+{
+ uint32_t fuse_data;
+
+ fuse_data = fm10k_read_switch_reg(sw, FM10K_SW_FUSE_DATA_0);
+ return FM10K_SW_REG_FIELD(fuse_data, FUSE_SKU);
+}
+
+static const char *
+fm10k_switch_get_sku_name(unsigned int sku)
+{
+ const char *name;
+
+ switch (sku) {
+ case FM10K_SW_FUSE_SKU_FM10840:
+ name = "FM10840";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10420:
+ name = "FM10420";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10064:
+ name = "FM10064";
+ break;
+ default:
+ name = "FM10xxx-unknown";
+ break;
+ }
+
+ return name;
+}
+
+
+static void
+fm10k_switch_describe(struct fm10k_switch *sw)
+{
+ unsigned int i, pair;
+ unsigned int reset_state, active_state;
+ uint32_t device_cfg, resets;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("switch: device is %s, fabric clock %u MHz",
+ fm10k_switch_get_sku_name(fm10k_switch_get_sku(sw)),
+ fm10k_switch_get_fabric_clock(sw));
+
+ device_cfg = fm10k_read_switch_reg(sw, FM10K_SW_DEVICE_CFG);
+ FM10K_SW_INFO("switch: 100G Ethernet is %s",
+ (device_cfg & FM10K_SW_DEVICE_CFG_PCIE_100G_DIS) ?
+ "disabled" : "enabled");
+ switch (FM10K_SW_REG_FIELD(device_cfg, DEVICE_CFG_FEATURE)) {
+ case FM10K_SW_DEVICE_CFG_PCIE_FULL:
+ FM10K_SW_INFO("switch: PEPs 0-7 support 2x4 and 1x8 "
+ "modes, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_HALF:
+ FM10K_SW_INFO("switch: PEPs 0-3 support 2x4 and 1x8 "
+ "modes, PEPs 4 and 6 are 1x4, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_BASIC:
+ FM10K_SW_INFO("switch: PEP 0 supports, PEP 8 is x1, "
+ "only one can be used");
+ break;
+ }
+
+ resets = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (!(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_EN(i)))
+ continue;
+
+ pair = i / 2;
+ reset_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_RESET(i));
+ active_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_ACTIVE(i));
+ if (pair < FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS &&
+ !(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(pair))) {
+ if (i % 2 == 0)
+ FM10K_SW_INFO("switch: PEP[%u,%u] is enabled in 1x8 "
+ "mode (Reset=%u, Active=%u)", i, i + 1,
+ reset_state, active_state);
+ else
+ FM10K_SW_INFO("switch: PEP[%u] unexpectedly enabled when "
+ "PEP[%u,%u] is in 1x8 mode (Reset=%u, "
+ "Active=%u)", i, i - 1, i, reset_state,
+ active_state);
+ } else {
+ FM10K_SW_INFO("switch: PEP[%u] is enabled in 1x4 "
+ "mode (Reset=%u, Active=%u)", i, reset_state,
+ active_state);
+ }
+ }
+}
+
+
+static struct fm10k_switch *
+fm10k_switch_attach(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ int i;
+ int error = 0;
+
+ /*
+ * apparently no way to determine one's own PEP number.
+ */
+ switch (sw->info->num_peps) {
+ case 1:
+ sw->pepno = 0;
+ break;
+
+ case 2:
+ /*
+ * assumption is using even numbered PEPs
+ * starting with 0, and the highest numbered PEP
+ * is the master
+ */
+ sw->pepno = 2;
+ for (i = 0; i < sw->info->num_peps - 1; i++)
+ sw->pep_mask |= (1 << (i * 2));
+ break;
+
+ case 4:
+ sw->pepno = 6;
+ sw->pep_mask = 0x7f;
+ break;
+
+ default:
+ sw->pepno = 0;
+ }
+
+ /* When not zero, initialize serdes in 'near loopback' mode */
+ sw->serdes_loopback = 0;
+
+ pthread_mutex_init(&sw->lock, NULL);
+
+ fm10k_switch_determine_epls(sw);
+
+ sw->ext_ports = fm10k_ext_ports_attach(sw);
+ if (sw->ext_ports == NULL)
+ goto fail;
+
+ /* label all external ports with their logical port numbers */
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ sw->ext_ports->ports[i].lport = sw->epl_map[i].logical_port;
+
+ /* interrupt */
+ sem_init(&sw->intr_tq, 0, 0);
+ pthread_create(&sw->intr_task, NULL, fm10k_switch_process_intr, sw);
+
+ fm10k_switch_enable_interrupts(sw);
+ fm10k_switch_describe(sw);
+
+ sw->i2c = fm10k_i2c_attach(sw);
+ if (sw->i2c == NULL)
+ goto fail;
+
+ error = fm10k_switch_init(hw, sw);
+ if (error)
+ goto fail;
+
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ fm10k_ext_port_up(&sw->ext_ports->ports[i]);
+
+ usec_delay(10000);
+ fm10k_switch_leds_init(sw);
+
+ pthread_create(&sw->led_task, NULL, &fm10k_switch_leds_update, sw);
+ pthread_create(&sw->stats_task, NULL, &fm10k_switch_process_stats, sw);
+ sw->inited = 1;
+ return sw;
+fail:
+ if (sw != NULL)
+ fm10k_switch_detach(hw);
+ return NULL;
+}
+
+struct fm10k_switch*
+fm10k_switch_get(void)
+{
+ return &fm10k_sw;
+}
+
+static int
+fm10k_switch_dpdk_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].pf_no == pf_no)
+ return i;
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_mapped_ext_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ if (map->map_no[0] == pf_no)
+ return i;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ if (map->map_no[0] == pf_no || map->map_no[1] == pf_no)
+ return i;
+ }
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_start(struct fm10k_switch *sw,
+ struct fm10k_hw *master_hw, eth_fm10k_dev_init_half_func *func)
+{
+ int i, j;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_cfg_port_pf_map *dpdk_map;
+ int dpdk_port_no, ext_port_no = 0;
+ int pf_no;
+
+ sw->info = fm10k_get_device_info(master_hw);
+ sw->master_hw = master_hw;
+ sw->pep_map = fm10k_pep_port_map;
+ sw->epl_map = fm10k_epl_port_map;
+
+ sw->info->ext_port_speed = sw->dpdk_cfg->ext_port_speed;
+ fm10k_switch_set_sched_prog();
+ fm10k_switch_attach(master_hw, &fm10k_sw);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type != FM10K_CONFIG_DPDK_PF ||
+ map->type != FM10K_CONFIG_PORT_MAP_PFS)
+ continue;
+
+ for (j = 0; j < 2; j++) {
+ pf_no = map->map_no[j];
+ dpdk_port_no =
+ fm10k_switch_dpdk_port_get(sw, pf_no);
+ dpdk_map =
+ &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ dpdk_map->type = FM10K_CONFIG_PORT_MAP_PFSS;
+ dpdk_map->map_no[0] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ dpdk_map->map_no[1] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1];
+ }
+ }
+ }
+
+ /* do initialize all ports, after switch is ready */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ uint32_t sglort;
+ struct fm10k_dpdk_port *port = &sw->dpdk_cfg->ports[i];
+ struct fm10k_hw *hw = port->hw;
+
+ if (hw == NULL)
+ break;
+
+ map = &sw->dpdk_cfg->ext_port_map[ext_port_no];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_pf_glort_get(pf_no);
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ ext_port_no =
+ fm10k_switch_mapped_ext_port_get
+ (sw, pf_no);
+ sglort =
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]);
+ } else {
+ FM10K_SW_ERR("Unknown mapped port type %d!",
+ port->type);
+ return -1;
+ }
+ hw->mac.dglort_map = sglort | 0xffff0000;
+ }
+ func(hw);
+ }
+ return 0;
+}
+
+
+static int
+fm10k_switch_dpdk_port_reg(struct fm10k_switch *sw,
+ struct fm10k_hw *hw, void *rte_dev, uint8_t is_pf, bool master)
+{
+ int i;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg == NULL && is_pf) {
+ if (fm10k_config_init(sw, hw) < 0)
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ if (master && port->hw &&
+ port->type == FM10K_CONFIG_DPDK_PF) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (hw->mac.addr[5] - port->hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ if (port->hw == NULL) {
+ port->hw = hw;
+ port->rte_dev = rte_dev;
+ fm10k_flow_list_init(&port->flow_list);
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (is_pf) {
+ sw->dpdk_cfg->pf_bind++;
+ port->type = FM10K_CONFIG_DPDK_PF;
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ } else {
+ port->type = FM10K_CONFIG_DPDK_VF;
+ }
+ if (master)
+ sw->dpdk_cfg->master_hw = hw;
+ if (sw->dpdk_cfg->master_hw) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (sw->dpdk_cfg->master_hw->mac.addr[5] -
+ hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ return 0;
+ }
+ }
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == NULL)
+ break;
+
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == hw)
+ return i;
+ }
+ return -1;
+}
+
+void *
+fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].rte_dev;
+}
+
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].flow_list;
+}
+
+static int
+fm10k_switch_dpdk_cfg_check(struct fm10k_switch *sw)
+{
+ int i;
+ bool need_default = true;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg->master_hw == NULL) {
+ FM10K_SW_ERR("Master PF is not bound!!!");
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF &&
+ map->type != FM10K_CONFIG_PORT_MAP_NULL) {
+ need_default = false;
+ break;
+ }
+ }
+
+ if (need_default) {
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ }
+ }
+
+ return 0;
+}
+
+
+static void
+fm10k_switch_dpdk_cfg_describe(struct fm10k_switch *sw)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K DYNAMIC CONFIG ---\n");
+ FM10K_SW_INFO(" PF Bind : %d\n", sw->dpdk_cfg->pf_bind);
+
+ FM10K_SW_INFO("--- PF ---\n");
+ for (i = 0; i < FM10K_SW_PEP_PORTS_MAX; i++) {
+ uint8_t *mac;
+ struct rte_eth_dev *dev;
+ struct rte_pci_device *pdev;
+
+ if (sw->dpdk_cfg->pf_hw[i] == NULL)
+ continue;
+ dev = (struct rte_eth_dev *)
+ fm10k_switch_dpdk_port_rte_dev_get
+ (sw->dpdk_cfg->pf_hw[i]);
+ pdev = RTE_ETH_DEV_TO_PCI(dev);
+ mac = sw->dpdk_cfg->pf_hw[i]->mac.addr;
+ FM10K_SW_INFO(" PF%d : Logical %d Glort %#x "
+ "PCI Addr %02x:%02x.%x "
+ "MAC Addr %02x:%02x:%02x:%02x:%02x:%02x\n",
+ i, fm10k_pep_port_map[i].logical_port,
+ fm10k_switch_pf_glort_get(i), pdev->addr.bus,
+ pdev->addr.devid,
+ pdev->addr.function,
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+ }
+
+ FM10K_SW_INFO("--- DPDK PORT ---\n");
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].type == FM10K_CONFIG_DPDK_NULL)
+ break;
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ break;
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d\n",
+ i, map->map_no[0]);
+ } else {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d&PF%d Glort %#x\n",
+ i,
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+ }
+
+ FM10K_SW_INFO("--- EXT PORT ---\n");
+ for (i = 0; i < sw->dpdk_cfg->ext_port_num; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ FM10K_SW_INFO(" EXT PORT%d : Logical %d Glort %#x", i,
+ fm10k_epl_port_map[i].logical_port,
+ fm10k_switch_epl_glort_get(i));
+
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ FM10K_SW_INFO("\n");
+ else if (map->type == FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" Map to PF%d\n", map->map_no[0]);
+ else
+ FM10K_SW_INFO(" Map to PF%d&PF%d Glort %#x\n",
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+}
+
+
+int
+fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func)
+{
+ int ret;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (fm10k_switch_dpdk_port_reg(sw, hw, rte_dev, is_pf, master) != 0) {
+ FM10K_SW_ERR("Register ports failed!!!");
+ return -1;
+ }
+
+ /*
+ * After all pfs are started
+ * start switch here
+ */
+ if (fm10k_sw.dpdk_cfg->pf_max != 0 &&
+ fm10k_sw.dpdk_cfg->pf_bind == fm10k_sw.dpdk_cfg->pf_num) {
+ if (fm10k_switch_dpdk_cfg_check(&fm10k_sw) != 0)
+ return -1;
+
+ ret = fm10k_switch_start(&fm10k_sw,
+ fm10k_sw.dpdk_cfg->master_hw, func);
+ fm10k_switch_dpdk_cfg_describe(&fm10k_sw);
+ return ret;
+ }
+ return 0;
+}
+
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw)
+{
+ if (hw)
+ return;
+}
+
+
+/*
+ * for multi-host, only dpdk port 0 and 1 are real rx/tx packets
+ * so we need init/setup/start dpdk port queue for them.
+ * for dpdk port 2/3, we need init/setup except start.
+ * we map the pf queue to 0/1 dpdk queue first, then map
+ * the other pf queue to 2/3 dpdk queue.
+ */
+int
+fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue)
+{
+ int idx, pf_no;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ FM10K_SW_ERR("Not support yet!!!");
+ return -1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[0];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue / 2;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = (max_queue + 1) / 2 + queue / 2;
+ return 0;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_NULL) {
+ FM10K_SW_ERR("Unmapped dpdk port %d!!!", dpdk_port_no);
+ return -1;
+ }
+ FM10K_SW_ERR("Unknown mapped type!!!");
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw, struct fm10k_hw *hw_list[])
+{
+ int pf_no, pf_no_ext;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+ map = &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ hw_list[0] = hw;
+ hw_list[1] = NULL;
+ return 1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ hw_list[1] = NULL;
+ return 1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 2;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFSS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 0;
+ }
+
+ hw_list[0] = NULL;
+ hw_list[1] = NULL;
+ return 0;
+}
+
+void
+fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].tx_queue_num = num;
+}
+
+void
+fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].rx_queue_num = num;
+}
+
+int
+fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ return sw->dpdk_cfg->ports[port_no].pf_no;
+}
+
+
+void
+fm10k_switch_flowset_switchto(const char *name)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_ffu_flowset_switch(sw, name);
+}
+
+void
+fm10k_switch_show_port(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
+}
+
+void
+fm10k_switch_show_ffu(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_ffu_count_print(sw);
+}
+
+void
+fm10k_switch_show_bank(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_port_bank_print(sw);
+}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
` (3 preceding siblings ...)
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
@ 2020-02-20 13:59 ` Xiaojun Liu
2020-02-25 11:28 ` Wang, Xiao W
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-20 13:59 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Split dev init to 2 parts.
First only register the port in switch
management; second init hook will be
called after all the pf are registered
and switch initialization. It will finish
dev init. Also add switch interrupt support.
Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
to support mirror operation. Add fm10k_dev_filter_ctrl
to support flow operation.
Add dpdk port and pf mapping, so
the dpdk port can map to a specific pf
and 1 dpdk port can map to 2 pf to get
total 100G throughput.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/fm10k_ethdev.c | 587 ++++++++++++++++++++++++++++++++++++---
1 file changed, 546 insertions(+), 41 deletions(-)
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 581c690..16b41fd 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -13,6 +13,10 @@
#include "fm10k.h"
#include "base/fm10k_api.h"
+#ifdef ENABLE_FM10K_MANAGEMENT
+#include "switch/fm10k_regs.h"
+#include "switch/fm10k_switch.h"
+#endif
/* Default delay to acquire mailbox lock */
#define FM10K_MBXLOCK_DELAY_US 20
@@ -39,6 +43,10 @@
#define GLORT_PF_MASK 0xFFC0
#define GLORT_FD_MASK GLORT_PF_MASK
#define GLORT_FD_INDEX GLORT_FD_Q_BASE
+#ifdef ENABLE_FM10K_MANAGEMENT
+/* When the switch is ready, the status will be changed */
+static int fm10k_switch_ready;
+#endif
int fm10k_logtype_init;
int fm10k_logtype_driver;
@@ -461,9 +469,6 @@ struct fm10k_xstats_name_off {
PMD_INIT_FUNC_TRACE();
- if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG)
- dev->data->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_RSS_HASH;
-
/* multipe queue mode checking */
ret = fm10k_check_mq_mode(dev);
if (ret != 0) {
@@ -512,6 +517,15 @@ struct fm10k_xstats_name_off {
struct rte_eth_conf *dev_conf = &dev->data->dev_conf;
uint32_t mrqc, *key, i, reta, j;
uint64_t hf;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ uint16_t nb_rx_queues = dev->data->nb_rx_queues;
+ int mapped_num;
+ struct fm10k_hw *mapped_hws[2];
+
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num == 2)
+ nb_rx_queues /= 2;
+#endif
#define RSS_KEY_SIZE 40
static uint8_t rss_intel_key[RSS_KEY_SIZE] = {
@@ -641,27 +655,48 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_tx_init(struct rte_eth_dev *dev)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ uint32_t data;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_tx_queue *txq;
uint64_t base_addr;
uint32_t size;
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Disable TXINT to avoid possible interrupt */
for (i = 0; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_TXINT(i),
3 << FM10K_TXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_tx_queue_num_set(unmap_hw,
+ dev->data->nb_tx_queues);
+#endif
/* Setup TX queue */
for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id);
+#endif
txq = dev->data->tx_queues[i];
base_addr = txq->hw_ring_phys_addr;
size = txq->nb_desc * sizeof(struct fm10k_tx_desc);
/* disable queue to avoid issues while updating state */
- ret = tx_queue_disable(hw, i);
+ ret = tx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Enable use of FTAG bit in TX descriptor, PFVTCTL
@@ -669,7 +704,7 @@ struct fm10k_xstats_name_off {
*/
if (fm10k_check_ftag(dev->device->devargs)) {
if (hw->mac.type == fm10k_mac_pf) {
- FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_PFVTCTL(hw_queue_id),
FM10K_PFVTCTL_FTAG_DESC_ENABLE);
PMD_INIT_LOG(DEBUG, "FTAG mode is enabled");
} else {
@@ -679,15 +714,25 @@ struct fm10k_xstats_name_off {
}
/* set location and size for descriptor ring */
- FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_TDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_TDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_TDLEN(hw_queue_id), size);
/* assign default SGLORT for each TX queue by PF */
+#ifndef ENABLE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf)
- FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(i), hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw,
+ FM10K_TX_SGLORT(hw_queue_id),
+ hw->mac.dglort_map);
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ data = FM10K_SW_MAKE_REG_FIELD
+ (TX_SGLORT_SGLORT, hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(hw_queue_id), data);
+ }
+#endif
}
/* set up vector or scalar TX function as appropriate */
@@ -699,19 +744,27 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_rx_init(struct rte_eth_dev *dev)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct fm10k_macvlan_filter_info *macvlan;
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+ uint32_t logic_port = hw->mac.dglort_map;
+ uint16_t queue_stride = 0;
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_rx_queue *rxq;
uint64_t base_addr;
uint32_t size;
uint32_t rxdctl = FM10K_RXDCTL_WRITE_BACK_MIN_DELAY;
- uint32_t logic_port = hw->mac.dglort_map;
uint16_t buf_size;
- uint16_t queue_stride = 0;
+#ifndef ENABLE_FM10K_MANAGEMENT
/* enable RXINT for interrupt mode */
i = 0;
if (rte_intr_dp_is_en(intr_handle)) {
@@ -731,26 +784,36 @@ struct fm10k_xstats_name_off {
for (; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_RXINT(i),
3 << FM10K_RXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_rx_queue_num_set(unmap_hw, dev->data->nb_rx_queues);
+#endif
/* Setup RX queues */
for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_rx_queues, &hw, &hw_queue_id);
+#endif
rxq = dev->data->rx_queues[i];
base_addr = rxq->hw_ring_phys_addr;
size = rxq->nb_desc * sizeof(union fm10k_rx_desc);
/* disable queue to avoid issues while updating state */
- ret = rx_queue_disable(hw, i);
+ ret = rx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Setup the Base and Length of the Rx Descriptor Ring */
- FM10K_WRITE_REG(hw, FM10K_RDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_RDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_RDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_RDLEN(hw_queue_id), size);
/* Configure the Rx buffer size for one buff without split */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
@@ -764,7 +827,7 @@ struct fm10k_xstats_name_off {
*/
buf_size -= FM10K_RX_DATABUF_ALIGN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id),
(buf_size >> FM10K_SRRCTL_BSIZEPKT_SHIFT) |
FM10K_SRRCTL_LOOPBACK_SUPPRESS);
@@ -774,9 +837,9 @@ struct fm10k_xstats_name_off {
rxq->offloads & DEV_RX_OFFLOAD_SCATTER) {
uint32_t reg;
dev->data->scattered_rx = 1;
- reg = FM10K_READ_REG(hw, FM10K_SRRCTL(i));
+ reg = FM10K_READ_REG(hw, FM10K_SRRCTL(hw_queue_id));
reg |= FM10K_SRRCTL_BUFFER_CHAINING_EN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i), reg);
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id), reg);
}
/* Enable drop on empty, it's RO for VF */
@@ -796,6 +859,7 @@ struct fm10k_xstats_name_off {
/* update RX_SGLORT for loopback suppress*/
if (hw->mac.type != fm10k_mac_pf)
return 0;
+#ifndef ENABLE_FM10K_MANAGEMENT
macvlan = FM10K_DEV_PRIVATE_TO_MACVLAN(dev->data->dev_private);
if (macvlan->nb_queue_pools)
queue_stride = dev->data->nb_rx_queues / macvlan->nb_queue_pools;
@@ -804,6 +868,7 @@ struct fm10k_xstats_name_off {
logic_port++;
FM10K_WRITE_REG(hw, FM10K_RX_SGLORT(i), logic_port);
}
+#endif
return 0;
}
@@ -811,13 +876,31 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret;
+#endif
int err;
uint32_t reg;
struct fm10k_rx_queue *rxq;
+ uint16_t hw_queue_id = rx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ rx_queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id);
+ if (ret < 0)
+ return -EIO;
+ else if (ret != 1) /* reference port's queue don't need start */
+ return 0;
+#endif
+
rxq = dev->data->rx_queues[rx_queue_id];
err = rx_queue_reset(rxq);
if (err == -ENOMEM) {
@@ -836,23 +919,23 @@ struct fm10k_xstats_name_off {
* this comment and the following two register writes when the
* emulation platform is no longer being used.
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
/* Set PF ownership flag for PF devices */
- reg = FM10K_READ_REG(hw, FM10K_RXQCTL(rx_queue_id));
+ reg = FM10K_READ_REG(hw, FM10K_RXQCTL(hw_queue_id));
if (hw->mac.type == fm10k_mac_pf)
reg |= FM10K_RXQCTL_PF;
reg |= FM10K_RXQCTL_ENABLE;
/* enable RX queue */
- FM10K_WRITE_REG(hw, FM10K_RXQCTL(rx_queue_id), reg);
+ FM10K_WRITE_REG(hw, FM10K_RXQCTL(hw_queue_id), reg);
FM10K_WRITE_FLUSH(hw);
/* Setup the HW Rx Head and Tail Descriptor Pointers
* Note: this must be done AFTER the queue is enabled
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -878,22 +961,39 @@ struct fm10k_xstats_name_off {
static int
fm10k_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret;
+#endif
/** @todo - this should be defined in the shared code */
#define FM10K_TXDCTL_WRITE_BACK_MIN_DELAY 0x00010000
uint32_t txdctl = FM10K_TXDCTL_WRITE_BACK_MIN_DELAY;
struct fm10k_tx_queue *q = dev->data->tx_queues[tx_queue_id];
+ uint16_t hw_queue_id = tx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ tx_queue_id, dev->data->nb_tx_queues, &hw, &hw_queue_id);
+ if (ret < 0)
+ return -EIO;
+ else if (ret != 1)
+ return 0;
+#endif
+
q->ops->reset(q);
/* reset head and tail pointers */
- FM10K_WRITE_REG(hw, FM10K_TDH(tx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_TDT(tx_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDT(hw_queue_id), 0);
/* enable TX queue */
- FM10K_WRITE_REG(hw, FM10K_TXDCTL(tx_queue_id),
+ FM10K_WRITE_REG(hw, FM10K_TXDCTL(hw_queue_id),
FM10K_TXDCTL_ENABLE | txdctl);
FM10K_WRITE_FLUSH(hw);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -1084,9 +1184,22 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
int i, diag;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ struct fm10k_hw *mapped_hws[2];
+ int j, mapped_num;
+ uint32_t data;
+#endif
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > 2)
+ return -EIO;
+#endif
+
+
+#ifndef ENABLE_FM10K_MANAGEMENT
/* stop, init, then start the hw */
diag = fm10k_stop_hw(hw);
if (diag != FM10K_SUCCESS) {
@@ -1105,6 +1218,62 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
return -EIO;
}
+#else
+ for (j = 0; j < mapped_num; j++) {
+ struct rte_pci_device *pdev =
+ RTE_ETH_DEV_TO_PCI((struct rte_eth_dev *)
+ (fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[j])));
+ struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+ /* stop, init, then start the hw */
+ diag = fm10k_stop_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware stop failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_init_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware init failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_start_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
+ return -EIO;
+ }
+
+ /* Disable TXINT to avoid possible interrupt */
+ for (i = 0; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_TXINT(i),
+ 3 << FM10K_TXINT_TIMER_SHIFT);
+
+ /* enable RXINT for interrupt mode */
+ i = 0;
+ if (rte_intr_dp_is_en(intr_handle)) {
+ for (; i < dev->data->nb_rx_queues; i++) {
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_RXINT(i), Q2V(pdev, i));
+ if (mapped_hws[j]->mac.type == fm10k_mac_pf)
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_ITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ else
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_VFITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ }
+ }
+
+ /* Disable other RXINT to avoid possible interrupt */
+ for (; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_RXINT(i),
+ 3 << FM10K_RXINT_TIMER_SHIFT);
+ }
+#endif
diag = fm10k_dev_tx_init(dev);
if (diag) {
@@ -1156,12 +1325,29 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
}
}
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Update default vlan when not in VMDQ mode */
if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+#endif
fm10k_link_update(dev, 0);
+#ifdef ENABLE_FM10K_MANAGEMENT
+ /* Admit all VLANs */
+ for (j = 0; j <= 64; j++) {
+ for (i = 0; i < FM10K_SW_VLAN_TABLE_ENTRIES; i++)
+ FM10K_WRITE_REG(hw,
+ FM10K_SW_VLAN_TABLE_ENTRY(j, i),
+ 0xffffffff);
+ }
+
+ /* Disable PEP loopback */
+ data = FM10K_READ_REG(hw, FM10K_CTRL_EXT);
+ data &= ~FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK;
+ FM10K_WRITE_REG(hw, FM10K_CTRL_EXT, data);
+#endif
+
return 0;
}
@@ -1322,17 +1508,41 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
fm10k_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
uint64_t ipackets, opackets, ibytes, obytes, imissed;
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw =
FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_hw *mapped_hws[2];
+ int mapped_num;
+ uint16_t hw_queue_id;
+#endif
struct fm10k_hw_stats *hw_stats =
FM10K_DEV_PRIVATE_TO_STATS(dev->data->dev_private);
int i;
PMD_INIT_FUNC_TRACE();
+#ifndef ENABLE_FM10K_MANAGEMENT
fm10k_update_hw_stats(hw, hw_stats);
+#else
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(unmap_hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > 2)
+ return -EIO;
+
+ for (i = 0; i < mapped_num; i++) {
+ struct rte_eth_dev *mydev =
+ fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[i]);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS(mydev->data->dev_private);
+ fm10k_update_hw_stats(mapped_hws[i], hw_stats);
+ }
+#endif
ipackets = opackets = ibytes = obytes = imissed = 0;
+
+#ifndef ENABLE_FM10K_MANAGEMENT
for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
(i < hw->mac.max_queues); ++i) {
stats->q_ipackets[i] = hw_stats->q[i].rx_packets.count;
@@ -1346,6 +1556,36 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
obytes += stats->q_obytes[i];
imissed += stats->q_errors[i];
}
+#else
+ if (mapped_num) {
+ for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
+ (i < unmap_hw->mac.max_queues); ++i) {
+ hw_queue_id = i;
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, unmap_hw->mac.max_queues,
+ &hw, &hw_queue_id);
+ if (mapped_hws[1]) {
+ struct rte_eth_dev *mydev;
+ mydev = fm10k_switch_dpdk_port_rte_dev_get(hw);
+ hw_stats =
+ FM10K_DEV_PRIVATE_TO_STATS
+ (mydev->data->dev_private);
+ }
+ stats->q_ipackets[i] =
+ hw_stats->q[hw_queue_id].rx_packets.count;
+ stats->q_opackets[i] =
+ hw_stats->q[hw_queue_id].tx_packets.count;
+ stats->q_ibytes[i] =
+ hw_stats->q[hw_queue_id].rx_bytes.count;
+ stats->q_obytes[i] =
+ hw_stats->q[hw_queue_id].tx_bytes.count;
+ ipackets += stats->q_ipackets[i];
+ opackets += stats->q_opackets[i];
+ ibytes += stats->q_ibytes[i];
+ obytes += stats->q_obytes[i];
+ }
+ }
+#endif
stats->ipackets = ipackets;
stats->opackets = opackets;
stats->ibytes = ibytes;
@@ -1808,8 +2048,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_JUMBO_FRAME |
- DEV_RX_OFFLOAD_HEADER_SPLIT |
- DEV_RX_OFFLOAD_RSS_HASH);
+ DEV_RX_OFFLOAD_HEADER_SPLIT);
}
static int
@@ -1817,15 +2056,29 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_rxconf *conf, struct rte_mempool *mp)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
struct fm10k_dev_info *dev_info =
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
struct fm10k_rx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id) < 0)
+ return -EIO;
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads;
/* make sure the mempool element size can account for alignment. */
@@ -1871,7 +2124,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
q->port_id = dev->data->port_id;
q->queue_id = queue_id;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_RDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_RDT(hw_queue_id)];
q->offloads = offloads;
if (handle_rxconf(q, conf))
return -EINVAL;
@@ -2006,13 +2259,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
uint16_t nb_desc, unsigned int socket_id,
const struct rte_eth_txconf *conf)
{
+#ifndef ENABLE_FM10K_MANAGEMENT
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#else
+ struct fm10k_hw *hw;
+ struct fm10k_hw *unmap_hw =
+ FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
struct fm10k_tx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id) < 0)
+ return -EIO;
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.txmode.offloads;
/* make sure a valid number of descriptors have been requested */
@@ -2054,7 +2321,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
q->offloads = offloads;
q->ops = &def_txq_ops;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_TDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_TDT(hw_queue_id)];
if (handle_txconf(q, conf))
return -EINVAL;
@@ -2284,6 +2551,77 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
return 0;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+static int
+fm10k_mirror_rule_set(struct rte_eth_dev *dev,
+ struct rte_eth_mirror_conf *mirror_conf,
+ uint8_t sw_id, uint8_t on)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG,
+ "Mirror set, switch %d to port %d attach vlan %d on %d",
+ sw_id, mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0], on);
+
+ if (on) {
+ if (fm10k_switch_mirror_set(hw,
+ mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0]) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ } else {
+ if (fm10k_switch_mirror_reset(hw) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t sw_id)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG, "Mirror reset, switch %d", sw_id);
+
+ fm10k_switch_mirror_reset(hw);
+
+ return 0;
+}
+
+static int
+fm10k_dev_filter_ctrl(struct rte_eth_dev *dev,
+ enum rte_filter_type filter_type,
+ enum rte_filter_op filter_op,
+ void *arg)
+{
+ int ret = 0;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ switch (filter_type) {
+ case RTE_ETH_FILTER_GENERIC:
+ if (filter_op != RTE_ETH_FILTER_GET)
+ return -EINVAL;
+ *(const void **)arg = fm10k_flow_ops_get();
+ break;
+ default:
+ PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
+ filter_type);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+#endif
+
static void
fm10k_dev_enable_intr_pf(struct rte_eth_dev *dev)
{
@@ -2592,6 +2930,9 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
int status_mbx;
s32 err;
+#ifdef ENABLE_FM10K_MANAGEMENT
+ uint32_t writeback = 0;
+#endif
if (hw->mac.type != fm10k_mac_pf)
return;
@@ -2605,11 +2946,20 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle switch up/down */
- if (cause & FM10K_EICR_SWITCHNOTREADY)
- PMD_INIT_LOG(ERR, "INT: Switch is not ready");
+ if (cause & FM10K_EICR_SWITCHNOTREADY) {
+ PMD_INIT_LOG(INFO, "INT: Switch is not ready");
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_ready = 0;
+ writeback |= FM10K_EICR_SWITCHNOTREADY;
+#endif
+ }
if (cause & FM10K_EICR_SWITCHREADY) {
PMD_INIT_LOG(INFO, "INT: Switch is ready");
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_ready = 1;
+ writeback |= FM10K_EICR_SWITCHREADY;
+#endif
if (dev_info->sm_down == 1) {
fm10k_mbx_lock(hw);
@@ -2660,6 +3010,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle mailbox message */
+#ifndef ENABLE_FM10K_MANAGEMENT
fm10k_mbx_lock(hw);
err = hw->mbx.ops.process(hw, &hw->mbx);
fm10k_mbx_unlock(hw);
@@ -2667,10 +3018,33 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
if (err == FM10K_ERR_RESET_REQUESTED) {
PMD_INIT_LOG(INFO, "INT: Switch is down");
dev_info->sm_down = 1;
- _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
NULL);
}
+#else
+ if (cause & FM10K_EICR_MAILBOX) {
+ fm10k_mbx_lock(hw);
+ err = hw->mbx.ops.process(hw, &hw->mbx);
+ fm10k_mbx_unlock(hw);
+ writeback |= FM10K_EICR_MAILBOX;
+ if (err == FM10K_ERR_RESET_REQUESTED) {
+ PMD_INIT_LOG(INFO, "INT: Switch is down");
+ dev_info->sm_down = 1;
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
+ NULL);
+ }
+ }
+
+ /* Handle switch interrupt */
+ if (cause & FM10K_SW_EICR_SWITCH_INT)
+ fm10k_switch_intr(hw);
+#endif
+
/* Handle SRAM error */
if (cause & FM10K_EICR_SRAMERROR) {
PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
@@ -2682,15 +3056,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
/* Todo: print out error message after shared code updates */
}
+#ifndef ENABLE_FM10K_MANAGEMENT
/* Clear these 3 events if having any */
cause &= FM10K_EICR_SWITCHNOTREADY | FM10K_EICR_MAILBOX |
FM10K_EICR_SWITCHREADY;
if (cause)
FM10K_WRITE_REG(hw, FM10K_EICR, cause);
+#else
+ if (writeback)
+ FM10K_WRITE_REG(hw, FM10K_EICR, writeback);
+#endif
/* Re-enable interrupt from device side */
- FM10K_WRITE_REG(hw, FM10K_ITR(0), FM10K_ITR_AUTOMASK |
+#ifndef ENABLE_FM10K_MANAGEMENT
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_ITR_AUTOMASK |
FM10K_ITR_MASK_CLEAR);
+#else
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_SW_ITR_AUTO_MASK |
+ FM10K_SW_MAKE_REG_FIELD(ITR_MASK, FM10K_SW_ITR_MASK_W_ENABLE));
+#endif
/* Re-enable interrupt from host side */
rte_intr_ack(dev->intr_handle);
}
@@ -2897,6 +3283,11 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
.reta_query = fm10k_reta_query,
.rss_hash_update = fm10k_rss_hash_update,
.rss_hash_conf_get = fm10k_rss_hash_conf_get,
+#ifdef ENABLE_FM10K_MANAGEMENT
+ .mirror_rule_set = fm10k_mirror_rule_set,
+ .mirror_rule_reset = fm10k_mirror_rule_reset,
+ .filter_ctrl = fm10k_dev_filter_ctrl,
+#endif
};
static int ftag_check_handler(__rte_unused const char *key,
@@ -3075,13 +3466,87 @@ static void __attribute__((cold))
info->sm_down = false;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+static int eth_fm10k_dev_init_hook(struct fm10k_hw *hw)
+{
+ int i, switch_ready;
+ struct rte_eth_dev *dev =
+ (struct rte_eth_dev *)fm10k_switch_dpdk_port_rte_dev_get(hw);
+
+ /* Make sure Switch Manager is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ switch_ready = 0;
+
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ fm10k_mbx_lock(hw);
+ switch_ready = fm10k_switch_ready;
+ fm10k_mbx_unlock(hw);
+ if (switch_ready)
+ break;
+ /* Delay some time to acquire async LPORT_MAP info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (switch_ready == 0) {
+ PMD_INIT_LOG(ERR, "switch is not ready");
+ return -1;
+ }
+ }
+
+ /*
+ * Below function will trigger operations on mailbox, acquire lock to
+ * avoid race condition from interrupt handler. Operations on mailbox
+ * FIFO will trigger interrupt to PF/SM, in which interrupt handler
+ * will handle and generate an interrupt to our side. Then, FIFO in
+ * mailbox will be touched.
+ */
+ if (hw->mac.dglort_map == FM10K_DGLORTMAP_NONE) {
+ PMD_INIT_LOG(ERR, "dglort_map is not ready");
+ return -1;
+ }
+
+ fm10k_mbx_lock(hw);
+ /* Enable port first */
+ hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+ MAX_LPORT_NUM, 1);
+ /* Set unicast mode by default. App can change to other mode in other
+ * API func.
+ */
+ hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+ FM10K_XCAST_MODE_NONE);
+ fm10k_mbx_unlock(hw);
+
+ /* Make sure default VID is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (hw->mac.default_vid)
+ break;
+ /* Delay some time to acquire async port VLAN info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (hw->mac.default_vid == 0)
+ hw->mac.default_vid = 1;
+ }
+
+ /* Add default mac address */
+ fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+ MAIN_VSI_POOL_NUMBER);
+
+ return 0;
+}
+#endif
+
static int
eth_fm10k_dev_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
- int diag, i;
+ int diag;
+#ifndef ENABLE_FM10K_MANAGEMENT
+ int i;
+#endif
struct fm10k_macvlan_filter_info *macvlan;
PMD_INIT_FUNC_TRACE();
@@ -3118,7 +3583,9 @@ static void __attribute__((cold))
" Try to blacklist unused devices.");
return -EIO;
}
-
+#ifdef ENABLE_FM10K_MANAGEMENT
+ hw->sw_addr = (void *)pdev->mem_resource[4].addr;
+#endif
/* Store fm10k_adapter pointer */
hw->back = dev->data->dev_private;
@@ -3129,6 +3596,25 @@ static void __attribute__((cold))
return -EIO;
}
+#ifdef ENABLE_FM10K_MANAGEMENT
+ if (hw->mac.type == fm10k_mac_pf) {
+ if (hw->hw_addr == NULL || hw->sw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ } else {
+ if (hw->hw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ }
+
+ /* Store fm10k_adapter pointer */
+ hw->back = dev->data->dev_private;
+#endif
+
/* Initialize parameters */
fm10k_params_init(dev);
@@ -3209,6 +3695,7 @@ static void __attribute__((cold))
hw->mac.ops.update_int_moderator(hw);
/* Make sure Switch Manager is ready before going forward. */
+#ifndef ENABLE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf) {
bool switch_ready = false;
@@ -3216,13 +3703,13 @@ static void __attribute__((cold))
fm10k_mbx_lock(hw);
hw->mac.ops.get_host_state(hw, &switch_ready);
fm10k_mbx_unlock(hw);
- if (switch_ready == true)
+ if (switch_ready)
break;
/* Delay some time to acquire async LPORT_MAP info. */
rte_delay_us(WAIT_SWITCH_MSG_US);
}
- if (switch_ready == false) {
+ if (switch_ready == 0) {
PMD_INIT_LOG(ERR, "switch is not ready");
return -1;
}
@@ -3268,11 +3755,25 @@ static void __attribute__((cold))
MAIN_VSI_POOL_NUMBER);
return 0;
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ bool master = FM10K_READ_REG(hw,
+ FM10K_CTRL) & FM10K_CTRL_BAR4_ALLOWED;
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 1, master, eth_fm10k_dev_init_hook);
+ } else { /* It may not work for VF */
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 0, false, eth_fm10k_dev_init_hook);
+ }
+#endif
}
static int
eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
{
+#ifdef ENABLE_FM10K_MANAGEMENT
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
PMD_INIT_FUNC_TRACE();
/* only uninitialize in the primary process */
@@ -3282,6 +3783,10 @@ static void __attribute__((cold))
/* safe to close dev here */
fm10k_dev_close(dev);
+#ifdef ENABLE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_port_stop(hw);
+#endif
+
return 0;
}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support Xiaojun Liu
@ 2020-02-25 11:28 ` Wang, Xiao W
2020-02-25 12:56 ` Xiaojun Liu
2020-03-05 1:23 ` Xiaojun Liu
0 siblings, 2 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-02-25 11:28 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E
Cc: dev, Ye, Xiaolong
Hi Xiaojun,
Gather some comments about the code changes to original file:
Based on the latest dpdk, "git am" will fail with the Makefile.
Add CONFIG_RTE_FM10K_MANAGEMENT=n as a default option into ./config is better. If user wants to turn on this switch feature, it's easier to just change "n" to "y".
No need to introduce another macro ENABLE_FM10K_MANAGEMENT, just use RTE_FM10K_MANAGEMENT to wrap your code.
For ethdev.c:
static int fm10k_switch_ready; This should be a per device value, right? I assume we may have >1 fm10k devices, then cannot share one state value.
DEV_RX_OFFLOAD_RSS_HASH is removed, it may affect current usage.
Log some debug info before return -EIO.
How about moving the commonly used variables like "struct fm10k_hw *hw" out of "ifdef else".
Put fm10k_link_update() at the end of dev_start().
For fm10k_stats_get(), it looks there's little code shared between #if and #else, you can just write a whole new function.
There's comment "It may not work for VF", have you addressed this uncertainty? We'd better not to leave this in upstream version.
BTW, when sending new versions, you need to document what has changed from last version in your cover letter, in below format:
V2:
* Fix what
* Fix what
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Thursday, February 20, 2020 10:00 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v2 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support
2020-02-25 11:28 ` Wang, Xiao W
@ 2020-02-25 12:56 ` Xiaojun Liu
2020-03-05 1:23 ` Xiaojun Liu
1 sibling, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-25 12:56 UTC (permalink / raw)
To: Wang, Xiao W, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E
Cc: dev, Ye, Xiaolong
Hi Xiao,
Thank you! I will follow your comments.
Best regards,
Xiaojun
________________________________
From: Wang, Xiao W <xiao.w.wang@intel.com>
Sent: Tuesday, February 25, 2020 7:28 PM
To: Xiaojun Liu <xiaojun.liu@silicom.co.il>; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>
Cc: dev@dpdk.org <dev@dpdk.org>; Ye, Xiaolong <xiaolong.ye@intel.com>
Subject: RE: [PATCH v2 5/5] net/fm10k: add switch management support
Hi Xiaojun,
Gather some comments about the code changes to original file:
Based on the latest dpdk, "git am" will fail with the Makefile.
Add CONFIG_RTE_FM10K_MANAGEMENT=n as a default option into ./config is better. If user wants to turn on this switch feature, it's easier to just change "n" to "y".
No need to introduce another macro ENABLE_FM10K_MANAGEMENT, just use RTE_FM10K_MANAGEMENT to wrap your code.
For ethdev.c:
static int fm10k_switch_ready; This should be a per device value, right? I assume we may have >1 fm10k devices, then cannot share one state value.
DEV_RX_OFFLOAD_RSS_HASH is removed, it may affect current usage.
Log some debug info before return -EIO.
How about moving the commonly used variables like "struct fm10k_hw *hw" out of "ifdef else".
Put fm10k_link_update() at the end of dev_start().
For fm10k_stats_get(), it looks there's little code shared between #if and #else, you can just write a whole new function.
There's comment "It may not work for VF", have you addressed this uncertainty? We'd better not to leave this in upstream version.
BTW, when sending new versions, you need to document what has changed from last version in your cover letter, in below format:
V2:
* Fix what
* Fix what
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Thursday, February 20, 2020 10:00 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v2 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 0/5] support fm10k switch management
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for " Xiaojun Liu
` (4 more replies)
0 siblings, 5 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
V1:
* Add I2C to control the inside LED and PHY.
* Add SBUS to communicate with spico.
* Add registers defination.
* Add switch management log API.
* Add switch management structures.
* Add epl serdes include loading spico,
controlling pcsl, dma, dfe, ical.
* Add spico code.
* Add state machine for epl lane and port.
* Add external port management.
* Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
* Add config file to configure debug log, port speed,
epl port mapping dpdk port, flowset.
* Add statistics includes epl port, ffu rule, dpdk port.
* Add flow interface to support offload flow into HW.
* Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
* Split dev_init to 2 parts in ethdev. First only register
the port in switch management; second init hook will be
called after all the pf are registered.
* Add switch interrupt support.
* Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset.
* Add fm10k_dev_filter_ctrl to support flow operation.
* Add dpdk port and pf mapping.
* Modify Makefile to add new files building
Xiaojun Liu (5):
net/fm10k: add basic functions for switch management
net/fm10k: add epl serdes and port control functions
net/fm10k: add ffu and statistics and config file functions
net/fm10k: add flow interface and switch management
net/fm10k: add switch management support
config/common_linux | 7 +
drivers/net/fm10k/Makefile | 23 +
drivers/net/fm10k/base/fm10k_type.h | 1 +
drivers/net/fm10k/fm10k_ethdev.c | 562 ++++-
drivers/net/fm10k/switch/fm10k_config.c | 863 ++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 ++
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 840 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_ffu.c | 1253 +++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 +++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 81 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
drivers/net/fm10k/switch/fm10k_stats.c | 1242 +++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
drivers/net/fm10k/switch/fm10k_switch.c | 2394 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_switch.h | 476 +++++
28 files changed, 17362 insertions(+), 42 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for switch management
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-03-16 6:37 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
` (3 subsequent siblings)
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add I2C to control the inside LED and PHY.
All the operations of I2C are using fm10k I2C register.
Add SBUS to communicate with spico(micro code in serdes)
by using fm10k SBUS register. This is like I2C operations.
Add registers defination, which include all the registers
will be used in the driver. Add switch management log API.
Add switch management structures. Modify Makefile to add
new files building. Add CONFIG_RTE_FM10K_MANAGEMENT=n
in config/common_linux.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
config/common_linux | 7 +
drivers/net/fm10k/Makefile | 14 +
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++++
drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 292 ++++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
9 files changed, 3373 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
diff --git a/config/common_linux b/config/common_linux
index 8168106..75a4fa8 100644
--- a/config/common_linux
+++ b/config/common_linux
@@ -66,3 +66,10 @@ CONFIG_RTE_LIBRTE_HINIC_PMD=y
# Hisilicon HNS3 PMD driver
#
CONFIG_RTE_LIBRTE_HNS3_PMD=y
+
+#
+# FM10K switch management
+#
+CONFIG_RTE_FM10K_MANAGEMENT=n
+
+
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 29e659d..15ea187 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -11,6 +11,9 @@ LIB = librte_pmd_fm10k.a
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
CFLAGS += -DALLOW_EXPERIMENTAL_API
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+CFLAGS += -DENABLE_FM10K_MANAGEMENT
+endif
EXPORT_MAP := rte_pmd_fm10k_version.map
@@ -49,6 +52,9 @@ endif
LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash
LDLIBS += -lrte_bus_pci
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+LDLIBS += -lpthread
+endif
#
# Add extra flags for base driver source files to disable warnings in them
@@ -58,6 +64,10 @@ $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER))
VPATH += $(SRCDIR)/base
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+VPATH += $(SRCDIR)/switch
+endif
+
#
# all source are stored in SRCS-y
# base driver is based on the package of cid-fm10k.2017.01.24.tar.gz
@@ -71,6 +81,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_common.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_mbx.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
+ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
endif
diff --git a/drivers/net/fm10k/switch/fm10k_debug.h b/drivers/net/fm10k/switch/fm10k_debug.h
new file mode 100644
index 0000000..f7b5c06
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_debug.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_DEBUG_H_
+#define _FM10K_DEBUG_H_
+
+
+#define FM10K_SW_ERR(...) PMD_INIT_LOG(ERR, __VA_ARGS__)
+#define FM10K_SW_INFO(...) PMD_INIT_LOG(INFO, __VA_ARGS__)
+#define FM10K_SW_TRACE(...) PMD_INIT_LOG(DEBUG, __VA_ARGS__)
+
+#define FM10K_SW_ASSERT(...) do {} while (0)
+
+#define FM10K_SW_STATS_TRACE_ENABLE 1
+#define FM10K_SW_FFU_CONF_TRACE_ENABLE 0
+#define FM10K_SW_MIRROR_TRACE_ENABLE 0
+
+#endif /* _FM10K_DEBUG_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.c b/drivers/net/fm10k/switch/fm10k_i2c.c
new file mode 100644
index 0000000..28b0c34
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.c
@@ -0,0 +1,310 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+
+static void fm10k_i2c_init(struct fm10k_i2c *);
+
+struct fm10k_i2c *
+fm10k_i2c_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_i2c *i2c;
+
+ FM10K_SW_TRACE("i2c: attaching");
+
+ i2c = (struct fm10k_i2c *)rte_zmalloc("fm10k_i2c",
+ sizeof(struct fm10k_i2c), 0);
+ if (i2c == NULL) {
+ FM10K_SW_INFO("i2c: failed to allocate context");
+ goto fail;
+ }
+
+ i2c->sw = sw;
+ pthread_mutex_init(&i2c->req_lock, NULL);
+ pthread_mutex_init(&i2c->bus_lock, NULL);
+ sem_init(&i2c->req_cv, 0, 0);
+
+ fm10k_i2c_init(i2c);
+
+ FM10K_SW_TRACE("i2c: attach successful");
+ return i2c;
+fail:
+ if (i2c)
+ fm10k_i2c_detach(i2c);
+ return NULL;
+}
+
+void
+fm10k_i2c_detach(struct fm10k_i2c *i2c)
+{
+ FM10K_SW_TRACE("i2c: detaching");
+
+ rte_free(i2c);
+}
+
+static void
+fm10k_i2c_init(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_device_info *cfg = sw->info;
+ uint32_t freq = FM10K_SW_I2C_CFG_DIVIDER_400_KHZ;
+ uint32_t data;
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ freq = FM10K_SW_I2C_CFG_DIVIDER_100_KHZ;
+
+ /* clear any pending interrupt */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ /* 400 KHz, master mode, unmask interrupt */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CFG);
+ data &= ~FM10K_SW_I2C_CFG_SLAVE_ENABLE;
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ FM10K_SW_REPLACE_REG_FIELD(data, I2C_CFG_DIVIDER, freq, data);
+ data &= ~FM10K_SW_I2C_CFG_INTERRUPT_MASK;
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CFG, data);
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ /* reset I2C */
+ fm10k_gpio_output_set(sw, 5, 1);
+}
+
+unsigned int
+fm10k_i2c_intr(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_i2c_req *req;
+ int i;
+ uint32_t data[3];
+ uint32_t ctrl;
+
+ req = i2c->cur_req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CTRL);
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ req->status = FM10K_SW_REG_FIELD(ctrl, I2C_CTRL_COMMAND_COMPLETED);
+
+ if ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->status == FM10K_SW_I2C_COMPLETION_NORMAL) {
+ for (i = 0; i < FM10K_SW_HOWMANY(req->read_len, 4, 4); i++)
+ data[i] = fm10k_read_switch_reg
+ (sw, FM10K_SW_I2C_DATA(i));
+
+ for (i = 0; i < req->read_len; i++)
+ req->msg[i] =
+ (data[i / 4] >> (24 - (i % 4) * 8)) & 0xff;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ sem_post(&i2c->req_cv);
+
+ return 1;
+}
+
+int
+fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ int i;
+ uint32_t ctrl;
+ uint32_t data[3];
+
+ if (((req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->write_len > FM10K_SW_I2C_MSG_MAX) ||
+ ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ ((req->read_len == 0 ||
+ req->read_len > FM10K_SW_I2C_MSG_MAX))))
+ return (-1);
+
+ FM10K_SW_TRACE("i2c: initiating command %u", req->cmd);
+
+ ctrl =
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_ADDR, req->addr << 1) |
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_COMMAND, req->cmd);
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) {
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_W, req->write_len);
+
+ data[0] = 0;
+ data[1] = 0;
+ data[2] = 0;
+
+ for (i = 0; i < req->write_len; i++)
+ data[i / 4] |= req->msg[i] << (24 - (i % 4) * 8);
+
+ for (i = 0; i < FM10K_SW_HOWMANY(req->write_len, 4, 4); i++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_I2C_DATA(i), data[i]);
+ }
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD)
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_R, req->read_len);
+
+ req->status = FM10K_SW_I2C_COMPLETION_RUNNING;
+ i2c->cur_req = req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* zero command field */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, 0);
+ /* initiate command */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, ctrl);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ while (req->status == FM10K_SW_I2C_COMPLETION_RUNNING)
+ sem_wait(&i2c->req_cv);
+
+ return 0;
+}
+
+int
+fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_RD;
+ req.read_len = 1;
+ req.msg[0] = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR_RD;
+ req.write_len = 1;
+ req.read_len = 1;
+ req.msg[0] = reg;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 1;
+ req.msg[0] = data;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 2;
+ req.msg[0] = data0;
+ req.msg[1] = data1;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.h b/drivers/net/fm10k/switch/fm10k_i2c.h
new file mode 100644
index 0000000..f835afe
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_I2C_H_
+#define _FM10K_SW_I2C_H_
+
+#include <semaphore.h>
+#include <pthread.h>
+#include "rte_spinlock.h"
+#include "fm10k_debug.h"
+
+#define FM10K_SW_I2C_MSG_MAX 12
+
+struct fm10k_i2c_req {
+ uint8_t addr; /* 7-bit address */
+ uint8_t cmd; /* FM10K_SW_I2C_COMMAND_* */
+ uint8_t write_len;
+ uint8_t read_len;
+ uint8_t status; /* FM10K_SW_I2C_COMPLETION_ */
+ uint8_t msg[FM10K_SW_I2C_MSG_MAX];
+};
+
+struct fm10k_i2c {
+ struct fm10k_switch *sw;
+ pthread_mutex_t bus_lock;
+ pthread_mutex_t req_lock;
+ sem_t req_cv;
+ struct fm10k_i2c_req *cur_req;
+};
+
+#define FM10K_SW_I2C_LOCK(i2c_) \
+ pthread_mutex_lock(&(i2c_)->bus_lock)
+#define FM10K_SW_I2C_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&(i2c_)->bus_lock)
+
+#define FM10K_SW_I2C_REQ_LOCK(i2c_) \
+ pthread_mutex_lock(&((i2c_)->req_lock))
+#define FM10K_SW_I2C_REQ_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&((i2c_)->req_lock))
+
+struct fm10k_i2c *fm10k_i2c_attach(struct fm10k_switch *sw);
+void fm10k_i2c_detach(struct fm10k_i2c *i2c);
+unsigned int fm10k_i2c_intr(struct fm10k_i2c *i2c);
+int fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req);
+int fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result);
+int fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result);
+int fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data);
+int fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1);
+int fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr);
+
+#endif /* _FM10K_SW_I2C_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_regs.h b/drivers/net/fm10k/switch/fm10k_regs.h
new file mode 100644
index 0000000..2663773
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_regs.h
@@ -0,0 +1,2302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_REGS_H_
+#define _FM10K_REGS_H_
+
+#include <stdint.h>
+
+
+/* Convert a 32-bit word offset into a byte offset */
+#define FM10K_SW_REG_OFF(wo_) (wo_)
+#define FM10K_SW_MASK32(max_bit_, min_bit_) \
+ ((0xffffffffU << (min_bit_)) & \
+ (0xffffffffU >> (31 - (max_bit_))))
+#define FM10K_SW_REG_FIELD(r_, name_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) >> FM10K_SW_##name_##_lsb)
+#define FM10K_SW_REG_FIELD_IDX(r_, name_, i_, i1_, i2_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i_), \
+ FM10K_SW_##name_##_lsb(i2_))) >> \
+ FM10K_SW_##name_##_lsb(i3_))
+#define FM10K_SW_MAKE_REG_FIELD(name_, v_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb))
+#define FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_, i1_, i2_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb(i_)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) | \
+ FM10K_SW_MAKE_REG_FIELD(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_)))
+#define FM10K_SW_MASK64(max_bit_, min_bit_) \
+ ((0xffffffffffffffffULL << (min_bit_)) & \
+ (0xffffffffffffffffULL >> (63 - (max_bit_))))
+#define FM10K_SW_REG_FIELD64(r_, name_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) >> \
+ FM10K_SW_##name_##_lsb64)
+#define FM10K_SW_REG_FIELD_IDX64(r_, name_, i_, i1_, i2_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i_), \
+ FM10K_SW_##name_##_lsb64(i1_))) >> \
+ FM10K_SW_##name_##_lsb64(i2_))
+#define FM10K_SW_MAKE_REG_FIELD64(name_, v_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64))
+#define FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_, i1_, i2_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64(i_)) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD64(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) | \
+ FM10K_SW_MAKE_REG_FIELD64(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX64(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_)))
+
+/* These operate on arrays of 32-bit words */
+#define FM10K_SW_SET_ARRAY_BIT(a_, name_, b_, a1_) \
+ ((a_)[FM10K_SW_##name_##_bit / 32] = \
+ ((a1_)[FM10K_SW_##name_##_bit / 32] & \
+ ~(1 << (FM10K_SW_##name_##_bit % 32))) | \
+ (((b_) & 0x1) << (FM10K_SW_##name_##_bit % 32)))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_MAKE_ARRAY_FIELD(name_, v_) \
+ (((uint32_t)(v_) << (FM10K_SW_##name_##_lsb % 32)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_REPLACE_ARRAY_FIELD(a_, name_, v_, a1_) \
+ ((a_)[FM10K_SW_##name_##_lsb / 32] = \
+ (((a1_)[FM10K_SW_##name_##_lsb / 32] & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32)) | \
+ FM10K_SW_MAKE_ARRAY_FIELD(name_, v_)))
+
+
+/*
+ * BAR0 registers
+ */
+/* Interrupt throttle timer selection values */
+#define FM10K_SW_INT_TIMER_0 0
+#define FM10K_SW_INT_TIMER_1 1
+#define FM10K_SW_INT_TIMER_IMMEDIATE 2
+#define FM10K_SW_INT_TIMER_DISABLED 3
+
+#define FM10K_SW_CTRL FM10K_SW_REG_OFF(0x0)
+#define FM10K_SW_CTRL_BAR4_ALLOWED (1 << 2)
+#define FM10K_SW_CTRL_EXT FM10K_SW_REG_OFF(0x1)
+#define FM10K_SW_CTRL_EXT_NS_DIS (1 << 0)
+#define FM10K_SW_CTRL_EXT_RO_DIS (1 << 1)
+#define FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK (1 << 2)
+#define FM10K_SW_EXVET FM10K_SW_REG_OFF(0x2)
+#define FM10K_SW_GCR FM10K_SW_REG_OFF(0x3)
+#define FM10K_SW_FACTPS FM10K_SW_REG_OFF(0x4)
+#define FM10K_SW_GCR_EXT FM10K_SW_REG_OFF(0x5)
+#define FM10K_SW_EICR FM10K_SW_REG_OFF(0x6)
+#define FM10K_SW_EICR_PCA_FAULT_SHIFT 0
+#define FM10K_SW_EICR_PCA_FAULT \
+ (1 << FM10K_SW_EICR_PCA_FAULT_SHIFT)
+#define FM10K_SW_EICR_THI_FAULT_SHIFT 2
+#define FM10K_SW_EICR_THI_FAULT \
+ (1 << FM10K_SW_EICR_THI_FAULT_SHIFT)
+#define FM10K_SW_EICR_FUM_FAULT_SHIFT 5
+#define FM10K_SW_EICR_FUM_FAULT \
+ (1 << FM10K_SW_EICR_FUM_FAULT_SHIFT)
+#define FM10K_SW_EICR_MAILBOX_SHIFT 6
+#define FM10K_SW_EICR_MAILBOX \
+ (1 << FM10K_SW_EICR_MAILBOX_SHIFT)
+#define FM10K_SW_EICR_SWITCH_READY_SHIFT 7
+#define FM10K_SW_EICR_SWITCH_READY \
+ (1 << FM10K_SW_EICR_SWITCH_READY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_NREADY_SHIFT 8
+#define FM10K_SW_EICR_SWITCH_NREADY \
+ (1 << FM10K_SW_EICR_SWITCH_NREADY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_INT_SHIFT 9
+#define FM10K_SW_EICR_SWITCH_INT \
+ (1 << FM10K_SW_EICR_SWITCH_INT_SHIFT)
+#define FM10K_SW_EICR_SRAM_ERROR_SHIFT 10
+#define FM10K_SW_EICR_SRAM_ERROR \
+ (1 << FM10K_SW_EICR_SRAM_ERROR_SHIFT)
+#define FM10K_SW_EICR_VFLR_SHIFT 11
+#define FM10K_SW_EICR_VFLR \
+ (1 << FM10K_SW_EICR_VFLR_SHIFT)
+#define FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT 12
+#define FM10K_SW_EICR_MAX_HOLD_TIME \
+ (1 << FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT)
+#define FM10K_SW_EIMR FM10K_SW_REG_OFF(0x7)
+#define FM10K_SW_EIMR_DISABLE_ALL 0x55555555
+#define FM10K_SW_EIMR_NO_CHANGE 0x0
+#define FM10K_SW_EIMR_DISABLE 0x1
+#define FM10K_SW_EIMR_ENABLE 0x2
+#define FM10K_SW_EIMR_FIELD(i_, v_) \
+ (FM10K_SW_EIMR_##v_ << (FM10K_SW_EICR_##i_##_SHIFT * 2))
+#define FM10K_SW_EIMR_ENABLED(e_, i_) \
+ ((((e_) >> (FM10K_SW_EICR_##i_##_SHIFT * 2)) & 0x3) == \
+ FM10K_SW_EIMR_ENABLE)
+#define FM10K_SW_PCA_FAULT FM10K_SW_REG_OFF(0x8)
+#define FM10K_SW_THI_FAULT FM10K_SW_REG_OFF(0x10)
+#define FM10K_SW_FUM_FAULT FM10K_SW_REG_OFF(0x1C)
+#define FM10K_SW_MAXHOLDQ(n_) FM10K_SW_REG_OFF(0x20 + (n_))
+#define FM10K_SW_MAXHOLDQ_ENTRIES 8
+#define FM10K_SW_SM_AREA FM10K_SW_REG_OFF(0x28)
+#define FM10K_SW_DGLORTMAP(n_) FM10K_SW_REG_OFF(0x30 + (n_))
+#define FM10K_SW_DGLORTMAP_ENTRIES 8
+#define FM10K_SW_DGLORTMAP_MATCH_ANY 0x00000000
+#define FM10K_SW_DGLORTMAP_MATCH_NONE 0x0000ffff
+#define FM10K_SW_DGLORTMAP_VALUE_lsb 0
+#define FM10K_SW_DGLORTMAP_VALUE_msb 15
+#define FM10K_SW_DGLORTMAP_MASK_lsb 16
+#define FM10K_SW_DGLORTMAP_MASK_msb 31
+#define FM10K_SW_DGLORTDEC(n_) FM10K_SW_REG_OFF(0x38 + (n_))
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_lsb 0
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_msb 3
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_lsb 4
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_msb 6
+#define FM10K_SW_DGLORTDEC_VSI_BASE_lsb 7
+#define FM10K_SW_DGLORTDEC_VSI_BASE_msb 13
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_lsb 14
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_msb 15
+#define FM10K_SW_DGLORTDEC_Q_BASE_lsb 16
+#define FM10K_SW_DGLORTDEC_Q_BASE_msb 23
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_MAX 7
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_lsb 24
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_msb 26
+#define FM10K_SW_DGLORTDEC_INNTER_RSS (1 << 27)
+#define FM10K_SW_TUNNEL_CFG FM10K_SW_REG_OFF(0x40)
+#define FM10K_SW_SWPRI_MAP(pri_) FM10K_SW_REG_OFF(0x50 + (pri_))
+#define FM10K_SW_RSSRK(f_, n_) \
+ FM10K_SW_REG_OFF(0x800 + 0x10 * (f_) + (n_))
+#define FM10K_SW_RSSRK_ENTRIES 10
+#define FM10K_SW_RETA(f_, n_) \
+ FM10K_SW_REG_OFF(0x1000 + 0x20 * (f_) + (n_))
+#define FM10K_SW_RETA_ENTRIES 32
+#define FM10K_SW_RETA_ENTRY(e0_, e1_, e2_, e3_) \
+ (((e0_) & 0xff) | (((e1_) & 0xff) << 8) | \
+ (((e2_) & 0xff) << 16) | (((e3_) & 0xff) << 24))
+#define FM10K_SW_TC_CREDIT(g_) FM10K_SW_REG_OFF(0x2000 + (g_))
+#define FM10K_SW_TC_MAXCREDIT(g_) FM10K_SW_REG_OFF(0x2040 + (g_))
+#define FM10K_SW_TC_RATE(g_) FM10K_SW_REG_OFF(0x2080 + (g_))
+#define FM10K_SW_TC_RATE_STATUS FM10K_SW_REG_OFF(0x20C0)
+#define FM10K_SW_PAUSE FM10K_SW_REG_OFF(0x20C2)
+#define FM10K_SW_DMA_CTRL FM10K_SW_REG_OFF(0x20C3)
+#define FM10K_SW_DMA_CTRL_TX_ENABLE (1 << 0)
+#define FM10K_SW_DMA_CTRL_TX_HOST_PENDING (1 << 1)
+#define FM10K_SW_DMA_CTRL_TX_DATA (1 << 2)
+#define FM10K_SW_DMA_CTRL_TX_ACTIVE (1 << 3)
+#define FM10K_SW_DMA_CTRL_RX_ENABLE (1 << 4)
+#define FM10K_SW_DMA_CTRL_RX_HOST_PENDING (1 << 5)
+#define FM10K_SW_DMA_CTRL_RX_DATA (1 << 6)
+#define FM10K_SW_DMA_CTRL_RX_ACTIVE (1 << 7)
+#define FM10K_SW_DMA_CTRL_RX_DESC_SIZE_32 (1 << 8)
+#define FM10K_SW_DMA_CTRL_MIN_MSS_lsb 9
+#define FM10K_SW_DMA_CTRL_MIN_MSS_msb 22
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_lsb 23
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_msb 27
+#define FM10K_SW_DMA_CTRL_DATA_PATH_RESET (1 << 29)
+#define FM10K_SW_DMA_CTRL_MAX_QS_lsb 30
+#define FM10K_SW_DMA_CTRL_MAX_QS_msb 31
+#define FM10K_SW_DMA_CTRL_MAX_QS_256 0
+#define FM10K_SW_DMA_CTRL_MAX_QS_128 1
+#define FM10K_SW_DMA_CTRL_MAX_QS_64 2
+#define FM10K_SW_DMA_CTRL2 FM10K_SW_REG_OFF(0x20C4)
+#define FM10K_SW_DTXTCPFLGL FM10K_SW_REG_OFF(0x20C5)
+#define FM10K_SW_DTXTCPFLGH FM10K_SW_REG_OFF(0x20C6)
+#define FM10K_SW_TPH_CTRL FM10K_SW_REG_OFF(0x20C7)
+#define FM10K_SW_MRQC(f_) \
+ FM10K_SW_REG_OFF(0x2100 + (f_))
+#define FM10K_SW_MRQC_TCPIPV4 (1 << 0)
+#define FM10K_SW_MRQC_IPV4 (1 << 1)
+#define FM10K_SW_MRQC_IPV6 (1 << 4)
+#define FM10K_SW_MRQC_TCPIPV6 (1 << 5)
+#define FM10K_SW_MRQC_UDPIPV4 (1 << 6)
+#define FM10K_SW_MRQC_UDPIPV6 (1 << 7)
+#define FM10K_SW_TQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x2800 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_RQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x3000 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_STATS_TIMEOUT FM10K_SW_REG_OFF(0x3800)
+#define FM10K_SW_STATS_UR FM10K_SW_REG_OFF(0x3801)
+#define FM10K_SW_STATS_CA FM10K_SW_REG_OFF(0x3802)
+#define FM10K_SW_STATS_UM FM10K_SW_REG_OFF(0x3803)
+#define FM10K_SW_STATS_XEC FM10K_SW_REG_OFF(0x3804)
+#define FM10K_SW_STATS_VLAN_DROP FM10K_SW_REG_OFF(0x3805)
+#define FM10K_SW_STATS_LOOPBACK_DROP FM10K_SW_REG_OFF(0x3806)
+#define FM10K_SW_STATS_NODESC_DROP FM10K_SW_REG_OFF(0x3807)
+#define FM10K_SW_RRTIME_CFG FM10K_SW_REG_OFF(0x3808)
+#define FM10K_SW_RRTIME_LIMIT(n_) \
+ FM10K_SW_REG_OFF(0x380C + 0x40 * (n_))
+#define FM10K_SW_RRTIME_COUNT(n_) \
+ FM10K_SW_REG_OFF(0x3810 + 0x40 * (n_))
+#define FM10K_SW_SYSTIME FM10K_SW_REG_OFF(0x3814)
+#define FM10K_SW_SYSTIME0 FM10K_SW_REG_OFF(0x3816)
+#define FM10K_SW_SYSTIME_CFG FM10K_SW_REG_OFF(0x3818)
+#define FM10K_SW_PFVFBME FM10K_SW_REG_OFF(0x381A)
+#define FM10K_SW_PHYADDR FM10K_SW_REG_OFF(0x381C)
+#define FM10K_SW_RDBAL(q_) \
+ FM10K_SW_REG_OFF(0x4000 + 0x40 * (q_))
+#define FM10K_SW_RDBAH(q_) \
+ FM10K_SW_REG_OFF(0x4001 + 0x40 * (q_))
+#define FM10K_SW_RDLEN(q_) \
+ FM10K_SW_REG_OFF(0x4002 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x4003 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_RXCTRL_HEADER_TPHEN (1 << 6)
+#define FM10K_SW_TPH_RXCTRL_PAYLOAD_TPHEN (1 << 7)
+#define FM10K_SW_TPH_RXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_RXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_RXCTRL_DATA_WRITE_RO_EN (1 << 13)
+#define FM10K_SW_TPH_RXCTRL_REP_HEADER_RO_EN (1 << 15)
+#define FM10K_SW_RDH(q_) \
+ FM10K_SW_REG_OFF(0x4004 + 0x40 * (q_))
+#define FM10K_SW_RDT(q_) \
+ FM10K_SW_REG_OFF(0x4005 + 0x40 * (q_))
+#define FM10K_SW_RDT_RDT_lsb 0
+#define FM10K_SW_RDT_RDT_msb 15
+#define FM10K_SW_RXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x4006 + 0x40 * (q_))
+#define FM10K_SW_RXQCTL_ENABLE (1 << 0)
+#define FM10K_SW_RXQCTL_VF_lsb 2
+#define FM10K_SW_RXQCTL_VF_msb 7
+#define FM10K_SW_RXQCTL_OWNED_BY_VF (1 << 8)
+#define FM10K_SW_RXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x4007 + 0x40 * (q_))
+#define FM10K_SW_RXDCTL_MAX_TIME_lsb 0
+#define FM10K_SW_RXDCTL_MAX_TIME_msb 7
+#define FM10K_SW_RXDCTL_WRITE_BACK_IMM (1 << 8)
+#define FM10K_SW_RXDCTL_DROP_ON_EMPTY (1 << 9)
+#define FM10K_SW_RXDCTL_WRITE_RSS_HASH (1 << 10)
+#define FM10K_SW_RXINT(q_) \
+ FM10K_SW_REG_OFF(0x4008 + 0x40 * (q_))
+#define FM10K_SW_RXINT_INT_lsb 0
+#define FM10K_SW_RXINT_INT_msb 7
+#define FM10K_SW_RXINT_INT_TIMER_lsb 8
+#define FM10K_SW_RXINT_INT_TIMER_msb 9
+#define FM10K_SW_SRRCTL(q_) \
+ FM10K_SW_REG_OFF(0x4009 + 0x40 * (q_))
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_lsb 0
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_msb 7
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_lsb 8
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_msb 13
+#define FM10K_SW_SRRCTL_DESC_TYPE_lsb 14
+#define FM10K_SW_SRRCTL_DESC_TYPE_msb 15
+#define FM10K_SW_SRRCTL_DESC_TYPE_NO_SPLIT 0
+#define FM10K_SW_SRRCTL_DESC_TYPE_HDR_SPLIT 1
+#define FM10K_SW_SRRCTL_DESC_TYPE_SIZE_SPLIT 2
+#define FM10K_SW_SRRCTL_PSR_TYPE_lsb 16
+#define FM10K_SW_SRRCTL_PSR_TYPE_msb 29
+#define FM10K_SW_SRRCTL_LOOPBACK_SUPPRESS (1 << 30)
+#define FM10K_SW_SRRCTL_BUFFER_CHAINING_EN (1 << 31)
+#define FM10K_SW_QPRC(q_) \
+ FM10K_SW_REG_OFF(0x400A + 0x40 * (q_))
+#define FM10K_SW_QPRDC(q_) \
+ FM10K_SW_REG_OFF(0x400B + 0x40 * (q_))
+#define FM10K_SW_QBRC_L(q_) \
+ FM10K_SW_REG_OFF(0x400C + 0x40 * (q_))
+#define FM10K_SW_QBRC_H(q_) \
+ FM10K_SW_REG_OFF(0x400D + 0x40 * (q_))
+#define FM10K_SW_RX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x400E + 0x40 * (q_))
+#define FM10K_SW_TDBAL(q_) \
+ FM10K_SW_REG_OFF(0x8000 + 0x40 * (q_))
+#define FM10K_SW_TDBAH(q_) \
+ FM10K_SW_REG_OFF(0x8001 + 0x40 * (q_))
+#define FM10K_SW_TDLEN(q_) \
+ FM10K_SW_REG_OFF(0x8002 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x8003 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_TXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_TXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_TXCTRL_DATA_READ_RO_EN (1 << 13)
+#define FM10K_SW_TDH(q_) \
+ FM10K_SW_REG_OFF(0x8004 + 0x40 * (q_))
+#define FM10K_SW_TDT(q_) \
+ FM10K_SW_REG_OFF(0x8005 + 0x40 * (q_))
+#define FM10K_SW_TDT_TDT_lsb 0
+#define FM10K_SW_TDT_TDT_msb 15
+#define FM10K_SW_TXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x8006 + 0x40 * (q_))
+#define FM10K_SW_TXDCTL_PTHRESH_lsb 0
+#define FM10K_SW_TXDCTL_PTHRESH_msb 6
+#define FM10K_SW_TXDCTL_HTHRESH_lsb 7
+#define FM10K_SW_TXDCTL_HTHRESH_msb 13
+#define FM10K_SW_TXDCTL_ENABLE (1 << 14)
+#define FM10K_SW_TXDCTL_MAX_TIME_lsb 16
+#define FM10K_SW_TXDCTL_MAX_TIME_msb 27
+#define FM10K_SW_TXDCTL_PUSH_DESC (1 << 28)
+#define FM10K_SW_TXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x8007 + 0x40 * (q_))
+#define FM10K_SW_TXQCTL_VF_lsb 0
+#define FM10K_SW_TXQCTL_VF_msb 5
+#define FM10K_SW_TXQCTL_OWNED_BY_VF (1 << 6)
+#define FM10K_SW_TXQCTL_PC_lsb 7
+#define FM10K_SW_TXQCTL_PC_msb 9
+#define FM10K_SW_TXQCTL_TC_lsb 10
+#define FM10K_SW_TXQCTL_TC_msb 15
+#define FM10K_SW_TXQCTL_VID_lsb 16
+#define FM10K_SW_TXQCTL_VID_msb 27
+#define FM10K_SW_TXQCTL_UNLIMITED_BW (1 << 28)
+#define FM10K_SW_TXQCTL_PUSH_MODE_DIS (1 << 29)
+#define FM10K_SW_TXINT(q_) \
+ FM10K_SW_REG_OFF(0x8008 + 0x40 * (q_))
+#define FM10K_SW_TXINT_INT_lsb 0
+#define FM10K_SW_TXINT_INT_msb 7
+#define FM10K_SW_TXINT_INT_TIMER_lsb 8
+#define FM10K_SW_TXINT_INT_TIMER_msb 9
+#define FM10K_SW_QPTC(q_) \
+ FM10K_SW_REG_OFF(0x8009 + 0x40 * (q_))
+#define FM10K_SW_QBTC_L(q_) \
+ FM10K_SW_REG_OFF(0x800A + 0x40 * (q_))
+#define FM10K_SW_QBTC_H(q_) \
+ FM10K_SW_REG_OFF(0x800B + 0x40 * (q_))
+#define FM10K_SW_TQDLOC(q_) \
+ FM10K_SW_REG_OFF(0x800C + 0x40 * (q_))
+#define FM10K_SW_TQDLOC_BASE_lsb 0
+#define FM10K_SW_TQDLOC_BASE_msb 15
+#define FM10K_SW_TQDLOC_SIZE_lsb 16
+#define FM10K_SW_TQDLOC_SIZE_msb 19
+#define FM10K_SW_TQDLOC_SIZE_32 5
+#define FM10K_SW_TQDLOC_SIZE_64 6
+#define FM10K_SW_TQDLOC_SIZE_128 7
+#define FM10K_SW_TX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x800D + 0x40 * (q_))
+#define FM10K_SW_TX_SGLORT_SGLORT_lsb 0
+#define FM10K_SW_TX_SGLORT_SGLORT_msb 15
+#define FM10K_SW_PFVTCTL(q_) \
+ FM10K_SW_REG_OFF(0x800E + 0x40 * (q_))
+#define FM10K_SW_TX_DESC(q_, d_, w_) \
+ FM10K_SW_REG_OFF(0x40000 + \
+ 0x400 * (q_) + 0x4 * (d_) + (w_))
+#define FM10K_SW_PBACL(n_) \
+ FM10K_SW_REG_OFF(0x10000 + (n_))
+#define FM10K_SW_INT_MAP(n_) \
+ FM10K_SW_REG_OFF(0x10080 + (n_))
+#define FM10K_SW_INT_MAP_ENTRIES 8
+#define FM10K_SW_INT_MAP_INDEX_MAILBOX 0
+#define FM10K_SW_INT_MAP_INDEX_FAULT 1
+#define FM10K_SW_INT_MAP_INDEX_SWITCH_UP_DOWN 2
+#define FM10K_SW_INT_MAP_INDEX_SWITCH 3
+#define FM10K_SW_INT_MAP_INDEX_SRAM 4
+#define FM10K_SW_INT_MAP_INDEX_VFLR 5
+#define FM10K_SW_INT_MAP_INDEX_MAX_HOLD_TIME 6
+#define FM10K_SW_INT_MAP_INT_lsb 0
+#define FM10K_SW_INT_MAP_INT_msb 7
+#define FM10K_SW_INT_MAP_INT_TIMER_lsb 8
+#define FM10K_SW_INT_MAP_INT_TIMER_msb 9
+#define FM10K_SW_MSIX_VECTOR(v_) \
+ FM10K_SW_REG_OFF(0x11000 + 0x4 * (v_))
+#define FM10K_SW_INT_CTRL \
+ FM10K_SW_REG_OFF(0x12000)
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_lsb 0
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_msb 9
+#define FM10K_SW_INT_CTRL_ENABLE_MODERATOR (1 << 10)
+#define FM10K_SW_ITR(v_) \
+ FM10K_SW_REG_OFF(0x12400 + (v_))
+
+/*
+ * Interrupt throttle timer intervals in microseconds. These provide the
+ * direct values for programming the ITR interval field when using a Gen3
+ * PCLK, otherwise they need to be scaled appropriately.
+ */
+#define FM10K_SW_ITR_INTERVAL_20K 50
+#define FM10K_SW_ITR_INTERVAL_40K 25
+#define FM10K_SW_ITR_INTERVAL_0_lsb 0
+#define FM10K_SW_ITR_INTERVAL_0_msb 11
+#define FM10K_SW_ITR_INTERVAL_1_lsb 12
+#define FM10K_SW_ITR_INTERVAL_1_msb 23
+#define FM10K_SW_ITR_TIMER_0_EXPIRED (1 << 24)
+#define FM10K_SW_ITR_TIMER_1_EXPIRED (1 << 25)
+#define FM10K_SW_ITR_PENDING_0 (1 << 26)
+#define FM10K_SW_ITR_PENDING_1 (1 << 27)
+#define FM10K_SW_ITR_PENDING_2 (1 << 28)
+#define FM10K_SW_ITR_AUTO_MASK (1 << 29)
+#define FM10K_SW_ITR_MASK_lsb 30
+#define FM10K_SW_ITR_MASK_msb 31
+#define FM10K_SW_ITR_MASK_R_ENABLED 0
+#define FM10K_SW_ITR_MASK_R_BLOCKED 1
+#define FM10K_SW_ITR_MASK_W_KEEP 0
+#define FM10K_SW_ITR_MASK_W_BLOCK 1
+#define FM10K_SW_ITR_MASK_W_ENABLE 2
+#define FM10K_SW_ITR2(v_) \
+ FM10K_SW_REG_OFF(0x12800 + 0x2 * (v_))
+#define FM10K_SW_IP FM10K_SW_REG_OFF(0x13000)
+#define FM10K_SW_IP_HOT_RESET (1 << 0)
+#define FM10K_SW_IP_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IP_MAILBOX (1 << 2)
+#define FM10K_SW_IP_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IP_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IP_PFLR (1 << 5)
+#define FM10K_SW_IP_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IP_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IP_NOT_IN_RESET (1 << 8)
+#define FM10K_SW_IP_TIMEOUT (1 << 9)
+#define FM10K_SW_IP_VFLR (1 << 10)
+#define FM10K_SW_IM FM10K_SW_REG_OFF(0x13001)
+#define FM10K_SW_IM_ALL FM10K_SW_MASK32(10, 0)
+#define FM10K_SW_IM_HOT_RESET (1 << 0)
+#define FM10K_SW_IM_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IM_MAILBOX (1 << 2)
+#define FM10K_SW_IM_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IM_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IM_PFLR (1 << 5)
+#define FM10K_SW_IM_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IM_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IM_TIMEOUT (1 << 9)
+#define FM10K_SW_IM_VFLR (1 << 10)
+#define FM10K_SW_IB FM10K_SW_REG_OFF(0x13002)
+#define FM10K_SW_SRAM_IP FM10K_SW_REG_OFF(0x13003)
+#define FM10K_SW_SRAM_IM FM10K_SW_REG_OFF(0x13004)
+#define FM10K_SW_VLAN_TABLE(f_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_))
+#define FM10K_SW_VLAN_TABLE_ENTRIES 128
+#define FM10K_SW_VLAN_TABLE_ENTRY(f_, n_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + (n_))
+#define FM10K_SW_VLAN_TABLE_VLAN_ENTRY(f_, vl_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + ((vl_) >> 5))
+#define FM10K_SW_VLAN_TABLE_VLAN_BIT(vl_) (1 << ((vl_) & 0x1f))
+#define FM10K_SW_MBMEM(n_) \
+ FM10K_SW_REG_OFF(0x18000 + (n_))
+#define FM10K_SW_MBX(vf_) \
+ FM10K_SW_REG_OFF(0x18800 + (vf_))
+#define FM10K_SW_MBICR(vf_) \
+ FM10K_SW_REG_OFF(0x18840 + ((vf_) >> 5))
+#define FM10K_SW_GMBX FM10K_SW_REG_OFF(0x18842)
+#define FM10K_SW_GMBX_GLOBAL_REQ (1 << 1)
+#define FM10K_SW_GMBX_GLOBAL_ACK (1 << 2)
+#define FM10K_SW_GMBX_PF_REQ_INTERRUPT (1 << 3)
+#define FM10K_SW_GMBX_PF_ACK_INTERRUPT (1 << 4)
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_lsb 5
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_msb 6
+#define FM10K_SW_GMBX_INTERRUPT_NO_CHANGE 0
+#define FM10K_SW_GMBX_INTERRUPT_ENABLE 1
+#define FM10K_SW_GMBX_INTERRUPT_DISABLE 2
+#define FM10K_SW_GMBX_PF_REQ (1 << 7)
+#define FM10K_SW_GMBX_PF_ACK (1 << 8)
+#define FM10K_SW_GMBX_GLOBAL_REQ_INTERRUPT (1 << 9)
+#define FM10K_SW_GMBX_GLOBAL_ACK_INTERRUPT (1 << 10)
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_lsb 11
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_msb 12
+#define FM10K_SW_PFVFLRE FM10K_SW_REG_OFF(0x18844)
+#define FM10K_SW_PFVFLREC FM10K_SW_REG_OFF(0x18846)
+#define FM10K_SW_TEST_CFG0 FM10K_SW_REG_OFF(0x18849)
+#define FM10K_SW_INT_SRAM_CTRL FM10K_SW_REG_OFF(0x18850)
+#define FM10K_SW_FUM_SRAM_CTRL FM10K_SW_REG_OFF(0x18852)
+#define FM10K_SW_PCA_SRAM_CTRL FM10K_SW_REG_OFF(0x18854)
+#define FM10K_SW_PP_SRAM_CTRL FM10K_SW_REG_OFF(0x18858)
+#define FM10K_SW_PCW_SRAM_CTRL FM10K_SW_REG_OFF(0x1885C)
+#define FM10K_SW_RHI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18872)
+#define FM10K_SW_RHI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18860)
+#define FM10K_SW_THI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18864)
+#define FM10K_SW_THI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18868)
+#define FM10K_SW_TIMEOUT_CFG FM10K_SW_REG_OFF(0x1886B)
+#define FM10K_SW_LVMMC FM10K_SW_REG_OFF(0x18880)
+#define FM10K_SW_LVMMI FM10K_SW_REG_OFF(0x18881)
+#define FM10K_SW_HOST_MISC FM10K_SW_REG_OFF(0x19000)
+#define FM10K_SW_HOST_LANE_CTRL FM10K_SW_REG_OFF(0x19001)
+#define FM10K_SW_SERDES_CTRL(l_) \
+ FM10K_SW_REG_OFF(0x19010 + 0x2 * (l_))
+
+/*
+ * BAR4 registers
+ */
+
+/*
+ * Access to non-master PEP registers via BAR4
+ */
+#define FM10K_SW_PCIE_GLOBAL(p_, r_) \
+ (FM10K_SW_REG_OFF(((p_) + 1) << 20) | FM10K_SW_##r_)
+
+/*
+ * SBUS register fields for use with both the PCIE and EPL SBUS interfaces.
+ */
+#define FM10K_SW_SBUS_CFG_SBUS_CONTROLLER_RESET (1 << 0)
+#define FM10K_SW_SBUS_CFG_ROM_ENABLE (1 << 1)
+#define FM10K_SW_SBUS_CFG_ROM_BUSY (1 << 2)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_PASS (1 << 3)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_FAIL (1 << 4)
+
+#define FM10K_SW_SBUS_COMMAND_REGISTER_lsb 0
+#define FM10K_SW_SBUS_COMMAND_REGISTER_msb 7
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_lsb 8
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_msb 15
+#define FM10K_SW_SBUS_COMMAND_OP_lsb 16
+#define FM10K_SW_SBUS_COMMAND_OP_msb 23
+#define FM10K_SW_SBUS_OP_RESET 0x20
+#define FM10K_SW_SBUS_OP_WRITE 0x21
+#define FM10K_SW_SBUS_OP_READ 0x22
+#define FM10K_SW_SBUS_COMMAND_EXECUTE (1 << 24)
+#define FM10K_SW_SBUS_COMMAND_BUSY (1 << 25)
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_lsb 26
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_msb 28
+#define FM10K_SW_SBUS_RESULT_RESET 0x00
+#define FM10K_SW_SBUS_RESULT_WRITE 0x01
+#define FM10K_SW_SBUS_RESULT_READ 0x04
+
+#define FM10K_SW_SBUS_ADDR_EPL_RMON2 1
+#define FM10K_SW_SBUS_ADDR_EPL_LANE(e_, l_) (4 * (e_) + (l_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_SERDES(s_) ((s_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_RMON3 38
+#define FM10K_SW_SBUS_ADDR_EPL_PMRO 39
+
+/* Common to both EPL and PCIE
+ */
+#define FM10K_SW_SBUS_ADDR_SPICO 253
+#define FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER 254
+#define FM10K_SW_SBUS_ADDR_BROADCAST 0xFF
+
+
+#define FM10K_SW_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x000000)
+#define FM10K_SW_MGMT_REG(wo_) \
+ (FM10K_SW_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_FATAL_CODE FM10K_SW_MGMT_REG(0x0)
+#define FM10K_SW_LAST_FATAL_CODE FM10K_SW_MGMT_REG(0x1)
+#define FM10K_SW_FATAL_COUNT FM10K_SW_MGMT_REG(0x2)
+#define FM10K_SW_SOFT_RESET FM10K_SW_MGMT_REG(0x3)
+#define FM10K_SW_SOFT_RESET_COLD_RESET (1 << 0)
+#define FM10K_SW_SOFT_RESET_EPL_RESET (1 << 1)
+#define FM10K_SW_SOFT_RESET_SWITCH_RESET (1 << 2)
+#define FM10K_SW_SOFT_RESET_SWITCH_READY (1 << 3)
+#define FM10K_SW_SOFT_RESET_PCIE_RESET(p_) (1 << ((p_) + 4))
+#define FM10K_SW_SOFT_RESET_PCIE_ACTIVE(p_) (1 << ((p_) + 13))
+#define FM10K_SW_DEVICE_CFG FM10K_SW_MGMT_REG(0x4)
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS 4
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(p_) (1 << (p_))
+#define FM10K_SW_DEVICE_CFG_PCIE_100G_DIS (1 << 4)
+#define FM10K_SW_DEVICE_CFG_FEATURE_lsb 5
+#define FM10K_SW_DEVICE_CFG_FEATURE_msb 6
+#define FM10K_SW_DEVICE_CFG_PCIE_FULL 0
+#define FM10K_SW_DEVICE_CFG_PCIE_HALF 1
+#define FM10K_SW_DEVICE_CFG_PCIE_BASIC 2
+#define FM10K_SW_DEVICE_CFG_PCIE_EN(p_) (1 << (7 + (p_)))
+#define FM10K_SW_RESET_CFG FM10K_SW_MGMT_REG(0x5)
+#define FM10K_SW_WATCHDOG_CFG FM10K_SW_MGMT_REG(0x6)
+#define FM10K_SW_MGMT_SCRATCH(n_) FM10K_SW_MGMT_REG(0x8 + (n_))
+#define FM10K_SW_VITAL_PRODUCT_DATA FM10K_SW_MGMT_REG(0x304)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x400)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_lsb64 0
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_msb64 8
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_lsb64 9
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_msb64 17
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_lsb64 18
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_msb64 26
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_lsb64 27
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_msb64 28
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CORE (1ULL << 29)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SOFTWARE (1ULL << 30)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_GPIO (1ULL << 31)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C (1ULL << 32)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_MDIO (1ULL << 33)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CRM (1ULL << 34)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_TAIL (1ULL << 35)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_HEAD (1ULL << 36)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PINS (1ULL << 39)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FIBM (1ULL << 40)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_BSM (1ULL << 41)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_INT \
+ FM10K_SW_MGMT_REG(0x402)
+#define FM10K_SW_INTERRUPT_MASK_PCIE(p_) \
+ FM10K_SW_MGMT_REG(0x420 + 0x2 * (p_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_lsb64 0
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_msb64 8
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_lsb64 9
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_msb64 17
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_lsb64 18
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_msb64 26
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_lsb64 27
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_msb64 28
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CORE (1ULL << 29)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SOFTWARE (1ULL << 30)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_GPIO (1ULL << 31)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_I2C (1ULL << 32)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_MDIO (1ULL << 33)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CRM (1ULL << 34)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_TAIL (1ULL << 35)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_HEAD (1ULL << 36)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PINS (1ULL << 39)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FIBM (1ULL << 40)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_BSM (1ULL << 41)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_ALL FM10K_SW_MASK64(42, 0)
+#define FM10K_SW_INTERRUPT_MASK_FIBM FM10K_SW_MGMT_REG(0x440)
+#define FM10K_SW_INTERRUPT_MASK_BSM FM10K_SW_MGMT_REG(0x442)
+#define FM10K_SW_CORE_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x444)
+#define FM10K_SW_CORE_INTERRUPT_MASK FM10K_SW_MGMT_REG(0x445)
+#define FM10K_SW_SRAM_ERR_IP FM10K_SW_MGMT_REG(0x446)
+#define FM10K_SW_SRAM_ERR_IM FM10K_SW_MGMT_REG(0x448)
+#define FM10K_SW_PINS_STAT FM10K_SW_MGMT_REG(0x44A)
+#define FM10K_SW_PINS_IP FM10K_SW_MGMT_REG(0x44B)
+#define FM10K_SW_PINS_IM FM10K_SW_MGMT_REG(0x44C)
+#define FM10K_SW_SW_IP FM10K_SW_MGMT_REG(0x44D)
+#define FM10K_SW_SW_IM FM10K_SW_MGMT_REG(0x44E)
+#define FM10K_SW_SW_TEST_AND_SET FM10K_SW_MGMT_REG(0x44F)
+#define FM10K_SW_LSM_CLKOBS_CTRL FM10K_SW_MGMT_REG(0x450)
+#define FM10K_SW_CHIP_VERSION FM10K_SW_MGMT_REG(0x452)
+#define FM10K_SW_BSM_SCRATCH(n_) FM10K_SW_MGMT_REG(0x800 + (n_))
+#define FM10K_SW_BSM_CTRL FM10K_SW_MGMT_REG(0xC00)
+#define FM10K_SW_BSM_ARGS FM10K_SW_MGMT_REG(0xC01)
+#define FM10K_SW_BSM_ADDR_OFFSET(n_) FM10K_SW_MGMT_REG(0xC04 + (n_))
+#define FM10K_SW_BSM_COUNTER(n_) FM10K_SW_MGMT_REG(0xC08 + (n_))
+#define FM10K_SW_BSM_SRAM_CTRL FM10K_SW_MGMT_REG(0xC0A)
+#define FM10K_SW_BSM_IP FM10K_SW_MGMT_REG(0xC0B)
+#define FM10K_SW_BSM_IM FM10K_SW_MGMT_REG(0xC0C)
+#define FM10K_SW_PIN_STRAP_STAT FM10K_SW_MGMT_REG(0xC0D)
+#define FM10K_SW_FUSE_DATA_0 FM10K_SW_MGMT_REG(0xC0E)
+#define FM10K_SW_FUSE_SKU_lsb 11
+#define FM10K_SW_FUSE_SKU_msb 15
+#define FM10K_SW_FUSE_SKU_FM10840 0
+#define FM10K_SW_FUSE_SKU_FM10420 1
+#define FM10K_SW_FUSE_SKU_FM10064 2
+#define FM10K_SW_FUSE_DATA_1 FM10K_SW_MGMT_REG(0xC0F)
+#define FM10K_SW_BIST_CTRL FM10K_SW_MGMT_REG(0xC10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_MODE_EPL (1ULL << 41)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC (1ULL << 42)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL (1ULL << 43)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_BSM (1ULL << 44)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_CRM (1ULL << 45)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FIBM (1ULL << 46)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_SBM (1ULL << 47)
+#define FM10K_SW_REI_CTRL FM10K_SW_MGMT_REG(0xC12)
+#define FM10K_SW_REI_STAT FM10K_SW_MGMT_REG(0xC13)
+#define FM10K_SW_GPIO_CFG FM10K_SW_MGMT_REG(0xC15)
+#define FM10K_SW_GPIO_DATA FM10K_SW_MGMT_REG(0xC16)
+#define FM10K_SW_GPIO_IP FM10K_SW_MGMT_REG(0xC17)
+#define FM10K_SW_GPIO_IM FM10K_SW_MGMT_REG(0xC18)
+#define FM10K_SW_I2C_CFG FM10K_SW_MGMT_REG(0xC19)
+#define FM10K_SW_I2C_CFG_SLAVE_ENABLE (1 << 0)
+#define FM10K_SW_I2C_CFG_ADDR_lsb 1
+#define FM10K_SW_I2C_CFG_ADDR_msb 7
+#define FM10K_SW_I2C_CFG_DIVIDER_lsb 8
+#define FM10K_SW_I2C_CFG_DIVIDER_msb 19
+#define FM10K_SW_I2C_CFG_DIVIDER_100_KHZ 52
+#define FM10K_SW_I2C_CFG_DIVIDER_400_KHZ 10
+#define FM10K_SW_I2C_CFG_INTERRUPT_MASK (1 << 20)
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_lsb 21
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_msb 27
+#define FM10K_SW_I2C_DATA(n_) FM10K_SW_MGMT_REG(0xC1C + (n_))
+#define FM10K_SW_I2C_CTRL FM10K_SW_MGMT_REG(0xC20)
+#define FM10K_SW_I2C_CTRL_ADDR_lsb 0
+#define FM10K_SW_I2C_CTRL_ADDR_msb 7
+#define FM10K_SW_I2C_CTRL_COMMAND_lsb 8
+#define FM10K_SW_I2C_CTRL_COMMAND_msb 9
+#define FM10K_SW_I2C_COMMAND_NULL 0
+#define FM10K_SW_I2C_COMMAND_WR 1
+#define FM10K_SW_I2C_COMMAND_WR_RD 2
+#define FM10K_SW_I2C_COMMAND_RD 3
+#define FM10K_SW_I2C_CTRL_LENGTH_W_lsb 10
+#define FM10K_SW_I2C_CTRL_LENGTH_W_msb 13
+#define FM10K_SW_I2C_CTRL_LENGTH_R_lsb 14
+#define FM10K_SW_I2C_CTRL_LENGTH_R_msb 17
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_lsb 18
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_msb 21
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_lsb 22
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_msb 25
+#define FM10K_SW_I2C_COMPLETION_RUNNING 0
+#define FM10K_SW_I2C_COMPLETION_NORMAL 1
+#define FM10K_SW_I2C_COMPLETION_PREMATURE 2
+#define FM10K_SW_I2C_COMPLETION_NO_DEVICE 3
+#define FM10K_SW_I2C_COMPLETION_TIMEOUT 4
+#define FM10K_SW_I2C_COMPLETION_LOST_ARB 5
+#define FM10K_SW_I2C_COMPLETION_BUS_WAIT 6
+#define FM10K_SW_I2C_COMPLETION_INVALID 7
+#define FM10K_SW_I2C_CTRL_INTERRUPT_PENDING (1 << 26)
+#define FM10K_SW_MDIO_CFG FM10K_SW_MGMT_REG(0xC22)
+#define FM10K_SW_MDIO_DATA FM10K_SW_MGMT_REG(0xC23)
+#define FM10K_SW_MDIO_CTRL FM10K_SW_MGMT_REG(0xC24)
+#define FM10K_SW_SPI_TX_DATA FM10K_SW_MGMT_REG(0xC26)
+#define FM10K_SW_SPI_RX_DATA FM10K_SW_MGMT_REG(0xC27)
+#define FM10K_SW_SPI_HEADER FM10K_SW_MGMT_REG(0xC28)
+#define FM10K_SW_SPI_CTRL FM10K_SW_MGMT_REG(0xC29)
+#define FM10K_SW_LED_CFG FM10K_SW_MGMT_REG(0xC2B)
+#define FM10K_SW_SCAN_DATA_IN FM10K_SW_MGMT_REG(0xC2D)
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_lsb 0
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_msb 24
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
+#define FM10K_SW_SCAN_DATA_IN_INJECT (1 << 28)
+#define FM10K_SW_SCAN_DATA_IN_DRAIN (1 << 29)
+#define FM10K_SW_SCAN_DATA_IN_PASSTHRU (1 << 30)
+#define FM10K_SW_SCAN_DATA_IN_SINGLE (1 << 31)
+#define FM10K_SW_CRM_DATA(m_, n_) \
+ FM10K_SW_MGMT_REG(0x1000 + 0x2 * (m_) + (n_))
+
+#define FM10K_SW_CRM_CTRL FM10K_SW_MGMT_REG(0x2000)
+#define FM10K_SW_CRM_STATUS FM10K_SW_MGMT_REG(0x2001)
+#define FM10K_SW_CRM_TIME FM10K_SW_MGMT_REG(0x2002)
+#define FM10K_SW_CRM_SRAM_CTRL FM10K_SW_MGMT_REG(0x2004)
+#define FM10K_SW_CRM_IP FM10K_SW_MGMT_REG(0x2008)
+#define FM10K_SW_CRM_IM FM10K_SW_MGMT_REG(0x200C)
+#define FM10K_SW_CRM_COMMAND(n_) FM10K_SW_MGMT_REG(0x2080 + 0x2 * (n_))
+#define FM10K_SW_CRM_REGISTER(n_) FM10K_SW_MGMT_REG(0x2100 + 0x2 * (n_))
+#define FM10K_SW_CRM_PERIOD(n_) FM10K_SW_MGMT_REG(0x2180 + 0x2 * (n_))
+#define FM10K_SW_CRM_PARAM(n_) FM10K_SW_MGMT_REG(0x2200 + (n_))
+#define FM10K_SW_PLL_PCIE_CTRL FM10K_SW_MGMT_REG(0x2241)
+#define FM10K_SW_PLL_PCIE_STAT FM10K_SW_MGMT_REG(0x2242)
+#define FM10K_SW_SBUS_PCIE_CFG FM10K_SW_MGMT_REG(0x2243)
+#define FM10K_SW_SBUS_PCIE_COMMAND FM10K_SW_MGMT_REG(0x2244)
+#define FM10K_SW_SBUS_PCIE_REQUEST FM10K_SW_MGMT_REG(0x2245)
+#define FM10K_SW_SBUS_PCIE_RESPONSE FM10K_SW_MGMT_REG(0x2246)
+#define FM10K_SW_SBUS_PCIE_SPICO_IN FM10K_SW_MGMT_REG(0x2247)
+#define FM10K_SW_SBUS_PCIE_SPICO_OUT FM10K_SW_MGMT_REG(0x2248)
+#define FM10K_SW_SBUS_PCIE_IP FM10K_SW_MGMT_REG(0x2249)
+#define FM10K_SW_SBUS_PCIE_IM FM10K_SW_MGMT_REG(0x224A)
+#define FM10K_SW_MGMT_SYSTIME_CFG FM10K_SW_MGMT_REG(0x224C)
+#define FM10K_SW_MGMT_SYSTIME FM10K_SW_MGMT_REG(0x224E)
+#define FM10K_SW_MGMT_SYSTIME0 FM10K_SW_MGMT_REG(0x2250)
+#define FM10K_SW_SYSTIME_PULSE_(n_) FM10K_SW_MGMT_REG(0x2252 + (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_LO(n_) FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_HI(n_) \
+ FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_) + 0x1)
+#define FM10K_SW_PCIE_XPLL_CTRL FM10K_SW_MGMT_REG(0x3000)
+#define FM10K_SW_PCIE_CLK_CTRL FM10K_SW_MGMT_REG(0x3001)
+#define FM10K_SW_PCIE_CLK_CTRL2 FM10K_SW_MGMT_REG(0x3002)
+#define FM10K_SW_PCIE_CLKMON_RATIO_CFG FM10K_SW_MGMT_REG(0x3003)
+#define FM10K_SW_PCIE_CLKMON_TOLERANCE_CFG FM10K_SW_MGMT_REG(0x3004)
+#define FM10K_SW_PCIE_CLKMON_DEADLINES_CFG FM10K_SW_MGMT_REG(0x3005)
+#define FM10K_SW_PCIE_CLK_STAT FM10K_SW_MGMT_REG(0x3006)
+#define FM10K_SW_PCIE_CLK_IP FM10K_SW_MGMT_REG(0x3007)
+#define FM10K_SW_PCIE_CLK_IM FM10K_SW_MGMT_REG(0x3008)
+#define FM10K_SW_PCIE_WARM_RESET_DELAY FM10K_SW_MGMT_REG(0x3009)
+#define FM10K_SW_EPL_BASE FM10K_SW_REG_OFF(0x0E0000)
+#define FM10K_SW_EPL_PORT_REG(p_, r_) \
+ (FM10K_SW_EPL_BASE + FM10K_SW_REG_OFF(0x400 * (p_) + (r_)))
+#define FM10K_SW_EPL_LANE_REG(p_, l_, r_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x80 * (l_) + (r_))
+
+/* EPL enumerated types
+ */
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX 0
+#define FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1 1
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_L4 2
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_L4_XX 3
+#define FM10K_SW_EPL_QPL_MODE_XX_L4_XX_XX 4
+#define FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX 5
+#define FM10K_SW_EPL_PCS_SEL_DISABLE 0
+#define FM10K_SW_EPL_PCS_SEL_AN_73 1
+#define FM10K_SW_EPL_PCS_SEL_SGMII_10 2
+#define FM10K_SW_EPL_PCS_SEL_SGMII_100 3
+#define FM10K_SW_EPL_PCS_SEL_SGMII_1000 4
+#define FM10K_SW_EPL_PCS_SEL_1000BASEX 5
+#define FM10K_SW_EPL_PCS_SEL_10GBASER 6
+#define FM10K_SW_EPL_PCS_SEL_40GBASER 7
+#define FM10K_SW_EPL_PCS_SEL_100GBASER 8
+
+#define FM10K_SW_LANE_OVERRIDE_NORMAL 0
+#define FM10K_SW_LANE_OVERRIDE_FORCE_GOOD 1
+#define FM10K_SW_LANE_OVERRIDE_FORCE_BAD 2
+
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSTHRU 0
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSSTHRU_CHECK 1
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_GOOD 2
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_BAD 3
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL 4
+
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_DRAIN 0
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL 1
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_NORMAL 2
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_HOLD 3
+
+#define FM10K_SW_TX_MAC_FAULT_MODE_NORMAL 0
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_IDLE 1
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LOCAL_FAULT 2
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_REMOTE_FAULT 3
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LINK_INTERRUPTION 4
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_OK 5
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_ERROR 6
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_USER_VAL 7
+
+#define FM10K_SW_EPL_IP(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x300)
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_lsb 0
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_msb 3
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT(n_) (1 << ((n_) + 0))
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_lsb 4
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_msb 7
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT(n_) (1 << ((n_) + 4))
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_lsb 8
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_msb 11
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT(n_) (1 << ((n_) + 8))
+#define FM10K_SW_EPL_IP_ERROR_INTERRUPT (1 << 12)
+
+#define FM10K_SW_EPL_ERROR_IP(p_) FM10K_SW_EPL_PORT_REG((p_), 0x301)
+#define FM10K_SW_EPL_ERROR_IM(p_) FM10K_SW_EPL_PORT_REG((p_), 0x302)
+#define FM10K_SW_EPL_ERROR_IM_ALL FM10K_SW_MASK32(8, 0)
+#define FM10K_SW_EPL_BIST_STATUS(p_) FM10K_SW_EPL_PORT_REG((p_), 0x303)
+#define FM10K_SW_EPL_CFG_A(p_) FM10K_SW_EPL_PORT_REG((p_), 0x304)
+#define FM10K_SW_EPL_CFG_A_SPEED_UP (1 << 0)
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_lsb 1
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_msb 6
+#define FM10K_SW_EPL_CFG_A_ACTIVE(p_) (1 << ((p_) + 7))
+#define FM10K_SW_EPL_CFG_A_ACTIVE_QUAD (0xf << 7)
+
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_lsb 11
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_msb 16
+
+#define FM10K_SW_EPL_CFG_B(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x305)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_lsb(p_) ((p_) * 4)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_msb(p_) (((p_) * 4) + 3)
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_lsb 16
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_msb 18
+#define FM10K_SW_EPL_LED_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x306)
+
+#define FM10K_SW_EPL_LED_STATUS_PORT_RESET(n_) (1 << ((n_) * 6 + 0))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP(n_) (1 << ((n_) * 6 + 1))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LOCAL_FAULT(n_) (1 << ((n_) * 6 + 2))
+#define FM10K_SW_EPL_LED_STATUS_PORT_REMOTE_FAULT(n_) (1 << ((n_) * 6 + 3))
+#define FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING(n_) (1 << ((n_) * 6 + 4))
+#define FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING(n_) (1 << ((n_) * 6 + 5))
+
+#define FM10K_SW_EPL_FIFO_ERROR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x307)
+#define FM10K_SW_EPL_TX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x308)
+#define FM10K_SW_EPL_TX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x309)
+#define FM10K_SW_EPL_TX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30A)
+#define FM10K_SW_EPL_TX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30B)
+#define FM10K_SW_EPL_TX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30C)
+#define FM10K_SW_EPL_TX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30D)
+#define FM10K_SW_EPL_RX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30E)
+#define FM10K_SW_EPL_RX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30F)
+#define FM10K_SW_EPL_RX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x310)
+#define FM10K_SW_EPL_RX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x311)
+#define FM10K_SW_EPL_RX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x312)
+#define FM10K_SW_EPL_RX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x313)
+#define FM10K_SW_PCS_ML_BASER_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x314)
+#define FM10K_SW_PCS_ML_BASER_RX_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x315)
+#define FM10K_SW_PCS_100GBASER_BIP_0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x317)
+#define FM10K_SW_PCS_100GBASER_BIP_1(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x318)
+#define FM10K_SW_PCS_100GBASER_BIP_2(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x319)
+#define FM10K_SW_PCS_100GBASER_BIP_3(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31A)
+#define FM10K_SW_PCS_100GBASER_BIP_4(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31B)
+#define FM10K_SW_PCS_100GBASER_BIP_5(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31C)
+#define FM10K_SW_PCS_100GBASER_BIP_6(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31D)
+#define FM10K_SW_PCS_100GBASER_BIP_7(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31E)
+#define FM10K_SW_PCS_100GBASER_BIP_8(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31F)
+#define FM10K_SW_PCS_100GBASER_BIP_9(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x320)
+#define FM10K_SW_PCS_100GBASER_BIP_10(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x321)
+#define FM10K_SW_PCS_100GBASER_BIP_11(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x322)
+#define FM10K_SW_PCS_100GBASER_BIP_12(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x323)
+#define FM10K_SW_PCS_100GBASER_BIP_13(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x324)
+#define FM10K_SW_PCS_100GBASER_BIP_14(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x325)
+#define FM10K_SW_PCS_100GBASER_BIP_15(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x326)
+#define FM10K_SW_PCS_100GBASER_BIP_16(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x327)
+#define FM10K_SW_PCS_100GBASER_BIP_17(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x328)
+#define FM10K_SW_PCS_100GBASER_BIP_18(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x329)
+#define FM10K_SW_PCS_100GBASER_BIP_19(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32A)
+#define FM10K_SW_PCS_100GBASER_BLOCK_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32B)
+#define FM10K_SW_PCS_100GBASER_AMPS_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32C)
+#define FM10K_SW_RS_FEC_UNCORRECTED(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x33D)
+#define FM10K_SW_RS_FEC_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32E)
+#define FM10K_SW_RS_FEC_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x330)
+#define FM10K_SW_EPL_SYSTIME(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x331)
+#define FM10K_SW_EPL_SYSTIME0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x332)
+#define FM10K_SW_EPL_SYSTIME_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x333)
+#define FM10K_SW_PORT_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x0)
+#define FM10K_SW_AN_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1)
+
+#define FM10K_SW_AN_IM_ALL FM10K_SW_MASK32(18, 0)
+#define FM10K_SW_LINK_IM(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2)
+#define FM10K_SW_LINK_IM_ALL FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_AN_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3)
+#define FM10K_SW_LINK_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4)
+#define FM10K_SW_MP_CFG(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_73_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_LINK_RULES(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xC)
+
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_lsb 0
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_msb 3
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_lsb 4
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_msb 8
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_lsb 9
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_msb 12
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_lsb 13
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_msb 17
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_lsb 18
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_msb 21
+#define FM10K_SW_MAC_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x10)
+#define FM10K_SW_MAC_CFG_ARRAY_SIZE 8
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_lsb 0
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_msb 5
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_lsb 6
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_msb 9
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_lsb 10
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_msb 17
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_lsb 18
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_msb 25
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_lsb 26
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_msb 31
+
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_lsb 32
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_msb 47
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE_bit 48
+
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_lsb 49
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_msb 51
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_lsb 52
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_msb 55
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_lsb 56
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_msb 63
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_lsb 64
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_msb 65
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_lsb 66
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_msb 71
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_lsb 72
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_msb 83
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_lsb 84
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_msb 95
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_lsb 96
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_msb 107
+#define FM10K_SW_MAC_CFG_TX_LP_IDLE_REQUEST_bit 108
+#define FM10K_SW_MAC_CFG_TX_LPI_AUTOMATIC_bit 109
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_lsb 110
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_msb 117
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_lsb 118
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_msb 119
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_lsb 120
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_msb 127
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_lsb 128
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_msb 129
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_lsb 130
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_msb 132
+#define FM10K_SW_MAC_CFG_TX_OBEY_LINT_bit 133
+#define FM10K_SW_MAC_CFG_TX_IDLE_ENABLE_DIC_bit 134
+#define FM10K_SW_MAC_CFG_CJPAT_ENABLE_bit 135
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_lsb 136
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_msb 143
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_lsb 144
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_msb 159
+#define FM10K_SW_MAC_CFG_START_CHAR_D_lsb 160
+#define FM10K_SW_MAC_CFG_START_CHAR_D_msb 167
+#define FM10K_SW_MAC_CFG_IEEE_1588_ENABLE_bit 168
+#define FM10K_SW_MAC_CFG_FCS_START_bit 169
+#define FM10K_SW_MAC_CFG_PREAMBLE_MODE_bit 170
+#define FM10K_SW_MAC_CFG_COUNTER_WRAP_bit 171
+#define FM10K_SW_MAC_CFG_LINK_FAULT_DISABLE_bit 172
+#define FM10K_SW_MAC_CFG_RX_DRAIN_bit 173
+#define FM10K_SW_MAC_CFG_RX_FCS_FORCE_BAD_bit 174
+#define FM10K_SW_MAC_CFG_RX_IGNORE_CODE_ERRORS_bit 175
+#define FM10K_SW_MAC_CFG_RX_IGNORE_UNDERSIZE_ERRORS_bit 176
+#define FM10K_SW_MAC_CFG_RX_IGNORE_OVERSIZE_ERRORS_bit 177
+#define FM10K_SW_MAC_CFG_RX_IGNORE_FCS_ERRORS_bit 178
+#define FM10K_SW_MAC_CFG_RX_IGNORE_PREAMBLE_ERRORS_bit 179
+#define FM10K_SW_MAC_CFG_RX_IGNORE_IFG_ERRORS_bit 180
+#define FM10K_SW_MAC_CFG_ERR_WRITE_lsb 181
+#define FM10K_SW_MAC_CFG_ERR_WRITE_msb 182
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_lsb 183
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_msb 186
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_lsb 187
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_msb 191
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_lsb 192
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_msb 196
+
+#define FM10K_SW_TX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1A)
+#define FM10K_SW_RX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1C)
+#define FM10K_SW_MAC_1588_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1E)
+#define FM10K_SW_WAKE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x20)
+#define FM10K_SW_MAC_OVERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x21)
+#define FM10K_SW_MAC_JABBER_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x22)
+#define FM10K_SW_MAC_UNDERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x23)
+#define FM10K_SW_MAC_RUNT_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x24)
+#define FM10K_SW_MAC_OVERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x25)
+#define FM10K_SW_MAC_UNDERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x26)
+#define FM10K_SW_MAC_CODE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x27)
+#define FM10K_SW_EPL_TX_FRAME_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x28)
+#define FM10K_SW_MAC_LINK_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x29)
+#define FM10K_SW_PCS_1000BASEX_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2A)
+#define FM10K_SW_PCS_1000BASEX_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2B)
+#define FM10K_SW_PCS_1000BASEX_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2C)
+#define FM10K_SW_PCS_10GBASER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2D)
+#define FM10K_SW_PCS_10GBASER_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2E)
+#define FM10K_SW_PCS_10GBASER_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2F)
+#define FM10K_SW_AN_37_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x30)
+#define FM10K_SW_AN_37_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_37_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_SGMII_AN_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_SGMII_AN_TX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_TX_CONFIG_LOOPBACK(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_RX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x32)
+#define FM10K_SW_AN_73_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x33)
+#define FM10K_SW_AN_73_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_73_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x36)
+#define FM10K_SW_AN_73_TX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x38)
+#define FM10K_SW_AN_73_RX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3A)
+#define FM10K_SW_PCSL_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3C)
+
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_lsb 0
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_msb 7
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE (1 << 8)
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL (1 << 9)
+#define FM10K_SW_PCSL_CFG_RX_GB_NARROW (1 << 10)
+#define FM10K_SW_PCSL_CFG_TX_GB_NARROW (1 << 11)
+
+#define FM10K_SW_MP_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_1000BASEX_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_10GBASER_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_MP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_40GBASER_RX_BIP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_10GBASER_RX_BER_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_DISPARITY_ERROR_8B10B(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_LANE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x45)
+#define FM10K_SW_LANE_SERDES_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x46)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x47)
+
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_msb 1
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT (1 << 5)
+
+#define FM10K_SW_LANE_ACTIVITY_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x48)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x49)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_msb 1
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT (1 << 5)
+#define FM10K_SW_LANE_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4A)
+#define FM10K_SW_LANE_SERDES_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4B)
+
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_lsb 0
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_msb 7
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_lsb 8
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_msb 23
+#define FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK (1 << 12)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_RDY (1 << 24)
+#define FM10K_SW_LANE_SERDES_STATUS_TX_RDY (1 << 25)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_IDLE_DETECT (1 << 26)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_ACTIVITY (1 << 27)
+#define FM10K_SW_SERDES_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4C)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IM_CORE_STATUS_4
+#define FM10K_SW_SERDES_IM_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IM_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IM_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IM_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IM_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IM_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IM_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IM_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IM_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IM_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IM_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_SERDES_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4D)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IP_CORE_STATUS_4
+#define FM10K_SW_SERDES_IP_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IP_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IP_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IP_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IP_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IP_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IP_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IP_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IP_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IP_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IP_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_LANE_ANALOG_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4E)
+#define FM10K_SW_LANE_ANALOG_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4F)
+#define FM10K_SW_LANE_SAI_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x50)
+#define FM10K_SW_LANE_SAI_CFG_CODE_lsb64 0
+#define FM10K_SW_LANE_SAI_CFG_CODE_msb64 15
+#define FM10K_SW_LANE_SAI_CFG_DATA_lsb64 16
+#define FM10K_SW_LANE_SAI_CFG_DATA_msb64 31
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_lsb64 32
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_msb64 47
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_lsb64 48
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_msb64 49
+#define FM10K_SW_LANE_SAI_CFG_REQUEST (1ULL << 50)
+#define FM10K_SW_LANE_SAI_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x52)
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_lsb 0
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_msb 15
+#define FM10K_SW_LANE_SAI_STATUS_COMPLETE (1 << 16)
+#define FM10K_SW_LANE_SAI_STATUS_ACCESS_REQUEST (1 << 17)
+#define FM10K_SW_LANE_SAI_STATUS_IN_PROGRESS (1 << 18)
+#define FM10K_SW_LANE_SAI_STATUS_BUSY (1 << 19)
+#define FM10K_SW_LANE_SAI_STATUS_REQUEST_ERROR (1 << 20)
+
+#define FM10K_SW_PORTS_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x0E8000)
+#define FM10K_SW_PORTS_MGMT_REG(wo_) \
+ (FM10K_SW_PORTS_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_PLL_EPL_CTRL FM10K_SW_PORTS_MGMT_REG(0x0)
+#define FM10K_SW_PLL_EPL_STAT FM10K_SW_PORTS_MGMT_REG(0x1)
+#define FM10K_SW_PLL_FABRIC_CTRL FM10K_SW_PORTS_MGMT_REG(0x2)
+
+#define FM10K_SW_PLL_FABRIC_REFDIV_lsb 3
+#define FM10K_SW_PLL_FABRIC_REFDIV_msb 8
+#define FM10K_SW_PLL_FABRIC_FBDIV4_lsb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV4_msb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV255_lsb 10
+#define FM10K_SW_PLL_FABRIC_FBDIV255_msb 17
+#define FM10K_SW_PLL_FABRIC_OUTDIV_lsb 18
+#define FM10K_SW_PLL_FABRIC_OUTDIV_msb 23
+#define FM10K_SW_PLL_FABRIC_STAT FM10K_SW_PORTS_MGMT_REG(0x3)
+#define FM10K_SW_PLL_FABRIC_LOCK FM10K_SW_PORTS_MGMT_REG(0x4)
+#define FM10K_SW_PLL_FABRIC_FREQSEL_lsb 4
+#define FM10K_SW_PLL_FABRIC_FREQSEL_msb 7
+#define FM10K_SW_PLL_FABRIC_FREQSEL_CTRL 0
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F600 1
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F500 2
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F400 3
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F300 4
+#define FM10K_SW_SBUS_EPL_CFG FM10K_SW_PORTS_MGMT_REG(0x5)
+#define FM10K_SW_SBUS_EPL_COMMAND FM10K_SW_PORTS_MGMT_REG(0x6)
+#define FM10K_SW_SBUS_EPL_REQUEST FM10K_SW_PORTS_MGMT_REG(0x7)
+#define FM10K_SW_SBUS_EPL_RESPONSE FM10K_SW_PORTS_MGMT_REG(0x8)
+#define FM10K_SW_SBUS_EPL_SPICO_IN FM10K_SW_PORTS_MGMT_REG(0x9)
+#define FM10K_SW_SBUS_EPL_SPICO_OUT FM10K_SW_PORTS_MGMT_REG(0xA)
+#define FM10K_SW_SBUS_EPL_IP FM10K_SW_PORTS_MGMT_REG(0xB)
+#define FM10K_SW_SBUS_EPL_IM FM10K_SW_PORTS_MGMT_REG(0xC)
+
+#define FM10K_SW_ETHCLK_CFG(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0xE + (n_))
+#define FM10K_SW_ETHCLK_RATIO(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0x10 + (n_))
+#define FM10K_SW_PM_CLKOBS_CTRL FM10K_SW_PORTS_MGMT_REG(0x12)
+
+#define FM10K_SW_PCIE_CFG_BASE FM10K_SW_REG_OFF(0x120000)
+#define FM10K_SW_PCIE_CFG_REG(wo_) \
+ (FM10K_SW_PCIE_CFG_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * These register offsets can also be passed to fm10k_read_config(), which
+ * will mask off the upper bits
+ */
+#define FM10K_SW_PCIE_CFG_ID FM10K_SW_PCIE_CFG_REG(0x0)
+#define FM10K_SW_PCIE_CFG_CMD FM10K_SW_PCIE_CFG_REG(0x1)
+#define FM10K_SW_PCIE_CFG_1 FM10K_SW_PCIE_CFG_REG(0x2)
+#define FM10K_SW_PCIE_CFG_2 FM10K_SW_PCIE_CFG_REG(0x3)
+#define FM10K_SW_PCIE_CFG_BAR0 FM10K_SW_PCIE_CFG_REG(0x4)
+#define FM10K_SW_PCIE_CFG_BAR1 FM10K_SW_PCIE_CFG_REG(0x5)
+#define FM10K_SW_PCIE_CFG_BAR2 FM10K_SW_PCIE_CFG_REG(0x6)
+#define FM10K_SW_PCIE_CFG_BAR3 FM10K_SW_PCIE_CFG_REG(0x7)
+#define FM10K_SW_PCIE_CFG_BAR4 FM10K_SW_PCIE_CFG_REG(0x8)
+#define FM10K_SW_PCIE_CFG_BAR5 FM10K_SW_PCIE_CFG_REG(0x9)
+#define FM10K_SW_PCIE_CFG_CARDBUS FM10K_SW_PCIE_CFG_REG(0xA)
+#define FM10K_SW_PCIE_CFG_SUBID FM10K_SW_PCIE_CFG_REG(0xB)
+#define FM10K_SW_PCIE_CFG_EXP_ROM FM10K_SW_PCIE_CFG_REG(0xC)
+#define FM10K_SW_PCIE_CFG_CAP_PTR FM10K_SW_PCIE_CFG_REG(0xD)
+#define FM10K_SW_PCIE_CFG_RSVD FM10K_SW_PCIE_CFG_REG(0xE)
+#define FM10K_SW_PCIE_CFG_INT FM10K_SW_PCIE_CFG_REG(0xF)
+#define FM10K_SW_PCIE_CFG_PM_CAP FM10K_SW_PCIE_CFG_REG(0x10)
+#define FM10K_SW_PCIE_CFG_PM_CTRL FM10K_SW_PCIE_CFG_REG(0x11)
+#define FM10K_SW_PCIE_CFG_PCIE_CAP FM10K_SW_PCIE_CFG_REG(0x1C)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP FM10K_SW_PCIE_CFG_REG(0x1D)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL FM10K_SW_PCIE_CFG_REG(0x1E)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CAP FM10K_SW_PCIE_CFG_REG(0x1F)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL FM10K_SW_PCIE_CFG_REG(0x20)
+
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_lsb 16
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_msb 19
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_2P5 1
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_5 2
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_8 3
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_lsb 20
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_msb 24
+
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP2 FM10K_SW_PCIE_CFG_REG(0x25)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL2 FM10K_SW_PCIE_CFG_REG(0x26)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL2 FM10K_SW_PCIE_CFG_REG(0x28)
+#define FM10K_SW_PCIE_CFG_MSIX_CAP FM10K_SW_PCIE_CFG_REG(0x2C)
+#define FM10K_SW_PCIE_CFG_MSIX_TABLE_OFFSET FM10K_SW_PCIE_CFG_REG(0x2D)
+#define FM10K_SW_PCIE_CFG_MSIX_PBA FM10K_SW_PCIE_CFG_REG(0x2E)
+#define FM10K_SW_PCIE_CFG_VPD_CAP FM10K_SW_PCIE_CFG_REG(0x34)
+#define FM10K_SW_PCIE_CFG_VPD_DATA FM10K_SW_PCIE_CFG_REG(0x35)
+#define FM10K_SW_PCIE_CFG_AER_HDR FM10K_SW_PCIE_CFG_REG(0x40)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_STATUS FM10K_SW_PCIE_CFG_REG(0x41)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_MASK FM10K_SW_PCIE_CFG_REG(0x42)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_SEVERITY FM10K_SW_PCIE_CFG_REG(0x43)
+#define FM10K_SW_PCIE_CFG_AER_COERR_STATUS FM10K_SW_PCIE_CFG_REG(0x44)
+#define FM10K_SW_PCIE_CFG_AER_COERR_MASK FM10K_SW_PCIE_CFG_REG(0x45)
+#define FM10K_SW_PCIE_CFG_AER_CTRL FM10K_SW_PCIE_CFG_REG(0x46)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG0 FM10K_SW_PCIE_CFG_REG(0x47)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG1 FM10K_SW_PCIE_CFG_REG(0x48)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG2 FM10K_SW_PCIE_CFG_REG(0x49)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG3 FM10K_SW_PCIE_CFG_REG(0x4A)
+#define FM10K_SW_PCIE_CFG_SPD_HDR FM10K_SW_PCIE_CFG_REG(0x52)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_L FM10K_SW_PCIE_CFG_REG(0x53)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_H FM10K_SW_PCIE_CFG_REG(0x54)
+#define FM10K_SW_PCIE_CFG_ARI_HDR FM10K_SW_PCIE_CFG_REG(0x56)
+#define FM10K_SW_PCIE_CFG_ARI_CTRL FM10K_SW_PCIE_CFG_REG(0x57)
+#define FM10K_SW_PCIE_CFG_SPCIE_HDR FM10K_SW_PCIE_CFG_REG(0x5A)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_CTRL3 FM10K_SW_PCIE_CFG_REG(0x5B)
+#define FM10K_SW_PCIE_CFG_SPCIE_ERR_STS FM10K_SW_PCIE_CFG_REG(0x5C)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ01 FM10K_SW_PCIE_CFG_REG(0x5D)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ23 FM10K_SW_PCIE_CFG_REG(0x5E)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ45 FM10K_SW_PCIE_CFG_REG(0x5F)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ67 FM10K_SW_PCIE_CFG_REG(0x60)
+#define FM10K_SW_PCIE_CFG_SRIOV_HDR FM10K_SW_PCIE_CFG_REG(0x62)
+#define FM10K_SW_PCIE_CFG_SRIOV_CAP FM10K_SW_PCIE_CFG_REG(0x63)
+#define FM10K_SW_PCIE_CFG_SRIOV_CTRL FM10K_SW_PCIE_CFG_REG(0x64)
+#define FM10K_SW_PCIE_CFG_SRIOV_CFG FM10K_SW_PCIE_CFG_REG(0x65)
+#define FM10K_SW_PCIE_CFG_SRIOV_NUM FM10K_SW_PCIE_CFG_REG(0x66)
+#define FM10K_SW_PCIE_CFG_SRIOV_MAP FM10K_SW_PCIE_CFG_REG(0x67)
+#define FM10K_SW_PCIE_CFG_SRIOV_DEVID FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_SUP FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_CFG FM10K_SW_PCIE_CFG_REG(0x6A)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR0 FM10K_SW_PCIE_CFG_REG(0x6B)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR1 FM10K_SW_PCIE_CFG_REG(0x6C)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR2 FM10K_SW_PCIE_CFG_REG(0x6D)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR3 FM10K_SW_PCIE_CFG_REG(0x6E)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR4 FM10K_SW_PCIE_CFG_REG(0x6F)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR5 FM10K_SW_PCIE_CFG_REG(0x70)
+#define FM10K_SW_PCIE_CFG_SRIOV_MIG FM10K_SW_PCIE_CFG_REG(0x71)
+#define FM10K_SW_PCIE_CFG_TPH_HDR FM10K_SW_PCIE_CFG_REG(0x72)
+#define FM10K_SW_PCIE_CFG_TPH_CAP FM10K_SW_PCIE_CFG_REG(0x73)
+#define FM10K_SW_PCIE_CFG_TPH_CTRL FM10K_SW_PCIE_CFG_REG(0x74)
+#define FM10K_SW_PCIE_CFG_ACS_HDR FM10K_SW_PCIE_CFG_REG(0x76)
+#define FM10K_SW_PCIE_CFG_ACS_CAP FM10K_SW_PCIE_CFG_REG(0x77)
+#define FM10K_SW_PCIE_PORTLOGIC FM10K_SW_PCIE_CFG_REG(0x1C0)
+#define FM10K_SW_PCIE_PORTLOGIC_LINK_STATE FM10K_SW_PCIE_CFG_REG(0x1CA)
+
+#define FM10K_SW_FFU_BASE FM10K_SW_REG_OFF(0xC00000)
+#define FM10K_SW_FFU_REG(wo_) \
+ (FM10K_SW_FFU_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_FFU_NUM_SLICES 32
+#define FM10K_SW_FFU_NUM_SCENARIOS 32
+
+/* FFU enumerated types */
+#define FM10K_SW_FFU_MUX_SEL_MAP_DIP_MAP_SIP 0
+#define FM10K_SW_FFU_MUX_SEL_MAP_DMAC_MAP_SMAC 1
+#define FM10K_SW_FFU_MUX_SEL_MAP_PROT_MAP_LENGTH 2
+#define FM10K_SW_FFU_MUX_SEL_MAP_SRC_MAP_TYPE 3
+#define FM10K_SW_FFU_MUX_SEL_USER 4
+#define FM10K_SW_FFU_MUX_SEL_FTYPE_SWPRI 5
+#define FM10K_SW_FFU_MUX_SEL_IPMISC 6
+#define FM10K_SW_FFU_MUX_SEL_TOS 7
+#define FM10K_SW_FFU_MUX_SEL_PROT 8
+#define FM10K_SW_FFU_MUX_SEL_TTL 9
+#define FM10K_SW_FFU_MUX_SEL_SRC_PORT 10
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID_11_8 11
+#define FM10K_SW_FFU_MUX_SEL_VID_7_0 12
+#define FM10K_SW_FFU_MUX_SEL_RXTAG 13
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_15_0 14
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_31_16 15
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_47_32 16
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_15_0 17
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_31_16 18
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_47_32 19
+#define FM10K_SW_FFU_MUX_SEL_DGLORT 20
+#define FM10K_SW_FFU_MUX_SEL_SGLORT 21
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID 22
+#define FM10K_SW_FFU_MUX_SEL_VPRI2_VID2 23
+#define FM10K_SW_FFU_MUX_SEL_L2_TYPE 24
+#define FM10K_SW_FFU_MUX_SEL_L4_DST 25
+#define FM10K_SW_FFU_MUX_SEL_L4_SRC 26
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_DST 27
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_SRC 28
+#define FM10K_SW_FFU_MUX_SEL_L4A 29
+#define FM10K_SW_FFU_MUX_SEL_L4B 30
+#define FM10K_SW_FFU_MUX_SEL_L4C 31
+#define FM10K_SW_FFU_MUX_SEL_L4D 32
+#define FM10K_SW_FFU_MUX_SEL_MAP_VPRI1_VID1 33
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_31_0 34
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_63_32 35
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_95_64 36
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_127_96 37
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_31_0 38
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_63_32 39
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64 40
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96 41
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_msb 19
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_TYPE_EXP (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_FLOOD_SET (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_msb 7
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_lsb 8
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_msb 20
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_msb 11
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_msb 17
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_PRI (1 << 19)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_msb 5
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_DSCP (1 << 17)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_PRI (1 << 19)
+
+/* FFU Registers */
+#define FM10K_SW_FFU_SLICE_TCAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x2000 * (sl_) + 0x4 * (n_))
+#define FM10K_SW_FFU_SLICE_TCAM_ENTRIES 1024
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_lsb64 0
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_msb64 31
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_lsb64 32
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_msb64 39
+#define FM10K_SW_FFU_SLICE_SRAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x1000 + 0x2000 * (sl_) + 0x2 * (n_))
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_lsb64 21
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_msb64 22
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_ARP 0
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT 1
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_BIT_SET 2
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET 3
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_lsb64 23
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_msb64 34
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_lsb64 35
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_msb64 36
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_lsb64 37
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_msb64 39
+#define FM10K_SW_FFU_SLICE_VALID(sl_) \
+ FM10K_SW_FFU_REG(0x1800 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_VALID_SCENARIO(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_FFU_SLICE_CASCADE_ACTION(sl_) \
+ FM10K_SW_FFU_REG(0x1804 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG(sl_, scen_) \
+ FM10K_SW_FFU_REG(0x1840 + 0x2 * (scen_) + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_lsb64 0
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_msb64 5
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_lsb64 6
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_msb64 11
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_lsb64 12
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_msb64 17
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_lsb64 18
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_msb64 23
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_lsb64 24
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_msb64 29
+#define FM10K_SW_FFU_SLICE_CFG_START_COMPARE (1ULL << 30)
+#define FM10K_SW_FFU_SLICE_CFG_START_ACTION (1ULL << 31)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_LOW (1ULL << 32)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_HIGH (1ULL << 33)
+#define FM10K_SW_FFU_SLICE_CFG_CASE_lsb64 34
+#define FM10K_SW_FFU_SLICE_CFG_CASE_msb64 37
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_lsb64 38
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_msb64 39
+
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_NOT_MAPPED 0
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_LOW_NIBBLE 1
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_HIGH_NIBBLE 2
+
+#define FM10K_SW_FFU_MASTER_VALID \
+ FM10K_SW_FFU_REG(0x40000)
+#define FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_SLICES_VALID FM10K_SW_MASK64(31, 0)
+#define FM10K_SW_FFU_MASTER_VALID_CHUNK_VALID(c_) (1ULL << ((c_) + 32))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_CHUNKS_VALID FM10K_SW_MASK64(63, 32)
+
+#define FM10K_SW_L2LOOKUP_BASE \
+ FM10K_SW_REG_OFF(0xC80000)
+#define FM10K_SW_L2LOOKUP_REG(wo_) \
+ (FM10K_SW_L2LOOKUP_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_MA_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x10000 * (t_) + 0x4 * (n_))
+#define FM10K_SW_INGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x20000 + 0x4 * (v_))
+#define FM10K_SW_INGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative
+ * to the start of the upper 64-bit word
+ */
+#define FM10K_SW_INGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_msb64 (89 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_REFLECT (1ULL << (90 - 64))
+#define FM10K_SW_INGRESS_VID_TABLE_TRAP_IGMP (1ULL << (91 - 64))
+#define FM10K_SW_EGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x24000 + 0x4 * (v_))
+#define FM10K_SW_EGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative to
+ * the start of the upper 64-bit word
+ */
+#define FM10K_SW_EGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_msb64 (86 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_lsb64 (87 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_msb64 (92 - 64)
+
+#define FM10K_SW_MA_USED_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28000 + 0x200 * (t_) + (n_))
+#define FM10K_SW_INGRESS_MST_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28400 + 0x200 * (t_) + 0x2 * (n_))
+
+#define FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE 24
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) \
+ (((l_) % FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) * 2)
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_msb64(l_) \
+ (FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) + 1)
+
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_DISABLE 0
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LISTENING 1
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LEARNING 2
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD 3
+
+#define FM10K_SW_EGRESS_MST_TABLE(t_) \
+ FM10K_SW_L2LOOKUP_REG(0x28800 + 0x2 * (t_))
+#define FM10K_SW_EGRESS_MST_TABLE_FORWARDING(l_) (1ULL << (l_))
+
+#define FM10K_SW_MA_TABLE_CFG_1 \
+ FM10K_SW_L2LOOKUP_REG(0x28A00)
+#define FM10K_SW_MA_TABLE_CFG_2 \
+ FM10K_SW_L2LOOKUP_REG(0x28A02)
+#define FM10K_SW_MTU_TABLE(n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28A08 + (n_))
+#define FM10K_SW_IEEE_RESERVED_MAC_ACTION \
+ FM10K_SW_L2LOOKUP_REG(0x28A10)
+#define FM10K_SW_IEEE_RESERVED_MAC_TRAP_PRIORITY \
+ FM10K_SW_L2LOOKUP_REG(0x28A14)
+#define FM10K_SW_IEEE_RESERVED_MAC_CFG \
+ FM10K_SW_L2LOOKUP_REG(0x28A16)
+
+
+#define FM10K_SW_GLORT_BASE FM10K_SW_REG_OFF(0xCE0000)
+#define FM10K_SW_GLORT_REG(wo_) \
+ (FM10K_SW_GLORT_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_GLORT_CAM_ENTRIES 256
+#define FM10K_SW_GLORT_DEST_TABLE(n_) \
+ FM10K_SW_GLORT_REG(0x0 + 0x2 * (n_))
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_lsb64 0
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_msb64 47
+
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_lsb64 48
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_msb64 59
+
+#define FM10K_SW_GLORT_CAM(n_) \
+ FM10K_SW_GLORT_REG(0x2000 + (n_))
+#define FM10K_SW_GLORT_CAM_MATCH_ANY 0x00000000
+#define FM10K_SW_GLORT_CAM_MATCH_NONE FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_GLORT_CAM_KEY_lsb 0
+#define FM10K_SW_GLORT_CAM_KEY_msb 15
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_lsb 16
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_msb 31
+#define FM10K_SW_GLORT_RAM(n_) \
+ FM10K_SW_GLORT_REG(0x2200 + 0x2 * (n_))
+#define FM10K_SW_GLORT_RAM_STRICT_lsb64 0
+#define FM10K_SW_GLORT_RAM_STRICT_msb64 1
+#define FM10K_SW_GLORT_RAM_STRICT_FTYPE 0
+#define FM10K_SW_GLORT_RAM_STRICT_RSVD 1
+#define FM10K_SW_GLORT_RAM_STRICT_HASHED 2
+#define FM10K_SW_GLORT_RAM_STRICT_STRICT 3
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_lsb64 2
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_msb64 13
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_lsb64 14
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_msb64 21
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_lsb64 22
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_msb64 29
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_lsb64 30
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_msb64 33
+#define FM10K_SW_GLORT_RAM_HASH_ROTATION (1ULL << 34)
+
+
+#define FM10K_SW_PARSER_BASE FM10K_SW_REG_OFF(0xCF0000)
+#define FM10K_SW_PARSER_REG(wo_) \
+ (FM10K_SW_PARSER_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_PARSER_PORT_CFG_1(p_) \
+ FM10K_SW_PARSER_REG(0x0 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_1_FTAG (1ULL << 0)
+
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(v_) (1ULL << (1 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(v_) (1ULL << (5 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_FIRST (1ULL << 9)
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_lsb 10
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_msb 21
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_lsb 22
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_lsb 26
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_msb 37
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_lsb 38
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_msb 41
+#define FM10K_SW_PARSER_PORT_CFG_1_USE_DEFAULT_VLAN (1ULL << 42)
+#define FM10K_SW_PARSER_PORT_CFG_2(p_) \
+ FM10K_SW_PARSER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_lsb 0
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_msb 3
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_lsb 4
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_msb 7
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_MPLS (1ULL << 8)
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_lsb 9
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_msb 11
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3 (1ULL << 12)
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4 (1ULL << 13)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV4_OPTIONS (1ULL << 14)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_HOP_BY_HOP (1ULL << 15)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_ROUTING (1ULL << 16)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_FRAG (1ULL << 17)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_DEST (1ULL << 18)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_AUTH (1ULL << 19)
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_lsb 20
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_TAGGED (1ULL << 26)
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_UNTAGGED (1ULL << 27)
+#define FM10K_SW_PARSER_PORT_CFG_2_USE_DEFAULT_DSCP (1ULL << 28)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_VLAN (1ULL << 29)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_DSCP (1ULL << 30)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_ISL (1ULL << 31)
+
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_PREFER_DSCP (1ULL << 32)
+
+#define FM10K_SW_PARSER_PORT_CFG_3(p_) \
+ FM10K_SW_PARSER_REG(0x100 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_ISL(p_) \
+ FM10K_SW_PARSER_REG(0x180 + (p_))
+
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_lsb 0
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_msb 15
+#define FM10K_SW_PORT_CFG_ISL_USR_lsb 16
+#define FM10K_SW_PORT_CFG_ISL_USR_msb 23
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_lsb 24
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_msb 27
+
+#define FM10K_SW_PARSER_VLAN_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C0 + (n_))
+#define FM10K_SW_PARSER_CUSTOM_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C4 + (n_))
+#define FM10K_SW_PARSER_MPLS_TAG \
+ FM10K_SW_PARSER_REG(0x1C8)
+#define FM10K_SW_PARSER_DI_CFG(n_) \
+ FM10K_SW_PARSER_REG(0x1D0 + 0x2 * (n_))
+#define FM10K_SW_RX_VPRI_MAP(p_) \
+ FM10K_SW_PARSER_REG(0x200 + 0x2 * (p_))
+#define FM10K_SW_DSCP_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x280 + (pri_))
+#define FM10K_SW_VPRI_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x2C0 + (pri_))
+
+#define FM10K_SW_HANDLER_BASE FM10K_SW_REG_OFF(0xD50000)
+#define FM10K_SW_HANDLER_REG(wo_) \
+ (FM10K_SW_HANDLER_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_SYS_CFG_1 \
+ FM10K_SW_HANDLER_REG(0x0)
+#define FM10K_SW_SYS_CFG_1_DROP_PAUSE (1 << 0)
+#define FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS (1 << 1)
+#define FM10K_SW_SYS_CFG_1_ENABLE_TRAP_PLUS_LOG (1 << 2)
+#define FM10K_SW_SYS_CFG_1_DROP_INVALID_SMAC (1 << 3)
+#define FM10K_SW_SYS_CFG_1_DROP_MAC_CTRL_ETHERTYPE (1 << 4)
+
+#define FM10K_SW_CPU_MAC FM10K_SW_HANDLER_REG(0x2)
+#define FM10K_SW_SYS_CFG_ROUTER FM10K_SW_HANDLER_REG(0x4)
+#define FM10K_SW_L34_HASH_CFG FM10K_SW_HANDLER_REG(0x5)
+#define FM10K_SW_L34_HASH_CFG_SYMMETRIC (1 << 0)
+#define FM10K_SW_L34_HASH_CFG_USE_SIP (1 << 1)
+#define FM10K_SW_L34_HASH_CFG_USE_DIP (1 << 2)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT (1 << 3)
+#define FM10K_SW_L34_HASH_CFG_USE_TCP (1 << 4)
+#define FM10K_SW_L34_HASH_CFG_USE_UDP (1 << 5)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT1 (1 << 6)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT2 (1 << 7)
+#define FM10K_SW_L34_HASH_CFG_USE_L4SRC (1 << 8)
+#define FM10K_SW_L34_HASH_CFG_USE_L4DST (1 << 9)
+
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_lsb 10
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_msb 11
+#define FM10K_SW_L34_HASH_CFG_PROT1_lsb 16
+#define FM10K_SW_L34_HASH_CFG_PROT1_msb 23
+#define FM10K_SW_L34_HASH_CFG_PROT2_lsb 24
+#define FM10K_SW_L34_HASH_CFG_PROT2_msb 31
+
+#define FM10K_SW_L34_FLOW_HASH_CFG_1 FM10K_SW_HANDLER_REG(0x6)
+#define FM10K_SW_L34_FLOW_HASH_CFG_2 FM10K_SW_HANDLER_REG(0x7)
+#define FM10K_SW_L234_HASH_CFG FM10K_SW_HANDLER_REG(0x8)
+#define FM10K_SW_L234_HASH_CFG_USE_L2_IF_IP (1 << 0)
+#define FM10K_SW_L234_HASH_CFG_USE_L34 (1 << 1)
+#define FM10K_SW_L234_HASH_CFG_SYMMETRIC (1 << 2)
+#define FM10K_SW_L234_HASH_CFG_USE_DMAC (1 << 3)
+#define FM10K_SW_L234_HASH_CFG_USE_SMAC (1 << 4)
+#define FM10K_SW_L234_HASH_CFG_USE_TYPE (1 << 5)
+#define FM10K_SW_L234_HASH_CFG_USE_VPRI (1 << 6)
+#define FM10K_SW_L234_HASH_CFG_USE_VID (1 << 8)
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_lsb 8
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_msb 9
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_lsb 10
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_msb 11
+#define FM10K_SW_CPU_TRAP_MASK_FH FM10K_SW_HANDLER_REG(0xA)
+#define FM10K_SW_TRAP_GLORT FM10K_SW_HANDLER_REG(0xC)
+#define FM10K_SW_RX_MIRROR_CFG FM10K_SW_HANDLER_REG(0xD)
+#define FM10K_SW_LOG_MIRROR_PROFILE FM10K_SW_HANDLER_REG(0xE)
+
+#define FM10K_SW_FH_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_HANDLER_REG(0x40 + (n_))
+#define FM10K_SW_PORT_CFG_2(p_) \
+ FM10K_SW_HANDLER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_3(p_) \
+ FM10K_SW_HANDLER_REG(0x100 + (p_))
+#define FM10K_SW_FH_LOOPBACK_SUPPRESS(p_) \
+ FM10K_SW_HANDLER_REG(0x140 + (p_))
+#define FM10K_SW_FH_HEAD_IP FM10K_SW_HANDLER_REG(0x180)
+#define FM10K_SW_FH_HEAD_IM FM10K_SW_HANDLER_REG(0x182)
+#define FM10K_SW_PARSER_EARLY_SRAM_CTRL FM10K_SW_HANDLER_REG(0x184)
+#define FM10K_SW_PARSER_LATE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x185)
+#define FM10K_SW_MAPPER_SRAM_CTRL FM10K_SW_HANDLER_REG(0x186)
+#define FM10K_SW_FFU_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1A0 + 0x4 * (n_))
+#define FM10K_SW_ARP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C0)
+#define FM10K_SW_VLAN_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C1)
+#define FM10K_SW_MA_TABLE_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1C4 + 0x2 * (n_))
+#define FM10K_SW_FID_GLORT_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C8)
+#define FM10K_SW_GLORT_RAM_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CA)
+#define FM10K_SW_GLORT_TABLE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CB)
+#define FM10K_SW_FH_HEAD_OUTPUT_FIFO_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CC)
+
+#define FM10K_SW_LAG_BASE FM10K_SW_REG_OFF(0xD90000)
+#define FM10K_SW_LAG_REG(wo_) \
+ (FM10K_SW_LAG_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_LAG_CFG(l_) FM10K_SW_LAG_REG(l_)
+#define FM10K_SW_LAG_CFG_LAG_SIZE_lsb 0
+#define FM10K_SW_LAG_CFG_LAG_SIZE_msb 3
+#define FM10K_SW_LAG_CFG_INDEX_lsb 4
+#define FM10K_SW_LAG_CFG_INDEX_msb 7
+#define FM10K_SW_LAG_CFG_HASH_ROTATION (1 << 8)
+#define FM10K_SW_LAG_CFG_IN_LAG (1 << 9)
+
+#define FM10K_SW_CANONICAL_GLORT_CAM(n_) \
+ FM10K_SW_LAG_REG((n_) + 0x40)
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_lsb 0
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_msb 15
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_lsb 16
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_msb 19
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_lsb 20
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_msb 22
+
+#define FM10K_SW_RX_STATS_BASE \
+ FM10K_SW_REG_OFF(0xE00000)
+#define FM10K_SW_RX_STATS_REG(wo_) \
+ (FM10K_SW_RX_STATS_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_RX_STATS_BANK(b_, i_) \
+ FM10K_SW_RX_STATS_REG(0x1000 * (b_) + 0x4 * (i_))
+#define FM10K_SW_RX_STATS_BANK_1_INDEX 0
+#define FM10K_SW_RX_STATS_BANK_2_INDEX 1
+#define FM10K_SW_RX_STATS_BANK_3_INDEX 2
+#define FM10K_SW_RX_STATS_BANK_4_INDEX 3
+#define FM10K_SW_RX_STATS_BANK_5_INDEX 4
+#define FM10K_SW_RX_STATS_BANK_6_INDEX 5
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_UCAST 0
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_MCAST 1
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_BCAST 2
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_UCAST 3
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_MCAST 4
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_BCAST 5
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_UCAST 6
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_MCAST 7
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_BCAST 8
+#define FM10K_SW_RX_STATS_BANK_1_IEEE802_3_PAUSE 9
+#define FM10K_SW_RX_STATS_BANK_1_CLASS_BASED_PAUSE 10
+#define FM10K_SW_RX_STATS_BANK_1_FRAMING_ERR 11
+#define FM10K_SW_RX_STATS_BANK_1_FCS_ERR 12
+#define FM10K_SW_RX_STATS_BANK_2_LEN_LT_64 0
+#define FM10K_SW_RX_STATS_BANK_2_LEN_EQ_64 1
+#define FM10K_SW_RX_STATS_BANK_2_LEN_65_127 2
+#define FM10K_SW_RX_STATS_BANK_2_LEN_128_255 3
+#define FM10K_SW_RX_STATS_BANK_2_LEN_256_511 4
+#define FM10K_SW_RX_STATS_BANK_2_LEN_512_1023 5
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1024_1522 6
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1523_2047 7
+#define FM10K_SW_RX_STATS_BANK_2_LEN_2048_4095 8
+#define FM10K_SW_RX_STATS_BANK_2_LEN_4096_8191 9
+#define FM10K_SW_RX_STATS_BANK_2_LEN_8192_10239 10
+#define FM10K_SW_RX_STATS_BANK_2_LEN_GE_10240 11
+#define FM10K_SW_RX_STATS_BANK_3_PRI(p_) (p_)
+#define FM10K_SW_RX_STATS_BANK_4_FID_FORWARDED 0
+#define FM10K_SW_RX_STATS_BANK_4_FLOOD_FORWARDED 1
+#define FM10K_SW_RX_STATS_BANK_4_SPECIALLY_HANDLED 2
+#define FM10K_SW_RX_STATS_BANK_4_PARSER_ERROR_DROP 3
+#define FM10K_SW_RX_STATS_BANK_4_ECC_ERROR_DROP 4
+#define FM10K_SW_RX_STATS_BANK_4_TRAPPED 5
+#define FM10K_SW_RX_STATS_BANK_4_PAUSE_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_4_STP_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_4_SECURITY_VIOLATIONS 8
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_TAG_DROPS 9
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_INGRESS_DROPS 10
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_EGRESS_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_4_GLORT_MISS_DROPS 12
+#define FM10K_SW_RX_STATS_BANK_4_FFU_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_4_TRIGGER_DROPS 14
+#define FM10K_SW_RX_STATS_BANK_5_POLICER_DROPS 0
+#define FM10K_SW_RX_STATS_BANK_5_TTL_DROPS 1
+#define FM10K_SW_RX_STATS_BANK_5_CM_PRIV_DROPS 2
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP0_DROPS 3
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP1_DROPS 4
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG0_DROPS 5
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG1_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG0_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG1_DROPS 8
+#define FM10K_SW_RX_STATS_BANK_5_TRIGGER_REDIRECTS 10
+#define FM10K_SW_RX_STATS_BANK_5_FLOOD_CONTROL_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_5_GLORT_FORWARDED 12
+#define FM10K_SW_RX_STATS_BANK_5_LOOPBACK_SUPP_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_5_OTHER_DROPS 14
+#define FM10K_SW_RX_PORT_STAT(b_, s_, p_, f_) \
+ (FM10K_SW_RX_STATS_BANK(FM10K_SW_RX_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_RX_STATS_##b_##_##s_) + \
+ ((f_) ? 0 : FM10K_SW_REG_OFF(2)))
+
+#define FM10K_SW_RX_STATS_CFG(p_) \
+ FM10K_SW_RX_STATS_REG(0x10000 + (p_))
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_lsb 0
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_msb 7
+#define FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS 0x00003f00
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_1 (1 << 8)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_2 (1 << 9)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_3 (1 << 10)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_4 (1 << 11)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_5 (1 << 12)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_6 (1 << 13)
+#define FM10K_SW_RX_STATS_CFG_SWITCH_PRI (1 << 14)
+
+#define FM10K_SW_HANDLER_TAIL_BASE \
+ FM10K_SW_REG_OFF(0xE30000)
+#define FM10K_SW_HANDLER_TAIL_REG(wo_) \
+ (FM10K_SW_HANDLER_TAIL_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_SAF_MATRIX(l_) \
+ FM10K_SW_HANDLER_TAIL_REG(0x2 * (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF(l_) (1ULL << (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS FM10K_SW_MASK64(47, 0)
+#define FM10K_SW_SAF_MATRIX_IGNORE_ERROR (1ULL << 50)
+
+#define FM10K_SW_FRAME_TIME_OUT FM10K_SW_HANDLER_TAIL_REG(0x80)
+#define FM10K_SW_SAF_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x81)
+#define FM10K_SW_EGRESS_PAUSE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x82)
+#define FM10K_SW_RX_STATS_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x84)
+#define FM10K_SW_POLICER_USAGE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x88)
+#define FM10K_SW_TCN_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x8C)
+#define FM10K_SW_FH_TAIL_IP FM10K_SW_HANDLER_TAIL_REG(0x8D)
+#define FM10K_SW_FH_TAIL_IM FM10K_SW_HANDLER_TAIL_REG(0x8E)
+#define FM10K_SW_TAIL_PERMIT_MGMT FM10K_SW_HANDLER_TAIL_REG(0x8F)
+#define FM10K_SW_TAIL_FORCE_IDLE FM10K_SW_HANDLER_TAIL_REG(0x90)
+
+
+#define FM10K_SW_CM_USAGE_BASE FM10K_SW_REG_OFF(0xE60000)
+#define FM10K_SW_CM_USAGE_REG(wo_) \
+ (FM10K_SW_CM_USAGE_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_CM_SWEEPER_SWITCH_PRI_TO_TC FM10K_SW_CM_USAGE_REG(0x0)
+#define FM10K_SW_CM_SWEEPER_TC_TO_SMP FM10K_SW_CM_USAGE_REG(0x2)
+#define FM10K_SW_CM_TX_TC_PRIVATE_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x200 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_TX_TC_HOG_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x400 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_PAUSE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x600 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_PRIVATE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x680 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_HOG_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x700 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_PAUSE_RESEND_INTERVAL(l_) \
+ FM10K_SW_CM_USAGE_REG(0x780 + (l_))
+#define FM10K_SW_CM_PAUSE_BASE_FREQ \
+ FM10K_SW_CM_USAGE_REG(0x7C0)
+#define FM10K_SW_CM_PAUSE_CFG(l_) \
+ FM10K_SW_CM_USAGE_REG(0x800 + (l_))
+#define FM10K_SW_CM_SHARED_WM(p_) \
+ FM10K_SW_CM_USAGE_REG(0x840 + (p_))
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_WM(s_) \
+ FM10K_SW_CM_USAGE_REG(0x850 + (s_))
+#define FM10K_SW_CM_GLOBAL_WM FM10K_SW_CM_USAGE_REG(0x852)
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_lsb 0
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_msb 14
+#define FM10K_SW_CM_GLOBAL_CFG FM10K_SW_CM_USAGE_REG(0x853)
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_lsb 0
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_msb 7
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_ON (1 << 8)
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_OFF (1 << 9)
+#define FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN (1 << 10)
+
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN (1 << 11)
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN (1 << 12)
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_lsb 13
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_msb 18
+
+#define FM10K_SW_CM_TC_PC_MAP(l_) FM10K_SW_CM_USAGE_REG(0x880 + (l_))
+#define FM10K_SW_CM_PC_SMP_MAP(l_) FM10K_SW_CM_USAGE_REG(0x8C0 + (l_))
+#define FM10K_SW_CM_SOFTDROP_WM(p_) FM10K_SW_CM_USAGE_REG(0x900 + (p_))
+
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_CFG(s_) \
+ FM10K_SW_CM_USAGE_REG(0x910 + 0x2 * (s_))
+#define FM10K_SW_TX_RATE_LIM_CFG(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xA00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_TX_RATE_LIM_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xC00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_BSG_MAP(l_) \
+ FM10K_SW_CM_USAGE_REG(0xE00 + (l_))
+#define FM10K_SW_CM_TX_TC_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x1000 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_USAGE(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x1200 + 0x2 * (l_) + (s_))
+#define FM10K_SW_MCAST_EPOCH_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1280 + (s_))
+#define FM10K_SW_CM_SHARED_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1282 + (s_))
+#define FM10K_SW_CM_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1284 + (s_))
+#define FM10K_SW_CM_GLOBAL_USAGE FM10K_SW_CM_USAGE_REG(0x1286)
+
+#define FM10K_SW_CM_PAUSE_GEN_STATE(l_) \
+ FM10K_SW_CM_USAGE_REG(0x12C0 + (l_))
+#define FM10K_SW_CM_PAUSE_RCV_TIMER FM10K_SW_CM_USAGE_REG(0x1300)
+#define FM10K_SW_CM_PAUSE_RCV_PORT_TIMER(l_) \
+ FM10K_SW_CM_USAGE_REG(0x1340 + (l_))
+
+/* FM10K_SW_CM_EGRESS_PAUSE_COUNT is also
+ * known as FM10K_SW_CM_PAUSE_RCV_STATE
+ */
+#define FM10K_SW_CM_EGRESS_PAUSE_COUNT(l_, dw_) \
+ FM10K_SW_CM_USAGE_REG(0x1400 + 0x4 * (l_) + 0x2 * (dw_))
+
+#define FM10K_SW_MOD_BASE FM10K_SW_REG_OFF(0xE80000)
+#define FM10K_SW_MOD_REG(wo_) \
+ (FM10K_SW_MOD_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_MOD_MAX_MGMT_WAIT_CYCLE FM10K_SW_MOD_REG(0xF8)
+#define FM10K_SW_MOD_IP FM10K_SW_MOD_REG(0x106)
+#define FM10K_SW_MOD_IM FM10K_SW_MOD_REG(0x108)
+#define FM10K_SW_MOD_SRAM_BIST_OUT FM10K_SW_MOD_REG(0x10A)
+#define FM10K_SW_MOD_SRAM_ERROR_WRITE FM10K_SW_MOD_REG(0x10C)
+#define FM10K_SW_MOD_PAUSE_SMAC FM10K_SW_MOD_REG(0x10E)
+#define FM10K_SW_MOD_ROUTER_SMAC(n_) \
+ FM10K_SW_MOD_REG(0x120 + 0x2 * (n_))
+#define FM10K_SW_MOD_MCAST_VLAN_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x10000 + 0x2 * (n_))
+#define FM10K_SW_MOD_VLAN_TAG_VID1_MAP(v_) \
+ FM10K_SW_MOD_REG(0x20000 + 0x2 * (v_))
+#define FM10K_SW_MOD_VID2_MAP(v_) \
+ FM10K_SW_MOD_REG(0x22000 + 0x2 * (v_))
+#define FM10K_SW_MOD_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x24000 + 0x2 * (n_))
+#define FM10K_SW_MOD_PER_PORT_CFG_1(p_) \
+ FM10K_SW_MOD_REG(0x24080 + 0x2 * (p_))
+#define FM10K_SW_MOD_PER_PORT_CFG_2(p_) \
+ FM10K_SW_MOD_REG(0x24100 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_lsb64 0
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_msb64 5
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP1_UPDATE (1ULL << 6)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP2_UPDATE (1ULL << 7)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI1_UPDATE (1ULL << 8)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI2_UPDATE (1ULL << 9)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_lsb64 10
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_msb64 11
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_lsb64 12
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_msb64 13
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DMAC_ROUTING (1ULL << 14)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_SMAC_ROUTING (1ULL << 15)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_TTL_DECREMENT (1ULL << 16)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DSCP_MODIFICATION (1ULL << 17)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_FTAG (1ULL << 18)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VID2_FIRST (1ULL << 19)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_lsb64 20
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_msb64 22
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_lsb64 23
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_msb64 30
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_TYPE (1ULL << 31)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_lsb64 32
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_msb64 47
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIN_FRAME_SIZE (1ULL << 48)
+
+#define FM10K_SW_MOD_VPRI1_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24180 + 0x2 * (p_))
+#define FM10K_SW_MOD_VPRI2_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24200 + 0x2 * (p_))
+#define FM10K_SW_MOD_VLAN_ETYPE(n_) \
+ FM10K_SW_MOD_REG(0x24280 + 0x2 * (n_))
+#define FM10K_SW_MOD_STATS_CFG(p_) \
+ FM10K_SW_MOD_REG(0x24300 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 (1 << 0)
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8 (1 << 1)
+#define FM10K_SW_MOD_STATS_BANK_FRAME(b_, i_) \
+ FM10K_SW_MOD_REG(0x25000 + 0x800 * (b_) + 0x2 * (i_))
+#define FM10K_SW_MOD_STATS_BANK_BYTE(b_, i_) \
+ FM10K_SW_MOD_REG(0x26000 + 0x800 * (b_) + 0x2 * (i_))
+
+#define FM10K_SW_MOD_STATS_BANK_7_INDEX 0
+#define FM10K_SW_MOD_STATS_BANK_8_INDEX 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_UCAST 0
+#define FM10K_SW_MOD_STATS_BANK_7_L2_MCAST 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_BCAST 2
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR 3
+#define FM10K_SW_MOD_STATS_BANK_7_TIMEOUT_DROP 4
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR_DROP 5
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ECC_DROP 6
+#define FM10K_SW_MOD_STATS_BANK_7_LOOPBACK_DROP 7
+#define FM10K_SW_MOD_STATS_BANK_7_TTL1_DROP 8
+#define FM10K_SW_MOD_STATS_BANK_7_PAUSE 9
+#define FM10K_SW_MOD_STATS_BANK_7_CB_PAUSE 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_LT_64 0
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_EQ_64 1
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_65_127 2
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_128_255 3
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_256_511 4
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_512_1023 5
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1024_1522 6
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1523_2047 7
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_2048_4095 8
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_4096_8191 9
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_8192_10239 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_GE_10240 11
+#define FM10K_SW_TX_PORT_STAT(b_, s_, p_, f_, p1_) \
+ ((f_) ? \
+ FM10K_SW_MOD_STATS_BANK_FRAME(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_MOD_STATS_##b_##_##s_) : \
+ FM10K_SW_MOD_STATS_BANK_BYTE(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p1_) + FM10K_SW_MOD_STATS_##b_##_##s_))
+
+#define FM10K_SW_SCHED_BASE FM10K_SW_REG_OFF(0xF00000)
+#define FM10K_SW_SCHED_REG(wo_) \
+ (FM10K_SW_SCHED_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * Schedule fields for both FM10K_SW_SCHED_RX_SCHEDULE and
+ * FM10K_SW_SCHED_TX_SCHEDULE
+ */
+#define FM10K_SW_SCHED_SCHEDULE_ENTRY(p_, l_, q_) \
+ (FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_PHYS_PORT, (p_)) | \
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_LOG_PORT, (l_)) | \
+ ((q_) ? FM10K_SW_SCHED_SCHEDULE_QUAD : 0))
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_lsb 0
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_msb 7
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_lsb 8
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_msb 13
+#define FM10K_SW_SCHED_SCHEDULE_QUAD (1 << 14)
+#define FM10K_SW_SCHED_SCHEDULE_COLOR (1 << 15)
+#define FM10K_SW_SCHED_SCHEDULE_IDLE (1 << 16)
+
+#define FM10K_SW_SCHED_RX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20000 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_TX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20400 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_SCHEDULE_CTRL \
+ FM10K_SW_SCHED_REG(0x20800)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE (1 << 0)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_PAGE (1 << 1)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_lsb 2
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_msb 10
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE (1 << 11)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_PAGE (1 << 12)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_lsb 13
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_msb 21
+#define FM10K_SW_SCHED_CONFIG_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x20801)
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x20840 + (l_))
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_lsb 0
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_msb 15
+#define FM10K_SW_SCHED_SSCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30050)
+#define FM10K_SW_SCHED_ESCHED_CFG_1(l_) \
+ FM10K_SW_SCHED_REG(0x30080 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_2(l_) \
+ FM10K_SW_SCHED_REG(0x300C0 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_3(l_) \
+ FM10K_SW_SCHED_REG(0x30100 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_ESCHED \
+ FM10K_SW_SCHED_REG(0x30240)
+#define FM10K_SW_SCHED_ESCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30242)
+#define FM10K_SW_SCHED_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x30244)
+#define FM10K_SW_SCHED_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_FREELIST_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30246)
+#define FM10K_SW_SCHED_MONITOR_DRR_Q_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x30400 + (q_))
+#define FM10K_SW_SCHED_MONITOR_DRR_CFG_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x30600 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_MONITOR \
+ FM10K_SW_SCHED_REG(0x30A40)
+#define FM10K_SW_SCHED_MONITOR_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30A41)
+#define FM10K_SW_SCHED_MCAST_LEN_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x34000 + (n_))
+#define FM10K_SW_SCHED_MCAST_DEST_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x38000 + 0x2 * (n_))
+#define FM10K_SW_SCHED_MCAST_LIMITED_SKEW_MULTICAST \
+ FM10K_SW_SCHED_REG(0x3A000)
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(n_) \
+ FM10K_SW_SCHED_REG(0x60400 + 0x2 * (n_))
+
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES 8
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_lsb 0
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_msb 9
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_lsb 10
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_msb 19
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_lsb 20
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_msb 24
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_lsb 25
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_msb 29
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_lsb 30
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_msb 39
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x60410)
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES 1024
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_msb 9
+
+#define FM10K_SW_SCHED_MGMT_TIMER_RXQ \
+ FM10K_SW_SCHED_REG(0x60412)
+#define FM10K_SW_SCHED_MGMT_TIMER_LG \
+ FM10K_SW_SCHED_REG(0x60413)
+#define FM10K_SW_SCHED_RXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x60414)
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60600 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60800 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60A00 + (q_))
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES 384
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_lsb 0
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_msb 15
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x62000)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_TXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62002)
+#define FM10K_SW_SCHED_FIFO_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62004)
+#define FM10K_SW_SCHED_IP \
+ FM10K_SW_SCHED_REG(0x62006)
+#define FM10K_SW_SCHED_IM \
+ FM10K_SW_SCHED_REG(0x62007)
+
+#endif /* _FM10K_REGS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.c b/drivers/net/fm10k/switch/fm10k_sbus.c
new file mode 100644
index 0000000..d7d656e
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.c
@@ -0,0 +1,292 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+#define FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX 4
+#define FM10K_SW_SBUS_COMMAND_TIMEOUT_US 500000
+#define FM10K_SW_SBUS_RING_DIVIDER 0x01
+
+static int fm10k_sbus_init(struct fm10k_sbus *);
+static int fm10k_sbus_exec(struct fm10k_sbus *, struct fm10k_sbus_req *);
+static int fm10k_sbus_reset_all(struct fm10k_sbus *);
+static int fm10k_sbus_sbm_reset(struct fm10k_sbus *);
+
+struct fm10k_sbus *
+fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name,
+ unsigned int cfg_reg)
+{
+ struct fm10k_sbus *sb;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: attaching", name);
+
+ sb = (struct fm10k_sbus *)rte_zmalloc
+ ("fm10k_sbus", sizeof(struct fm10k_sbus), 0);
+ if (sb == NULL) {
+ FM10K_SW_INFO("sbus %s: failed to allocate context", name);
+ goto fail;
+ }
+
+ sb->sw = sw;
+ sb->name = name;
+ pthread_mutex_init(&sb->lock, NULL);
+
+ sb->cfg_reg = cfg_reg;
+ sb->cmd_reg = cfg_reg + FM10K_SW_REG_OFF(1);
+ sb->req_reg = cfg_reg + FM10K_SW_REG_OFF(2);
+ sb->resp_reg = cfg_reg + FM10K_SW_REG_OFF(3);
+
+ error = fm10k_sbus_init(sb);
+ if (error)
+ goto fail;
+
+ FM10K_SW_TRACE("sbus %s: attach successful", name);
+ return sb;
+fail:
+ if (sb)
+ fm10k_sbus_detach(sb);
+ return NULL;
+}
+
+void
+fm10k_sbus_detach(struct fm10k_sbus *sb)
+{
+ FM10K_SW_TRACE("sbus %s: detaching", sb->name);
+
+ rte_free(sb);
+}
+
+static int
+fm10k_sbus_init(struct fm10k_sbus *sb)
+{
+ uint32_t data;
+ int error;
+
+ error = fm10k_sbus_sbm_reset(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_reset_all(sb);
+ if (error)
+ goto done;
+
+ /* set clock to REFCLK/2 */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ FM10K_SW_SBUS_RING_DIVIDER);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to set ring divider", sb->name);
+ goto done;
+ }
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ &data);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to read back ring divider",
+ sb->name);
+ goto done;
+ }
+ if (data != FM10K_SW_SBUS_RING_DIVIDER) {
+ FM10K_SW_ERR("sbus %s: ring divider "
+ "verify failed (expected %u, got %u)",
+ sb->name, FM10K_SW_SBUS_RING_DIVIDER, data);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return error;
+}
+
+static int
+fm10k_sbus_exec(struct fm10k_sbus *sb, struct fm10k_sbus_req *req)
+{
+ struct fm10k_switch *sw = sb->sw;
+ unsigned int total_usecs;
+ unsigned int drop_lock;
+ unsigned int delay_time;
+ int error = 0;
+ uint32_t data;
+ uint8_t expected_result;
+
+ FM10K_SW_SBUS_LOCK(sb);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (data & FM10K_SW_SBUS_COMMAND_BUSY) {
+ FM10K_SW_INFO("sbus %s: bus busy (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+
+ switch (req->op) {
+ case FM10K_SW_SBUS_OP_RESET:
+ FM10K_SW_TRACE("sbus %s: RESET dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_RESET;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_WRITE:
+ FM10K_SW_TRACE("sbus %s: WRITE dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_WRITE;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_READ:
+ FM10K_SW_TRACE("sbus %s: READ dev=0x%02x reg=0x%02x",
+ sb->name, req->dev, req->reg);
+ expected_result = FM10K_SW_SBUS_RESULT_READ;
+ req->data = 0;
+ break;
+ default:
+ FM10K_SW_INFO("sbus %s: invalid opcode 0x%02x",
+ sb->name, req->op);
+ error = -1;
+ goto done;
+ }
+
+ /* Clear the execute bit */
+ fm10k_write_switch_reg(sw, sb->cmd_reg, 0);
+
+ data =
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_OP, req->op) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_ADDRESS, req->dev) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_REGISTER, req->reg) |
+ FM10K_SW_SBUS_COMMAND_EXECUTE;
+ fm10k_write_switch_reg(sw, sb->cmd_reg, data);
+ fm10k_write_flush(sw);
+
+ total_usecs = 0;
+ delay_time = 1;
+ drop_lock = 0;
+ do {
+ if (drop_lock)
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_udelay(delay_time);
+ if (drop_lock)
+ FM10K_SW_SWITCH_LOCK(sw);
+ total_usecs += delay_time;
+ if (total_usecs >= FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX) {
+ drop_lock = 1;
+ delay_time <<= 1;
+ }
+
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (!(data & FM10K_SW_SBUS_COMMAND_BUSY)) {
+ if (FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE)
+ != expected_result) {
+ FM10K_SW_INFO("sbus %s: expected "
+ "result code %u, got %u", sb->name,
+ expected_result,
+ FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE));
+ error = -1;
+ goto done;
+ }
+ if (req->op == FM10K_SW_SBUS_OP_READ) {
+ req->data =
+ fm10k_read_switch_reg(sw, sb->resp_reg);
+ FM10K_SW_TRACE("sbus %s: READ data=0x%02x",
+ sb->name, req->data);
+ }
+ goto done;
+ }
+
+ } while (total_usecs < FM10K_SW_SBUS_COMMAND_TIMEOUT_US);
+
+ error = -1;
+ FM10K_SW_INFO("sbus %s: command timed out after %u us "
+ "(op=0x%02x dev=0x%02x reg=0x%02x data=0x%08x)", sb->name,
+ total_usecs, req->op, req->dev, req->reg, req->data);
+
+done:
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ FM10K_SW_SBUS_UNLOCK(sb);
+ return error;
+}
+
+static int
+fm10k_sbus_reset_all(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_BROADCAST;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: broadcast reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+static int
+fm10k_sbus_sbm_reset(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_SPICO;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: SBM reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+int
+fm10k_sbus_read(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t *data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_READ;
+ req.dev = dev;
+ req.reg = reg;
+
+ error = fm10k_sbus_exec(sb, &req);
+ *data = error ? 0 : req.data;
+
+ return error;
+}
+
+int
+fm10k_sbus_write(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_WRITE;
+ req.dev = dev;
+ req.reg = reg;
+ req.data = data;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ return error;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.h b/drivers/net/fm10k/switch/fm10k_sbus.h
new file mode 100644
index 0000000..e67967f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SBUS_H_
+#define _FM10K_SW_SBUS_H_
+
+#include <pthread.h>
+
+
+struct fm10k_sbus {
+ struct fm10k_switch *sw;
+ const char *name;
+ pthread_mutex_t lock;
+ unsigned int cfg_reg;
+ unsigned int cmd_reg;
+ unsigned int req_reg;
+ unsigned int resp_reg;
+};
+
+#define FM10K_SW_SBUS_LOCK(sb_) pthread_mutex_lock(&((sb_)->lock))
+#define FM10K_SW_SBUS_UNLOCK(sb_) pthread_mutex_unlock(&((sb_)->lock))
+
+struct fm10k_sbus_req {
+ uint8_t op;
+ uint8_t dev;
+ uint8_t reg;
+ uint32_t data;
+};
+
+
+struct fm10k_sbus *fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name, unsigned int cfg_reg);
+void fm10k_sbus_detach(struct fm10k_sbus *sb);
+int fm10k_sbus_read(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t *data);
+int fm10k_sbus_write(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t data);
+
+#endif /* _FM10K_SW_SBUS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
new file mode 100644
index 0000000..2423dbf
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SWITCH_H_
+#define _FM10K_SW_SWITCH_H_
+
+#include <rte_spinlock.h>
+#include <pthread.h>
+#include <sys/time.h>
+#include <semaphore.h>
+
+#include "../fm10k.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+
+
+/*
+ * EPL
+ */
+#define FM10K_SW_EXT_PORTS_MAX 4
+
+#define FM10K_SW_EPLS_MAX 9
+#define FM10K_SW_EPLS_SUPPORTED 2
+#define FM10K_SW_EPL_LANES 4
+#define FM10K_SW_EPLS_PEP_MAX 2
+
+/*
+ * PEP
+ */
+#define FM10K_SW_PEP_GROUP_MAX 10
+#define FM10K_SW_LOGICAL_PORTS_MAX 48
+
+#define FM10K_SW_PEPS_MAX 9
+#define FM10K_SW_PEPS_SUPPORTED 4
+#define FM10K_SW_PEP_PORTS_MAX 4
+
+/*
+ * GLORT
+ */
+#define FM10K_SW_GROUP_MAX 10
+#define FM10K_SW_VFS_MAX 64
+#define FM10K_SW_PFS_GLORT_START 0x1000
+#define FM10K_SW_VF_GLORT_START 0x2000
+#define FM10K_SW_MULTI_GLORT_START 0x3000
+#define FM10K_SW_DPORT_MASK(lport) (1ULL << (lport))
+
+
+/*
+ * CONFIG
+ */
+#define FM10K_SW_CONFIG_MAX 1000
+#define FM10K_SW_FFU_RULE_MAX 256
+#define FM10K_SW_FFU_RULE_ITEM_MAX 10
+
+/*
+ * FFU COUNT
+ */
+#define FM10K_SW_FFU_CNT_BANK 3
+#define FM10K_SW_POLICER_LAST 19
+#define FM10K_SW_FFU_CNT_START (FM10K_SW_POLICER_LAST + 1)
+#define FM10K_SW_FFU_CNT_MAX 100
+
+#define FM10K_SW_PEP_QUEUES_MAX 256
+#define FM10K_SW_VF_QUEUES_MAX (FM10K_SW_PEP_QUEUES_MAX - 1)
+
+#define FM10K_SW_FTAG_SIZE 8
+#define FM10K_SW_PACKET_SIZE_MIN 17
+#define FM10K_SW_PACKET_SIZE_MAX (15 * 1024)
+#define FM10K_SW_MTU_MAX \
+ (FM10K_SW_PACKET_SIZE_MAX - ETHER_HDR_LEN - ETHER_VLAN_ENCAP_LEN)
+#define FM10K_SW_SEG_SIZE_MAX (16 * 1024)
+#define FM10K_SW_TSO_SIZE_MAX (256 * 1024 - 1)
+
+#define FM10K_SW_MEM_POOL_SEG_SIZE 192
+#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
+#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+
+
+
+#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
+#define FM10K_SW_CARD(vname_, dname_) \
+ FM10K_SW_CARD_ID(FM10K_SW_VENDOR_ID_##vname_, \
+ FM10K_SW_DEV_ID_##dname_)
+
+/*
+ * All IDs that may appear in the vendor ID or subsystem vendor ID.
+ */
+#define FM10K_SW_VENDOR_ID_INTEL 0x8086
+#define FM10K_SW_VENDOR_ID_SILICOM 0x1374
+#define FM10K_SW_VENDOR_ID_SILICOM_RB 0x1B2E
+
+
+/*
+ * All IDs that may appear in the device ID or subsystem device ID.
+ */
+#define FM10K_SW_DEV_ID_FM10K 0x15a4
+
+/* Silicom cards */
+#define FM10K_SW_DEV_ID_PE310G4DBIR_T 0x01B0
+#define FM10K_SW_DEV_ID_PE310G4DBIR_SRD 0x01B1
+#define FM10K_SW_DEV_ID_PE310G4DBIR_LRD 0x01B2
+#define FM10K_SW_DEV_ID_PE310G4DBIR_ER 0x01B3
+#define FM10K_SW_DEV_ID_PE310G4DBIR_DA 0x01B4
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS41 0x01B8
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS43 0x01B9
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QL4 0x01BA
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4 0x01C0
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4_REV_2 0x01C4
+#define FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4 0x01C1
+#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
+#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
+/*
+ * SWITCH
+ */
+
+struct fm10k_device_info {
+ uint16_t subvendor;
+ uint16_t subdevice;
+ const char *desc;
+ uint8_t num_ext_ports;
+ uint8_t ext_port_speed;
+ uint8_t num_epls;
+ uint8_t num_peps;
+};
+
+
+struct fm10k_i2c;
+struct fm10k_sbus;
+struct fm10k_hw;
+struct fm10k_ext_ports;
+struct fm10k_dpdk_cfg;
+
+struct fm10k_sw_port_map {
+ uint32_t glort;
+ uint16_t logical_port; /* logical port number */
+ uint16_t physical_port; /* */
+};
+
+struct fm10k_switch {
+ uint32_t *hw_addr;
+ uint32_t *sw_addr;
+ struct fm10k_hw *master_hw;
+ struct fm10k_device_info *info;
+ pthread_mutex_t lock;
+ struct fm10k_i2c *i2c;
+ struct fm10k_sbus *epl_sbus;
+ struct fm10k_ext_ports *ext_ports;
+ sem_t intr_tq;
+ pthread_t intr_task;
+ pthread_t led_task;
+ pthread_t stats_task;
+ uint32_t detaching;
+ uint32_t glort_cam_ram_idx;
+ uint32_t glort_dest_table_idx;
+ uint32_t mcast_dest_table_idx;
+ uint32_t mcast_len_table_idx;
+ uint32_t mcast_vlan_table_idx;
+ uint32_t epl_serdes_code_version_build_id;
+ uint16_t pep_mask; /* mask of non-master peps */
+ uint8_t pepno;
+ uint8_t serdes_loopback;
+ uint8_t epla_no;
+ uint8_t eplb_no;
+ uint8_t mac_addr[6];
+ struct fm10k_dpdk_cfg *dpdk_cfg;
+ struct fm10k_sw_port_map *pep_map;
+ struct fm10k_sw_port_map *epl_map;
+ int inited;
+};
+
+#define FM10K_SW_SWITCH_LOCK(sw_) pthread_mutex_lock(&((sw_)->lock))
+#define FM10K_SW_SWITCH_UNLOCK(sw_) pthread_mutex_unlock(&((sw_)->lock))
+
+#define FM10K_SW_HOWMANY(x, y, y1) (((x) + (y) - 1) / (y1))
+
+
+static inline uint64_t
+fm10k_uptime_us(void)
+{
+ struct timeval tv;
+
+ gettimeofday(&tv, NULL);
+ return ((uint64_t)tv.tv_sec * 1000000 + tv.tv_usec);
+}
+
+
+static inline uint32_t
+fm10k_read_switch_reg(struct fm10k_switch *sw, uint32_t reg)
+{
+ return ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+}
+
+static inline uint64_t
+fm10k_read_switch_reg64(struct fm10k_switch *sw, uint32_t reg)
+{
+ uint64_t temp, result;
+
+ result = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+ temp = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1];
+ result |= temp << 32;
+
+ return result;
+}
+
+static inline void
+fm10k_read_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *results, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ results[i] =
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i];
+}
+
+
+static inline void
+fm10k_write_switch_reg(struct fm10k_switch *sw, uint32_t reg, uint32_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val;
+}
+
+static inline void
+fm10k_write_switch_reg64(struct fm10k_switch *sw, uint32_t reg, uint64_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] = val >> 32;
+}
+
+static inline void
+fm10k_write_switch_reg128(struct fm10k_switch *sw,
+ uint32_t reg, uint64_t val_hi, uint64_t val_lo)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] =
+ val_lo & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] =
+ val_lo >> 32;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 2] =
+ val_hi & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 3] =
+ val_hi >> 32;
+}
+
+static inline void
+fm10k_write_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *data, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i] =
+ data[i];
+}
+
+static inline uint32_t
+fm10k_write_flush(struct fm10k_switch *sw)
+{
+ return ((volatile uint32_t *)sw->master_hw->hw_addr)[FM10K_SW_CTRL];
+}
+
+static inline void
+fm10k_gpio_output_set(struct fm10k_switch *sw, int gpio_pin, int value)
+{
+ uint32_t data;
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_CFG);
+ data |= 1 << gpio_pin;
+ data &= ~(1 << (gpio_pin + 16));
+
+ /* set gpio output */
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_CFG, data);
+
+ /*
+ * Wait 1 msec (PCA spec specifies reset pulse width = 4 ns and
+ * and reset time = 100 ns)
+ */
+ usec_delay(1000);
+ /* set reset */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_DATA);
+ if (value == 0)
+ data &= ~(1 << gpio_pin);
+ else
+ data |= 1 << gpio_pin;
+
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
+}
+
+#define fm10k_udelay usec_delay
+typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
+
+unsigned int fm10k_switch_eplidx_to_eplno
+ (struct fm10k_switch *sw, unsigned int eplidx);
+void fm10k_switch_intr(struct fm10k_hw *hw);
+
+struct fm10k_switch *fm10k_switch_get(void);
+struct fm10k_device_info *fm10k_get_device_info(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func);
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue);
+int fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw,
+ struct fm10k_hw *hw_list[]);
+int fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+void *fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw);
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw);
+void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+
+uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
+uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
+uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
+uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
+uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
+uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
+uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new);
+
+int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
+
+void fm10k_switch_flowset_switchto(const char *name);
+void fm10k_switch_show_port(void);
+void fm10k_switch_show_ffu(void);
+void fm10k_switch_show_bank(void);
+
+void fm10k_flow_list_init(void *flow_list);
+const struct rte_flow_ops *fm10k_flow_ops_get(void);
+
+#endif
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-03-16 6:48 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
` (2 subsequent siblings)
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add epl serdes include loading spico,
controlling pcsl, dma, dfe, ical. Add spico code.
Add state machine for epl lane and port, it creates
a pthread to handle the state changing event.
Add external port management, which will use
state machine to handle the event from lane and port.
The lane state will change between DOWN, WAIT_PLL_LOCK,
WAIT_SIGNAL_OK, WAIT_DFE_ICAL, WAIT_DFE_PCAL, UP.
The port state will change between DOWN, WAIT_LANE_UP, UP.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 4 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 840 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 81 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
9 files changed, 6206 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 15ea187..0d3b688 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -84,6 +84,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.c b/drivers/net/fm10k/switch/fm10k_ext_port.c
new file mode 100644
index 0000000..7f8efd0
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.c
@@ -0,0 +1,840 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_serdes.h"
+#include "fm10k_sm.h"
+#include "fm10k_switch.h"
+
+
+#define LANE_EVENT(e_) FM10K_SW_EXT_PORT_LANE_EVENT_##e_
+#define LANE_TIMER(t_) FM10K_SW_EXT_PORT_LANE_TIMER_##t_
+#define LANE_TIMEOUT(t_) FM10K_SW_EXT_PORT_LANE_##t_##_TIMEOUT_MS
+#define LANE_POLL(t_) FM10K_SW_EXT_PORT_LANE_##t_##_POLL_MS
+#define LANE_STATE(s_) FM10K_SW_EXT_PORT_LANE_STATE_##s_
+#define LANE_FLAG(f_) FM10K_SW_EXT_PORT_LANE_FLAG_##f_
+#define LANE_FLAGS(f_) FM10K_SW_EXT_PORT_LANE_FLAGS_##f_
+#define LANE_FLAG_SET(l_, f_) ((l_)->flags & LANE_FLAG(f_))
+#define LANE_FLAGS_SET(l_, f_, f1_) \
+ (((l_)->flags & LANE_FLAGS(f_)) == LANE_FLAGS(f1_))
+
+#define PORT_EVENT(e_) FM10K_SW_EXT_PORT_EVENT_##e_
+#define PORT_STATE(s_) FM10K_SW_EXT_PORT_STATE_##s_
+
+static void fm10k_ext_ports_epl_intr
+ (struct fm10k_ext_ports *ports, unsigned int eplno);
+static void fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip);
+static void fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_bring_down(struct fm10k_ext_port *port);
+static void fm10k_ext_port_lane_process_event
+ (struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_lane_process_timer
+ (struct fm10k_sm *sm, uint16_t timer);
+static void fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_phy_enable(struct fm10k_ext_port *port);
+static void fm10k_ext_port_phy_disable(struct fm10k_ext_port *port);
+
+struct fm10k_ext_ports *
+fm10k_ext_ports_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int is_quad, lane_speed;
+ unsigned int i, j;
+ unsigned int eplidx, laneno;
+
+ FM10K_SW_TRACE("attaching external ports\n");
+
+ ports = (struct fm10k_ext_ports *)rte_zmalloc
+ ("fm10k_ext_ports", sizeof(struct fm10k_ext_ports), 0);
+ if (ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports\n");
+ goto fail;
+ }
+
+ ports->ports = (struct fm10k_ext_port *)rte_zmalloc
+ ("fm10k_ext_port_list",
+ sizeof(struct fm10k_ext_port) * cfg->num_ext_ports, 0);
+ if (ports->ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports list\n");
+ goto fail;
+ }
+ ports->sw = sw;
+ ports->num_ports = cfg->num_ext_ports;
+ ports->ports_per_epl = cfg->num_ext_ports / cfg->num_epls;
+ ports->epl_mask = 0;
+
+ switch (cfg->ext_port_speed) {
+ default:
+ case 10:
+ is_quad = 0;
+ lane_speed = 10;
+ break;
+ case 25:
+ is_quad = 0;
+ lane_speed = 25;
+ break;
+ case 40:
+ is_quad = 1;
+ lane_speed = 10;
+ break;
+ case 100:
+ is_quad = 1;
+ lane_speed = 25;
+ break;
+ }
+
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ port->ports = ports;
+ fm10k_ext_port_eplidx_lane(ports, i, &eplidx, &laneno);
+ port->portno = i;
+ port->eplno = eplidx ? sw->eplb_no : sw->epla_no;
+ snprintf(port->name, sizeof(port->name), "EPL[%u][%u]",
+ port->eplno, port->first_lane);
+ port->first_lane = laneno;
+ port->is_quad = is_quad;
+ port->num_lanes = port->is_quad ? 4 : 1;
+ port->lane_speed = lane_speed;
+ port->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_NUM_EVENTS, 0,
+ fm10k_ext_port_process_event, NULL,
+ (void *)port);
+ if (port->sm == NULL)
+ goto fail;
+ port->sm->portno = port->portno;
+ port->sm->laneno = 0;
+ port->an_im = FM10K_SW_AN_IM_ALL;
+ port->link_im = FM10K_SW_LINK_IM_ALL;
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ lane->port = port;
+ lane->abs_laneno = port->eplno * FM10K_SW_EPL_LANES + j;
+ lane->rel_laneno = j;
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+ lane->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_LANE_NUM_EVENTS,
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS,
+ fm10k_ext_port_lane_process_event,
+ fm10k_ext_port_lane_process_timer,
+ (void *)lane);
+ if (lane->sm == NULL)
+ goto fail;
+ lane->sm->portno = lane->port->portno;
+ lane->sm->laneno = lane->rel_laneno;
+ }
+
+ ports->epl_mask |= (1 << port->eplno);
+ }
+
+ return (ports);
+fail:
+ if (ports)
+ fm10k_ext_ports_detach(ports);
+ return NULL;
+}
+
+void
+fm10k_ext_ports_detach(struct fm10k_ext_ports *ports)
+{
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i, j;
+
+ FM10K_SW_TRACE("detaching external ports\n");
+
+ if (ports->ports) {
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+
+ if (port->sm)
+ fm10k_sm_detach(port->sm);
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ if (lane->sm)
+ fm10k_sm_detach(lane->sm);
+ }
+
+ fm10k_ext_port_phy_disable(port);
+ }
+ rte_free(ports->ports);
+ }
+
+ rte_free(ports);
+}
+
+unsigned int
+fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports, uint64_t gid)
+{
+ unsigned int i;
+ uint16_t epl_mask;
+
+ /*
+ * Process EPL interrupts for all EPLs that are in use and have
+ * their bit set in global interrupt detect.
+ */
+ epl_mask = ports->epl_mask &
+ FM10K_SW_REG_FIELD64(gid, GLOBAL_INTERRUPT_DETECT_EPL);
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++)
+ if (epl_mask & (1 << i))
+ fm10k_ext_ports_epl_intr(ports, i);
+
+ return ports->epl_mask;
+}
+
+static void
+fm10k_ext_ports_epl_intr(struct fm10k_ext_ports *ports, unsigned int eplno)
+{
+ struct fm10k_switch *sw = ports->sw;
+ struct fm10k_ext_port *port;
+ uint64_t gid;
+ unsigned int i, j;
+ uint32_t epl_ip;
+
+ /*
+ * The global interrupt detect architecture makes it somewhat
+ * challenging to ensure events are not lost and spurious interrupts
+ * are not generated.
+ *
+ * The EPLs have an internal interrupt hierarchy whose reporting is
+ * rooted at EPL_IP. Whenever an EPL transitions from having no
+ * unmasked interrupts pending to at least one unmasked interrupt
+ * pending or from having at least one unmasked interrupt pending to
+ * no unmasked interrupts pending, it sends a vector to the
+ * interrupt controller over a serial bus indicating the
+ * at-least-one/none pending state, and the controller updates the
+ * bit for that EPL in the global interrupt detect register
+ * accordingly.
+ *
+ * Because of the update-vector-in-flight potential, there is a race
+ * between re-enabling the EPL PCIe interrupt mask after servicing
+ * the EPL interrupts and the all-clear vector from the EPL reaching
+ * the interrupt controller - it's possible for the software to
+ * clear all the EPL-internal interrupt sources during servicing and
+ * then re-enable the EPL interrupt in the PCIe mask before the
+ * all-clear vector reaches the interrupt controller and clears the
+ * EPL pending bit in the global interrupt detect register. This
+ * would result in a spurious EPL interrupt, potentially for every
+ * necessary EPL interrupt.
+ *
+ * The approach taken here to avoid spurious interrupts is to first
+ * mask off all EPL-internal sources, then poll the EPL bits in the
+ * global interrupt detect register until they clear (indicating the
+ * update vector from the EPL has reached the interrupt controller).
+ * Then software interrupt processing proceeds, and the EPL-internal
+ * masks are restored to their desired state at the end. This
+ * ensures that when the EPL PCIe interrupt mask is re-enabled, the
+ * EPL pending bits in the global interrupt detect register will
+ * only be set if there are new, unprocessed EPL events of interest.
+ *
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* Grab epl_ip before it is cleared by the masks below */
+ epl_ip = fm10k_read_switch_reg(sw, FM10K_SW_EPL_IP(eplno));
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_AN_IM(eplno, port->first_lane),
+ FM10K_SW_AN_IM_ALL);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IM(eplno, port->first_lane),
+ FM10K_SW_LINK_IM_ALL);
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(eplno, j),
+ FM10K_SW_SERDES_IM_ALL);
+ }
+ }
+
+ /*
+ * Wait for the all-clear vector to propagate over the internal
+ * interrupt serial bus to the global interrupt detect.
+ */
+ do {
+ gid = fm10k_read_switch_reg64
+ (sw, FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ } while (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(eplno));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ /*
+ * Proceed with interrupt processing per-external-port.
+ */
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+ fm10k_ext_port_intr(port, epl_ip);
+ }
+}
+
+static void
+fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ uint32_t serdes_ip, data = 0;
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (epl_ip & FM10K_SW_EPL_IP_SERDES_INTERRUPT(i)) {
+ FM10K_SW_SWITCH_LOCK(sw);
+ serdes_ip = fm10k_read_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i));
+ serdes_ip &= ~lane->im;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i), serdes_ip);
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK)
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_STATUS(port->eplno,
+ i));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK) {
+ fm10k_sm_send_event(lane->sm,
+ (data &
+ FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK) ?
+ LANE_EVENT(SIGNAL_OK) : LANE_EVENT(SIGNAL_NOK));
+ }
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RX_PLL_LOCK));
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_TX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(TX_PLL_LOCK));
+ } else {
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RESTORE_IM));
+ }
+ }
+}
+
+void
+fm10k_ext_port_up(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_UP));
+}
+
+void
+fm10k_ext_port_down(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_DOWN));
+}
+
+unsigned int
+fm10k_ext_port_isup(struct fm10k_ext_port *port)
+{
+ /* no locked needed as we are just inspecting
+ * the current value of an int
+ */
+ unsigned int port_state = port->sm->state;
+
+ return (port_state == FM10K_SW_EXT_PORT_STATE_UP);
+}
+
+static void
+fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port *port = sm->ctx;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ unsigned int lanes_up;
+
+ switch (sm->state) {
+ case PORT_STATE(DOWN):
+ if (event == PORT_EVENT(BRING_UP)) {
+ fm10k_ext_port_phy_enable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event
+ (lane->sm, LANE_EVENT(BRING_UP));
+ }
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ }
+ break;
+
+ case PORT_STATE(WAITING_FOR_LANES):
+ if (event == PORT_EVENT(LANE_UP)) {
+ lanes_up = 0;
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (lane->sm->state != LANE_STATE(UP))
+ break;
+ lanes_up++;
+ }
+ if (lanes_up == port->num_lanes) {
+ sm->state = PORT_STATE(UP);
+ FM10K_SW_INFO
+ ("Port %d is LINK UP", port->portno);
+ }
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ }
+ break;
+
+ case PORT_STATE(UP):
+ if (event == PORT_EVENT(LANE_DOWN)) {
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ }
+ }
+}
+
+static void
+fm10k_ext_port_bring_down(struct fm10k_ext_port *port)
+{
+ struct fm10k_sm *sm = port->sm;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+
+ sm->state = PORT_STATE(DOWN);
+
+ fm10k_ext_port_phy_disable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event(lane->sm, LANE_EVENT(BRING_DOWN));
+ }
+}
+
+static void
+fm10k_ext_port_lane_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ struct fm10k_ext_port *port = lane->port;
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ int error;
+
+ switch (sm->state) {
+ case LANE_STATE(DOWN):
+ if (event == LANE_EVENT(BRING_UP)) {
+ lane->flags &= ~LANE_FLAGS(PLLS_LOCKED);
+
+ /*
+ * Clear all pending interrupts.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, lane->rel_laneno),
+ FM10K_SW_SERDES_IP_ALL);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->im &= ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+ }
+ break;
+
+ case LANE_STATE(WAIT_PLL_LOCK):
+ if (event == LANE_EVENT(RX_PLL_LOCK) ||
+ event == LANE_EVENT(TX_PLL_LOCK)) {
+ if (event == LANE_EVENT(RX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(RX_LOCKED);
+ if (event == LANE_EVENT(TX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(TX_LOCKED);
+ if (LANE_FLAGS_SET(lane, PLLS_LOCKED, PLLS_LOCKED)) {
+ error = fm10k_epl_serdes_continue_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_SIGNAL_OK);
+ /* unmask RX signal OK interrupt */
+ lane->im |=
+ (FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ lane->im &=
+ ~FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ }
+ }
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_SIGNAL_OK):
+ if (event == LANE_EVENT(SIGNAL_OK)) {
+ sm->state = LANE_STATE(WAIT_DFE_ICAL);
+
+ fm10k_epl_serdes_configure_for_dfe_tuning(lane);
+ fm10k_epl_serdes_start_dfe_ical(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ sm->state = LANE_STATE(WAIT_DFE_PCAL);
+
+ fm10k_epl_serdes_start_dfe_pcal(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+ sm->state = LANE_STATE(UP);
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_UP));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT)) ||
+ (event == LANE_EVENT(SIGNAL_NOK))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(UP):
+ if (event == LANE_EVENT(SIGNAL_NOK))
+ fm10k_ext_port_restart_lane(lane);
+ else if (event == LANE_EVENT(BRING_DOWN))
+ fm10k_ext_port_disable_lane(lane);
+ break;
+ }
+
+ /* Always restore the interrupt mask */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(port->eplno,
+ lane->rel_laneno),
+ lane->im);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_ext_port_lane_process_timer(struct fm10k_sm *sm, uint16_t timer)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ uint32_t status;
+
+ switch (sm->state) {
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_ICAL);
+ fm10k_epl_serdes_dfe_ical_status(lane, &status);
+
+ if (status == FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_ICAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else if (status == FM10K_SW_SERDES_DFE_ICAL_CONVERGED)
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_FAILED));
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_PCAL);
+
+ fm10k_epl_serdes_dfe_pcal_status(lane, &status);
+ if (status == FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_PCAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else {
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ }
+ }
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+ struct fm10k_ext_port *port = lane->port;
+ int error;
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->flags &= ~FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED;
+ lane->im |= FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ lane->im = ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_DOWN));
+}
+
+static void
+fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+
+ fm10k_sm_timer_cancel(sm, LANE_TIMER(DFE_TUNING));
+ fm10k_epl_serdes_disable(lane);
+
+ /*
+ * Disable all interrupts.
+ */
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+
+ sm->state = LANE_STATE(DOWN);
+}
+
+void
+fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane)
+{
+ FM10K_SW_ASSERT(ext_port_idx < ports->num_ports,
+ ("ext_port_idx out of range"));
+
+ *eplidx = ext_port_idx / ports->ports_per_epl;
+ *lane = ext_port_idx % ports->ports_per_epl;
+}
+
+
+static void
+fm10k_ext_port_phy_enable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the
+ * PCA9505 that the QSFP control lines are connected
+ * to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get
+ * at the PCA9505 that the QSFP control lines
+ * are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x10);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x10);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to enable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_phy_disable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x60);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x61);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to disable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.h b/drivers/net/fm10k/switch/fm10k_ext_port.h
new file mode 100644
index 0000000..9cc0574
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_EXT_PORT_H_
+#define _FM10K_SW_EXT_PORT_H_
+
+#include "fm10k_regs.h"
+#include "fm10k_sm.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+
+struct fm10k_sbus;
+struct fm10k_switch;
+
+#define FM10K_SW_TRACE_ENABLE 0
+
+enum fm10k_ext_port_lane_state {
+ FM10K_SW_EXT_PORT_LANE_STATE_DOWN,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_ICAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_PCAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_UP
+};
+
+enum fm10k_ext_port_lane_event {
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_TX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_NOK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_COMPLETE,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_FAILED,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_TIMED_OUT,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RESTORE_IM,
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_DOWN,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_EVENTS
+};
+
+enum fm10k_ext_port_lane_timer {
+ FM10K_SW_EXT_PORT_LANE_TIMER_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_DFE_TUNING,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS
+};
+
+#define FM10K_SW_EXT_PORT_LANE_PLL_LOCK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_SIGNAL_OK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_POLL_MS 200
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_TIMEOUT_MS 3000
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_POLL_MS 100
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_TIMEOUT_MS 2000
+
+
+enum fm10k_ext_port_state {
+ FM10K_SW_EXT_PORT_STATE_DOWN,
+ FM10K_SW_EXT_PORT_STATE_WAITING_FOR_LANES,
+ FM10K_SW_EXT_PORT_STATE_UP
+};
+
+enum fm10k_ext_port_event {
+ FM10K_SW_EXT_PORT_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_EVENT_BRING_DOWN,
+ FM10K_SW_EXT_PORT_EVENT_LANE_UP,
+ FM10K_SW_EXT_PORT_EVENT_LANE_DOWN,
+
+ FM10K_SW_EXT_PORT_NUM_EVENTS
+};
+
+struct fm10k_ext_port;
+
+struct fm10k_ext_port_lane {
+ struct fm10k_ext_port *port;
+ struct fm10k_sm *sm;
+ uint32_t im;
+ uint16_t dfe_poll_time_ms;
+ uint8_t abs_laneno;
+ uint8_t rel_laneno;
+ uint8_t flags;
+};
+
+#define FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED 0x01
+#define FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED 0x02
+
+#define FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED \
+ (FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED | \
+ FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED)
+
+struct fm10k_ext_ports;
+
+struct fm10k_ext_port {
+ struct fm10k_ext_ports *ports;
+ struct fm10k_sm *sm;
+ struct fm10k_ext_port_lane lanes[FM10K_SW_EPL_LANES];
+ char name[10];
+ uint8_t portno;
+ uint8_t eplno;
+ uint8_t lport; /* filled in by switch */
+ uint8_t first_lane; /* range is [0..3] */
+ uint8_t num_lanes;
+ uint8_t is_quad;
+ uint8_t lane_speed;
+ uint8_t last_led_flags;
+ uint32_t an_im;
+ uint32_t link_im;
+};
+
+#define FM10K_SW_EXT_PORT_LED_FLAG_UP 0x01
+#define FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE 0x02
+
+struct fm10k_ext_ports {
+ struct fm10k_switch *sw;
+ struct fm10k_ext_port *ports;
+ uint16_t epl_mask;
+ uint8_t num_ports;
+ uint8_t ports_per_epl;
+};
+
+#define FM10K_SW_EXT_PORTS_EPL_USED(p_, n_) ((p_)->epl_mask & (1 << (n_)))
+
+struct fm10k_ext_ports *fm10k_ext_ports_attach(struct fm10k_switch *sw);
+void fm10k_ext_ports_detach(struct fm10k_ext_ports *ports);
+unsigned int fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports,
+ uint64_t gid);
+void fm10k_ext_port_up(struct fm10k_ext_port *port);
+void fm10k_ext_port_down(struct fm10k_ext_port *port);
+unsigned int fm10k_ext_port_isup(struct fm10k_ext_port *port);
+void fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane);
+
+#endif /* _FM10K_SW_EXT_PORT_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.c b/drivers/net/fm10k/switch/fm10k_serdes.c
new file mode 100644
index 0000000..01b76a3
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.c
@@ -0,0 +1,1936 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_serdes.h"
+#include "fm10k_spico_code.h"
+#include "fm10k_switch.h"
+
+
+#define FM10K_SW_SERDES_BIST_TIMEOUT_MS 5000
+#define FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS 5000
+#define FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS 3000
+#define FM10K_SW_SERDES_INT02_TIMEOUT_MS 500
+
+#define FM10K_SW_SERDES_CONFIG_DELAY_US 20
+#define FM10K_SW_SERDES_RESET_DELAY_US 22
+#define FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US 200
+
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_25G 0xa5
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_10G 0x42
+
+#define FM10K_SW_SERDES_WIDTH_40 3 /* 25G */
+#define FM10K_SW_SERDES_WIDTH_20 1 /* 10G */
+
+#define FM10K_SW_SERDES_EQ_SEL_PRECUR 0
+#define FM10K_SW_SERDES_EQ_SEL_ATTEN 1
+#define FM10K_SW_SERDES_EQ_SEL_POSTCUR 2
+
+#define FM10K_SW_SERDES_DFE_DEFAULT_HF 0x00
+#define FM10K_SW_SERDES_DFE_DEFAULT_LF 0x0c
+#define FM10K_SW_SERDES_DFE_DEFAULT_DC 0x38
+#define FM10K_SW_SERDES_DFE_DEFAULT_BW 0x0f
+
+#define FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES 10
+#define FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES 500
+
+#define FM10K_SERDES_SPICO_REG_0X00 0x00
+#define FM10K_SERDES_SPICO_REG_0X01 0x01
+#define FM10K_SERDES_SPICO_REG_0X02 0x02
+#define FM10K_SERDES_SPICO_REG_0X03 0x03
+#define FM10K_SERDES_SPICO_REG_0X04 0x04
+#define FM10K_SERDES_SPICO_REG_0X05 0x05
+#define FM10K_SERDES_SPICO_REG_0X07 0x07
+#define FM10K_SERDES_SPICO_REG_0X08 0x08
+#define FM10K_SERDES_SPICO_REG_0X0A 0x0A
+#define FM10K_SERDES_SPICO_REG_0X0B 0x0b
+#define FM10K_SERDES_SPICO_REG_0X14 0x14
+#define FM10K_SERDES_SPICO_REG_0X16 0x16
+
+
+/*
+ * These values correspond to IES FM_PORT_LINK_OPTIMIZATION_BALANCE
+ * settings.
+ */
+#define FM10K_SW_LINK_OPT_PARAM_A10G 0x4800
+#define FM10K_SW_LINK_OPT_PARAM_B10G 0xA000
+#define FM10K_SW_LINK_OPT_PARAM_A25G 0x10001
+#define FM10K_SW_LINK_OPT_PARAM_B25G 0x10001
+
+#define FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD 10
+
+
+static int fm10k_spico_ram_bist(struct fm10k_sbus *sb);
+static int fm10k_load_epl_spico_code(struct fm10k_switch *sw);
+static int fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id);
+static int fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb);
+static int fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id);
+static int fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_swap_image_check_crc(struct fm10k_sbus *sb,
+ unsigned int crc_code);
+static int fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words);
+static int fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id);
+static int fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id);
+static int fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider);
+static int fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static void fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static int fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold);
+static int fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq);
+static int fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms);
+static int fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data,
+ uint32_t mask, uint32_t *result);
+static int fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result);
+static int fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param);
+static void fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane);
+static int fm10k_epl_serdes_configure_near_loopback
+ (struct fm10k_ext_port_lane *lane);
+static int fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param);
+static int fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status);
+static int fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged);
+
+
+int
+fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: initializing EPL serdes", sb->name);
+
+ error = fm10k_spico_ram_bist(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_load_epl_spico_code(sw);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_spico_ram_bist(struct fm10k_sbus *sb)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: starting SPICO RAM BIST", sb->name);
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x03);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x05);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_BIST_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, &data);
+ if (error)
+ goto done;
+
+ if (data & 0x18) {
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ if ((data & 0x18) != 0x08) {
+ FM10K_SW_INFO("sbus %s: SPICO RAM "
+ "BIST failed with bad status (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+ goto done;
+ }
+ }
+ fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SPICO RAM BIST timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_load_epl_spico_code(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ const uint16_t *sbm_code_image;
+ uint32_t sbm_code_size;
+ uint32_t sbm_code_version_build_id;
+ const uint16_t *serdes_code_image;
+ uint32_t serdes_code_size;
+ uint32_t serdes_code_version_build_id;
+ const uint16_t *swap_code_image;
+ uint32_t swap_code_size;
+ unsigned int swap_crc_code;
+ int error = 0;
+
+ FM10K_SW_TRACE("sbus %s: loading SPICO code", sb->name);
+
+ sbm_code_image = fm10000_sbus_master_code_prd;
+ sbm_code_size = fm10000_sbus_master_code_size_prd;
+ sbm_code_version_build_id =
+ fm10000_sbus_master_code_version_build_id_prd;
+
+ FM10K_SW_TRACE("sbus %s: sbm code version=%4.4x build=%4.4x",
+ sb->name, sbm_code_version_build_id >> 16,
+ sbm_code_version_build_id & 0xffff);
+
+ serdes_code_image = fm10000_serdes_spico_code_prd2;
+ serdes_code_size = fm10000_serdes_spico_code_size_prd2;
+ serdes_code_version_build_id =
+ fm10000_serdes_spico_code_version_build_id_prd2;
+ sw->epl_serdes_code_version_build_id = serdes_code_version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: serdes code version=%4.4x build=%4.4x",
+ sb->name, serdes_code_version_build_id >> 16,
+ serdes_code_version_build_id & 0xffff);
+
+ swap_code_image = fm10000_serdes_swap_code_prd2;
+ swap_code_size = fm10000_serdes_swap_code_size_prd2;
+
+ error = fm10k_sbm_spico_upload_image(sb, sbm_code_image,
+ sbm_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_check_crc_version_build_id(sb,
+ sbm_code_version_build_id);
+ if (error)
+ goto done;
+
+ if (swap_code_size > 0) {
+ if ((sbm_code_version_build_id & 0x00008000) == 0) {
+ error = fm10k_serdes_swap_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x1a;
+ } else {
+ error = fm10k_serdes_swap_alt_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x04;
+ }
+ if (error)
+ goto done;
+
+ error = fm10k_swap_image_check_crc(sb, swap_crc_code);
+ if (error)
+ goto done;
+ }
+
+ error = fm10k_serdes_spico_upload_image(sb,
+ FM10K_SW_SBUS_ADDR_BROADCAST,
+ serdes_code_image, serdes_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_check_crc_version_build_id(sw,
+ serdes_code_version_build_id);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+static int
+fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM SPICO image "
+ "(%u words)", sb->name, num_words);
+
+ data = (1 << 6) | (1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ data = 0x80000000 | ((uint32_t)image[i] << 16) | i;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM image upload time %u ms",
+ sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ data = (1 << 6);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X16, data);
+ if (error)
+ goto done;
+
+ data = (1 << 6) | (1 << 8);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id)
+{
+ int error;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SBM SPICO code CRC "
+ "and version", sb->name);
+
+ error = fm10k_sbm_spico_do_crc(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_get_version_build_id(sb, &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO code version "
+ "compare failed (expected 0x%08x, got 0x%08x)",
+ sb->name, expected_version_build_id, version_build_id);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM SPICO image "
+ "version=0x%4.4x build_id=0x%4.4x", sb->name,
+ version_build_id >> 16,
+ version_build_id & 0xffff);
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM SPICO CRC "
+ "check", sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x02, 0, &crc);
+ if (error)
+ goto done;
+
+ crc = crc >> 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO CRC check failed "
+ "(CRC interrupt returned 0xffff)", sb->name);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM SPICO CRC check "
+ "passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM SPICO CRC "
+ "check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id)
+{
+ int error;
+ uint32_t result;
+
+ FM10K_SW_TRACE("sbus %s: getting SBM code version",
+ sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x00, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id = result & 0xffff0000;
+ error = fm10k_sbm_spico_int(sb, 0x01, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id |= (result >> 16) & 0xffff;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ error = fm10k_sbm_spico_int(sb, 0x1c, 0, &addr);
+ if (error)
+ goto done;
+
+ addr >>= 16;
+ if (addr == 0xffff) {
+ FM10K_SW_INFO("sbus %s: invalid build - cannot "
+ "upload SBM swap image", sb->name);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, addr);
+ if (error)
+ goto done;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, 0x80000000 | addr);
+ if (error)
+ goto done;
+
+ for (i = 0; i < num_words - 2; i += 3) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0xc0000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10) |
+ ((uint32_t)image[i + 2] << 20));
+ if (error)
+ goto done;
+ }
+
+ if (num_words - i == 2) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x80000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10));
+ if (error)
+ goto done;
+ } else if (num_words - i == 1) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x40000000 | image[i]);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image upload time "
+ "%u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~(1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: alt-uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 10) | (1 << 11);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ addr = 0x0400;
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X04,
+ 0x8000 | ((uint32_t)image[i + 1] << 16) |
+ (addr + i));
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image alt-upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~((1 << 10) | (1 << 11));
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+static int
+fm10k_swap_image_check_crc(struct fm10k_sbus *sb, unsigned int crc_code)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM swap image CRC "
+ "check", sb->name);
+
+ error =
+ fm10k_sbm_spico_int(sb, crc_code, 0, &crc);
+ if (error)
+ goto done;
+
+ crc >>= 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM swap image CRC check "
+ "failed (CRC interrupt 0x%02x returned 0xffff)",
+ sb->name, crc_code);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM swap image CRC "
+ "check passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM swap image "
+ "CRC check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ uint32_t reg07;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading serdes SPICO image "
+ "to bus address 0x%02x (%u words)", sb->name, dev, num_words);
+
+ reg07 = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X03, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ data = (1 << 30);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, data);
+ if (error)
+ goto done;
+
+ data = (1 << 4) | (1 << 5);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i += 3) {
+ data = 0xc0000000 | image[i];
+ if (i + 1 < num_words) {
+ data |= (uint32_t)image[i + 1] << 10;
+
+ if (i + 2 < num_words)
+ data |= (uint32_t)image[i + 2] << 20;
+ }
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X0A, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: serdes SPICO image upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, 0);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X01, 0x20000000);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X0B, data);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 1);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ unsigned int first = 0;
+ unsigned int last = FM10K_SW_EPLS_MAX * FM10K_SW_EPL_LANES - 1;
+ unsigned int i;
+ int error;
+ uint32_t data;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SPICO code CRC and "
+ "version for serdes %u through %u", sb->name, first, last);
+
+ for (i = first; i <= last; i++) {
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(i), 0xff, &data);
+ if (error || data != 0x01) {
+ FM10K_SW_TRACE("sbus %s: skipping "
+ "serdes %u (error=%d, data=0x%08x)", sb->name, i,
+ error, data);
+ continue;
+ }
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, i);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_get_version_build_id(sw, i,
+ &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SERDES SPICO code "
+ "version compare failed (expected 0x%08x, got "
+ "0x%08x)", sb->name, expected_version_build_id,
+ version_build_id);
+ error = -1;
+ goto done;
+ }
+ }
+ FM10K_SW_TRACE("sbus %s: serdes %u through %u are OK",
+ sb->name, first, last);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: performing SPICO "
+ "CRC check", sb->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3c, 0, &crc);
+ if (error)
+ goto done;
+
+ if (crc != 0) {
+ FM10K_SW_INFO("sbus %s: serdes %u: SPICO CRC check "
+ "failed (CRC interrupt returned 0x%08x)",
+ sb->name, serdes, crc);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t data;
+ uint8_t addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: resetting SPICO",
+ sb->name, serdes);
+
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+
+ data = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X0B, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 1);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X08, 0);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: getting code "
+ "version", sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x00, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id = data << 16;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3f, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id |= data & 0xffff;
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: SBM interrupt 0x%02x "
+ "(param=0x%08x)", sb->name, int_num, param);
+
+ /* int write */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X02,
+ (param << 16) | int_num);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, &data);
+ if (error)
+ goto done;
+
+ data |= (1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X08, &data);
+ if (error)
+ goto done;
+
+ if ((data & 0x8000) || ((data & 0x3ff) == 0)) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ FM10K_SW_TRACE("sbus %s: SBM "
+ "interrupt 0x%02x (param=0x%08x, result=0x%08x)",
+ sb->name, int_num, param, data);
+ *result = data;
+ goto done;
+ }
+ }
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SBM interrupt timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ uint64_t start_time_us, elapsed_time_ms;
+ unsigned int sbus_addr;
+ int error;
+ uint32_t data, intr;
+
+ sbus_addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: (sbus addr 0x%02x) "
+ "interrupt 0x%04x (param=0x%08x)", sb->name, serdes, sbus_addr,
+ int_num, param);
+
+ /* int write */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ error = fm10k_sbus_write(sb, sbus_addr,
+ FM10K_SERDES_SPICO_REG_0X03,
+ ((uint32_t)int_num << 16) | param);
+ if (error)
+ goto done;
+ }
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "(2) after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ FM10K_SW_TRACE("sbus %s: serdes %u: interrupt "
+ "0x%04x (param=0x%08x, result=0x%08x)", sb->name, serdes,
+ int_num, param, data);
+ if (result)
+ *result = data;
+ }
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int divider, width_mode;
+ int error = 0;
+ uint32_t data;
+
+ /*
+ * Bit rate and width
+ */
+ if (speed == 25) {
+ /* 25G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_25G;
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_10G;
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ /*
+ * state = CONFIGURED
+ */
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring",
+ sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_bit_rate(sw, serdes, divider);
+ if (error)
+ goto done;
+
+ fm10k_epl_serdes_set_pcsl_width_mode(sw, serdes,
+ width_mode);
+
+ /*
+ * Data select
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX data select",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_spico_int02_retry(sw, serdes, 0x1ff,
+ FM10K_SW_SERDES_INT02_TIMEOUT_MS);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_reg_write(sw, serdes, 0x21, 0x0c00);
+ if (error)
+ goto done;
+
+ /*
+ * Configure TX equalization
+ *
+ * Use defaults - copper and optical are the same.
+ *
+ * DEFAULTS| precursor | cursor | postcursor |
+ * --------+-----------+--------+------------+
+ * copper | 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ * optical| 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursors",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_ATTEN, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_PRECUR, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_POSTCUR, 15);
+ if (error)
+ goto done;
+
+ /*
+ * Configure 'options', which means:
+ * - PLL calibration mode
+ * - Phase slip
+ * - RX termination
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting PLL calibration mode",
+ sw->epl_sbus->name, serdes);
+ /* enable PLL calibration */
+ data = (1 << 0) | (1 << 1);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x11, data);
+ if (error)
+ goto done;
+ /* There appears to be no action to take for phase slip */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring RX termination",
+ sw->epl_sbus->name, serdes);
+ /* RX termination appears to default to AVDD */
+ data = (1 << 0);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x2b, data);
+ if (error)
+ goto done;
+
+ /*
+ * Set TX and RX lane polarity.
+ * Assuming no inversion required.
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring TX/RX lane polarity",
+ sw->epl_sbus->name, serdes);
+ data = 0x0300;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x13, data, NULL);
+ if (error)
+ goto done;
+
+ /*
+ * Force local fault
+ */
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /*
+ * Enable TX/RX
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX/RX",
+ sw->epl_sbus->name, serdes);
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /*
+ * Clear and enable interrupts
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring interrupts",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_sbus_write(sw->epl_sbus,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes),
+ FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int width_mode;
+ int error = 0;
+ int near_loopback = sw->serdes_loopback;
+
+ if (speed == 25) {
+ /* 25G */
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_pcsl_bit_slip(sw, serdes);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_width_mode(sw, serdes, width_mode);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX output",
+ sw->epl_sbus->name, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x07);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_init_signal_ok(sw, serdes, 0);
+ if (error)
+ goto done;
+
+ if (near_loopback) {
+ /* enable near loopback */
+ error = fm10k_epl_serdes_configure_near_loopback(lane);
+ if (error)
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error = 0;
+
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /* is this delay necessary? in some places the IES SDK does this */
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting bit rate",
+ sw->epl_sbus->name, serdes);
+
+ data = divider & 0x7ff;
+ data |= (1 << 12);
+ data |= (1 << 15);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x05, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting width mode to %u",
+ sw->epl_sbus->name, serdes, width_mode);
+
+ data = FM10K_SW_SERDES_WIDTH_20 | (FM10K_SW_SERDES_WIDTH_20 << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+ data = width_mode | (width_mode << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+done:
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ return error;
+}
+
+static int
+fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursor %u to %u",
+ sw->epl_sbus->name, serdes, which, tx_eq);
+
+ data = (tx_eq & 0xff) | (which << 14);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x15, data);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+void
+fm10k_epl_serdes_set_signal_detect(struct fm10k_ext_port_lane *lane,
+ unsigned int override)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int eplno = lane->port->eplno;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal detect to %s",
+ sw->epl_sbus->name, lane->abs_laneno,
+ (override == 0) ? "NORMAL" :
+ (override == 1) ? "FORCE_GOOD" :
+ (override == 2) ? "FORCE_BAD" : "<unknown>");
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno));
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE,
+ override, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno), data);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ unsigned int eplno, lane;
+ uint32_t pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl width mode",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ if (width_mode == FM10K_SW_SERDES_WIDTH_40) {
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ } else {
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ }
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static int
+fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes)
+{
+ unsigned int eplno, lane;
+ int error;
+ uint32_t int_return, pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl bit slip",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x0c, (1 << 7), &int_return);
+ if (error)
+ goto done;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ if (int_return & 0x1)
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0c, (1 << 8));
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold)
+{
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal OK threshold to %u",
+ sw->epl_sbus->name, serdes, threshold);
+
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x20, 0x20, NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_read_modify_write(sw, serdes, 0x080,
+ (threshold & 0xf) << 2, 0x3c, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t result;
+ int error;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < timeout_ms) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x02, data, &result);
+ if (result == 0x02) {
+ error = 0;
+ break;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ }
+
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x8000 | (addr & 0xff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), result);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x1a, 0x00, result);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr,
+ uint32_t data, uint32_t mask, uint32_t *result)
+{
+ int error;
+ uint32_t read_data;
+
+ error = fm10k_epl_serdes_dma_esb_read(sw, serdes, addr, &read_data);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_write(sw, serdes, addr,
+ (data & mask) | (read_data & ~mask));
+ if (error)
+ goto done;
+ if (result)
+ *result = (data & mask) | (read_data & ~mask);
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param)
+{
+ int error;
+ uint32_t result;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, int_num, param, &result);
+ if (error)
+ goto done;
+ if (result != int_num) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+static void
+fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane)
+{
+ if (sw) {
+ *eplno = serdes / FM10K_SW_EPL_LANES;
+ *lane = serdes % FM10K_SW_EPL_LANES;
+ }
+}
+
+static int
+fm10k_epl_serdes_configure_near_loopback(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring near loopback",
+ sw->epl_sbus->name, serdes);
+
+ /* assuming no lane polarity has been set */
+ /* set near loopback mode */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ data = (1 << 8) | (1 << 0);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x08, data);
+ if (error)
+ goto done;
+
+ /* disable tx output, leave tx/ex enabled */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /* set signal detect override to normal */
+ fm10k_epl_serdes_set_signal_detect(lane, FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring serdes for dfe tuning",
+ sw->epl_sbus->name, serdes);
+
+ /*
+ * Configure the serdes to run DFE tuning
+ */
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (0 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_HF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (1 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_LF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (2 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_DC & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (3 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_BW & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ /* allow early link up */
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26, 0x5b01, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning initial calibration (ical)
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe ical",
+ sw->epl_sbus->name, serdes);
+
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_A25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_A10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x01);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_ical_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status, converged;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ if ((dfe_status & 0x11) == 0x11) {
+ *status = FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS;
+ } else {
+ error = fm10k_epl_serdes_get_ical_result(sw,
+ serdes, &converged);
+ if (error)
+ goto done;
+ if (converged)
+ *status = FM10K_SW_SERDES_DFE_ICAL_CONVERGED;
+ else
+ *status = FM10K_SW_SERDES_DFE_ICAL_FAILED;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning periodic calibration (pcal), just one
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe pcal",
+ sw->epl_sbus->name, serdes);
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_B25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_B10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x02);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_pcal_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+ if (dfe_status & 0x02)
+ *status = FM10K_SW_SERDES_DFE_PCAL_COMPLETE;
+ else
+ *status = FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_stop_dfe_tuning(struct fm10k_ext_port_lane *lane,
+ unsigned int force_reset)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ unsigned int ical_stopped, pcal_stopped;
+ unsigned int i;
+ uint32_t dfe_status, status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ ical_stopped = 0;
+ pcal_stopped = 0;
+
+ if (!force_reset) {
+ if (dfe_status & 0x3) {
+ /* stop all DFE tuning */
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes,
+ 0x0a, 0);
+ if (error)
+ goto done;
+
+ for (i = 0;
+ i < FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES;
+ i++) {
+ error = fm10k_epl_serdes_dfe_ical_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ ical_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+
+ if (ical_stopped)
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES;
+ i++) {
+ error =
+ fm10k_epl_serdes_dfe_pcal_status
+ (lane, &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 0;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+
+ if (ical_stopped && pcal_stopped && (dfe_status & 0x40)) {
+ error = fm10k_epl_serdes_start_dfe_pcal(lane);
+ if (error)
+ goto done;
+ pcal_stopped = 0;
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES; i++) {
+ error = fm10k_epl_serdes_dfe_pcal_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+ }
+
+ if (!ical_stopped || !pcal_stopped) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x18, 0x07);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param & 0xffff);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param >> 16);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126, (0x0b << 8),
+ status);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: dfe status: "
+ "coarse_active=%u fine_active=%u adaptive=%u",
+ sw->epl_sbus->name, serdes, *status & (1 << 0),
+ *status & (1 << 1), *status & (1 << 6));
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged)
+{
+ uint32_t val1;
+ uint32_t val2;
+ uint32_t diff;
+ int error;
+
+ *converged = 0;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12), &val1);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12) | (1 << 8), &val2);
+ if (error)
+ goto done;
+
+ /* sign extension */
+ if (val1 & 0x8000)
+ val1 |= 0xffff0000;
+ if (val2 & 0x8000)
+ val2 |= 0xffff0000;
+ diff = abs(val2 - val1);
+
+ if (diff >= FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD)
+ *converged = 1;
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.h b/drivers/net/fm10k/switch/fm10k_serdes.h
new file mode 100644
index 0000000..f13ad3a
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SERDES_H_
+#define _FM10K_SW_SERDES_H_
+
+
+#define FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_ICAL_CONVERGED 1
+#define FM10K_SW_SERDES_DFE_ICAL_FAILED 2
+#define FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_PCAL_COMPLETE 1
+
+
+int fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw);
+int fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_ical_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_pcal_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_stop_dfe_tuning
+ (struct fm10k_ext_port_lane *lane, unsigned int force_reset);
+void fm10k_epl_serdes_set_signal_detect
+ (struct fm10k_ext_port_lane *lane, unsigned int override);
+
+#endif /* _FM10K_SW_SERDES_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sm.c b/drivers/net/fm10k/switch/fm10k_sm.c
new file mode 100644
index 0000000..26fe7dd
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.c
@@ -0,0 +1,190 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <rte_malloc.h>
+
+#include "../base/fm10k_osdep.h"
+#include "fm10k_debug.h"
+#include "fm10k_sm.h"
+
+static void *fm10k_sm_handle_event(void *ptr);
+static void *fm10k_sm_handle_timer(void *ptr);
+static void fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap);
+static uint16_t fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap);
+static void fm10k_sm_event_heap_push
+ (struct fm10k_sm_event_heap *heap, uint16_t id);
+
+
+struct fm10k_sm *
+fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx)
+{
+ struct fm10k_sm *sm;
+ struct fm10k_sm_timer *timer;
+ unsigned int i;
+
+ sm = (struct fm10k_sm *)rte_zmalloc("fm10k_sm",
+ sizeof(struct fm10k_sm), 0);
+ if (sm == NULL)
+ goto fail;
+
+ sm->is_running = true;
+ sm->num_events = num_events;
+ fm10k_sm_event_heap_init(&sm->events);
+ pthread_create(&sm->event_task, NULL, fm10k_sm_handle_event, sm);
+ sm->num_timers = num_timers;
+ if (sm->num_timers > 0) {
+ sm->timers = (struct fm10k_sm_timer *)rte_zmalloc
+ ("fm10k_sm_timers",
+ sizeof(struct fm10k_sm_timer) * sm->num_timers, 0);
+ if (sm->timers == NULL)
+ goto fail;
+ }
+
+ sm->event_handler = event_handler;
+ sm->timer_handler = timer_handler;
+ sm->ctx = ctx;
+
+ for (i = 0; i < sm->num_timers; i++) {
+ timer = &sm->timers[i];
+ sem_init(&timer->tq, 0, 0);
+ timer->sm = sm;
+ pthread_create(&timer->t, NULL, fm10k_sm_handle_timer, timer);
+ timer->timer_id = i;
+ }
+
+ return sm;
+
+fail:
+ if (sm)
+ fm10k_sm_detach(sm);
+
+ return NULL;
+}
+
+void
+fm10k_sm_detach(struct fm10k_sm *sm)
+{
+ int i;
+
+ sm->is_running = false;
+ pthread_join(sm->event_task, NULL);
+ for (i = 0; i < sm->num_timers; i++)
+ pthread_join(sm->timers[i].t, NULL);
+ rte_free(sm);
+}
+
+void
+fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id)
+{
+ fm10k_sm_event_heap_push(&sm->events, event_id);
+}
+
+void
+fm10k_sm_timer_start(struct fm10k_sm *sm, uint16_t timer_id,
+ unsigned int timeout_ms)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+
+ gettimeofday(&timer->start, 0);
+ timer->timeout_ms = timeout_ms;
+ timer->running = 1;
+ sem_post(&timer->tq);
+}
+
+/*
+ * It is possible that an expiration events from this timer will
+ * occur after this is called.
+ */
+void
+fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+ timer->running = 0;
+}
+
+static void *
+fm10k_sm_handle_event(void *ctx)
+{
+ struct fm10k_sm *sm = ctx;
+ uint16_t event_id;
+
+ while (sm->is_running) {
+ event_id = fm10k_sm_event_heap_pop(&sm->events);
+ sm->event_handler(sm, event_id);
+ }
+ return NULL;
+}
+
+static void *
+fm10k_sm_handle_timer(void *ctx)
+{
+ struct fm10k_sm_timer *timer = ctx;
+ struct fm10k_sm *sm = timer->sm;
+ struct timeval now;
+ struct timeval sleep;
+ uint64_t cost, timeout;
+
+ while (sm->is_running) {
+ sem_wait(&timer->tq);
+ if (timer->running) {
+ gettimeofday(&now, 0);
+ cost = (now.tv_sec - timer->start.tv_sec) * 1000000;
+ cost += (now.tv_usec - timer->start.tv_usec);
+ if (cost < timer->timeout_ms * 1000) {
+ timeout = timer->timeout_ms * 1000;
+ timeout -= cost;
+ sleep.tv_sec = timeout / 1000000;
+ sleep.tv_usec = timeout % 1000000;
+ select(0, NULL, NULL, NULL, &sleep);
+ }
+ if (timer->running)
+ sm->timer_handler(sm, timer->timer_id);
+ }
+ }
+
+ return NULL;
+}
+
+
+static void
+fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap)
+{
+ heap->count = 0;
+ pthread_mutex_init(&heap->lock, NULL);
+ sem_init(&heap->s, 0, 0);
+}
+
+static uint16_t
+fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap)
+{
+ int i;
+ uint16_t id;
+
+ sem_wait(&heap->s);
+ pthread_mutex_lock(&heap->lock);
+ id = heap->heap[0];
+ for (i = 1; i < heap->count; i++)
+ heap->heap[i - 1] = heap->heap[i];
+ heap->count--;
+ pthread_mutex_unlock(&heap->lock);
+ return id;
+}
+
+static void
+fm10k_sm_event_heap_push(struct fm10k_sm_event_heap *heap, uint16_t id)
+{
+ pthread_mutex_lock(&heap->lock);
+ if (heap->count >= FM10K_SM_ID_HEAP_MAX) {
+ pthread_mutex_unlock(&heap->lock);
+ return;
+ }
+ heap->heap[heap->count] = id;
+ heap->count++;
+ pthread_mutex_unlock(&heap->lock);
+ sem_post(&heap->s);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sm.h b/drivers/net/fm10k/switch/fm10k_sm.h
new file mode 100644
index 0000000..8daadfa
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.h
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SM_H_
+#define _FM10K_SW_SM_H_
+
+#include <stdint.h>
+
+#include <signal.h>
+#include <sys/time.h>
+#include <pthread.h>
+#include <semaphore.h>
+#include <stdbool.h>
+
+
+/*
+ * This framework provides state machine implementation support for use in
+ * the rest of the driver.
+ *
+ * The model is that for a given state machine, there are N>0 events
+ * identified by the integers [0..N-1] and there are M>=0 timers identified
+ * by the integers [0..M-1].
+ *
+ * The timers are one-shot timers.
+ *
+ * Since all state machine processing occurs in the context of the
+ * task queue, if that task queue was created with only a single thread, then
+ * no locking is required in the provided event and timer handlers to
+ * prevent overlapping execution.
+ *
+ */
+
+struct fm10k_sm;
+
+struct fm10k_sm_timer {
+ struct fm10k_sm *sm;
+ pthread_t t;
+ sem_t tq;
+ struct timeval start;
+ uint16_t timeout_ms;
+ uint16_t timer_id;
+ uint8_t running;
+};
+
+typedef void (*fm10k_sm_process_event_t)(struct fm10k_sm *sm, uint16_t event);
+typedef void (*fm10k_sm_process_timer_t)(struct fm10k_sm *sm, uint16_t timer);
+
+#define FM10K_SM_ID_HEAP_MAX 128
+struct fm10k_sm_event_heap {
+ pthread_mutex_t lock;
+ sem_t s;
+ uint16_t count;
+ uint16_t heap[FM10K_SM_ID_HEAP_MAX];
+};
+
+struct fm10k_sm {
+ bool is_running;
+ pthread_t event_task;
+ struct fm10k_sm_timer *timers;
+ struct fm10k_sm_event_heap events;
+ fm10k_sm_process_event_t event_handler;
+ fm10k_sm_process_timer_t timer_handler;
+ void *ctx;
+ unsigned int state;
+ uint16_t num_events;
+ uint16_t num_timers;
+ uint8_t portno;
+ uint8_t laneno;
+};
+
+struct fm10k_sm *fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx);
+void fm10k_sm_detach(struct fm10k_sm *sm);
+void fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id);
+void fm10k_sm_timer_start(struct fm10k_sm *sm,
+ uint16_t timer_id, unsigned int timeout_ms);
+void fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id);
+
+#endif /* _FM10K_SW_SM_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.c b/drivers/net/fm10k/switch/fm10k_spico_code.c
new file mode 100644
index 0000000..cfab632
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.c
@@ -0,0 +1,2966 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include "fm10k_spico_code.h"
+
+
+/* Production versions */
+
+
+/* SBus master code (production version)
+ * source file: release_2015_01_14/sbus_master.0x1013_0001.rom
+ */
+
+const uint32_t fm10000_sbus_master_code_version_build_id_prd = 0x10130001;
+const uint32_t fm10000_sbus_master_code_size_prd = 3338;
+
+const uint16_t fm10000_sbus_master_code_prd[] = {
+ 0x132, 0x002, 0x006, 0x000, 0x390, 0x002, 0x080, 0x247,
+ 0x071, 0x247, 0x01f, 0x247, 0x020, 0x247, 0x02e, 0x247,
+ 0x02f, 0x247, 0x03f, 0x247, 0x040, 0x247, 0x042, 0x247,
+ 0x043, 0x247, 0x03d, 0x0c0, 0x02d, 0x247, 0x031, 0x247,
+ 0x032, 0x081, 0x18b, 0x247, 0x039, 0x248, 0x03a, 0x181,
+ 0x247, 0x034, 0x248, 0x036, 0x209, 0x247, 0x033, 0x248,
+ 0x035, 0x305, 0x034, 0x247, 0x037, 0x248, 0x038, 0x190,
+ 0x247, 0x03b, 0x248, 0x03c, 0x3d4, 0x1ff, 0x00c, 0x3d5,
+ 0x100, 0x00c, 0x3d4, 0x2ff, 0x00c, 0x3c7, 0x0fe, 0x018,
+ 0x0c2, 0x019, 0x044, 0x233, 0x000, 0x152, 0x01b, 0x347,
+ 0x01b, 0x030, 0x085, 0x187, 0x247, 0x03e, 0x3c7, 0x100,
+ 0x041, 0x080, 0x247, 0x065, 0x3c4, 0x3fc, 0x010, 0x3d4,
+ 0x22f, 0x00e, 0x347, 0x033, 0x008, 0x247, 0x008, 0x081,
+ 0x189, 0x227, 0x2c7, 0x044, 0x220, 0x2c7, 0x045, 0x220,
+ 0x2c7, 0x046, 0x220, 0x2c7, 0x047, 0x220, 0x2c7, 0x048,
+ 0x220, 0x2c7, 0x049, 0x220, 0x2c7, 0x04a, 0x220, 0x2c7,
+ 0x04b, 0x3a7, 0x05b, 0x1a7, 0x3a5, 0x08d, 0x2c7, 0x04c,
+ 0x0c0, 0x04d, 0x044, 0x07d, 0x001, 0x0df, 0x02c, 0x000,
+ 0x009, 0x322, 0x02c, 0x009, 0x005, 0x347, 0x02d, 0x02c,
+ 0x3d4, 0x3df, 0x00e, 0x307, 0x02c, 0x384, 0x01f, 0x229,
+ 0x220, 0x3b0, 0x0aa, 0x287, 0x199, 0x390, 0x000, 0x041,
+ 0x040, 0x045, 0x020, 0x12c, 0x000, 0x020, 0x163, 0x000,
+ 0x020, 0x175, 0x000, 0x020, 0x27e, 0x000, 0x020, 0x362,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x1de, 0x000, 0x020, 0x330, 0x001, 0x020, 0x270, 0x001,
+ 0x020, 0x2af, 0x001, 0x020, 0x2a1, 0x001, 0x020, 0x338,
+ 0x001, 0x020, 0x285, 0x001, 0x020, 0x2b9, 0x001, 0x020,
+ 0x2a8, 0x001, 0x020, 0x001, 0x002, 0x020, 0x105, 0x001,
+ 0x020, 0x19e, 0x001, 0x020, 0x3e6, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000,
+ 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x044, 0x140, 0x000, 0x054, 0x362, 0x03d,
+ 0x000, 0x001, 0x0b0, 0x364, 0x03d, 0x001, 0x001, 0x0c6,
+ 0x307, 0x02e, 0x044, 0x187, 0x000, 0x055, 0x141, 0x02e,
+ 0x307, 0x030, 0x302, 0x02e, 0x00b, 0x374, 0x0c0, 0x02e,
+ 0x327, 0x02d, 0x000, 0x367, 0x044, 0x140, 0x000, 0x3d4,
+ 0x3df, 0x00e, 0x207, 0x009, 0x00a, 0x04f, 0x04e, 0x055,
+ 0x3d5, 0x200, 0x014, 0x000, 0x35d, 0x054, 0x000, 0x3f8,
+ 0x055, 0x081, 0x364, 0x010, 0x001, 0x001, 0x00d, 0x364,
+ 0x011, 0x002, 0x009, 0x009, 0x3c4, 0x3fe, 0x010, 0x3d4,
+ 0x3bf, 0x00e, 0x080, 0x101, 0x364, 0x010, 0x002, 0x001,
+ 0x00a, 0x364, 0x011, 0x001, 0x001, 0x006, 0x3c4, 0x3fd,
+ 0x010, 0x111, 0x045, 0x347, 0x026, 0x018, 0x347, 0x027,
+ 0x019, 0x347, 0x028, 0x01b, 0x347, 0x029, 0x01c, 0x044,
+ 0x222, 0x000, 0x0a1, 0x000, 0x31e, 0x347, 0x026, 0x018,
+ 0x347, 0x027, 0x019, 0x044, 0x233, 0x000, 0x347, 0x01b,
+ 0x028, 0x347, 0x01c, 0x029, 0x0a2, 0x000, 0x30c, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x000, 0x001, 0x016, 0x362,
+ 0x01b, 0x001, 0x001, 0x013, 0x362, 0x01b, 0x002, 0x001,
+ 0x00d, 0x362, 0x01b, 0x003, 0x001, 0x013, 0x362, 0x01b,
+ 0x004, 0x001, 0x013, 0x04e, 0x045, 0x364, 0x03d, 0x002,
+ 0x001, 0x005, 0x044, 0x01a, 0x001, 0x000, 0x3f7, 0x348,
+ 0x01b, 0x01a, 0x000, 0x3f2, 0x348, 0x01b, 0x01a, 0x000,
+ 0x3ed, 0x101, 0x247, 0x018, 0x3d7, 0x0ff, 0x019, 0x000,
+ 0x074, 0x3c7, 0x100, 0x014, 0x364, 0x004, 0x100, 0x009,
+ 0x00f, 0x044, 0x08d, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x055, 0x0c1,
+ 0x03d, 0x0bf, 0x000, 0x2b7, 0x000, 0x2bc, 0x3c7, 0x3ff,
+ 0x028, 0x087, 0x0b6, 0x0c1, 0x04e, 0x044, 0x37c, 0x000,
+ 0x347, 0x067, 0x029, 0x2c7, 0x02a, 0x322, 0x04e, 0x009,
+ 0x00c, 0x3c2, 0x016, 0x067, 0x009, 0x00c, 0x0c1, 0x028,
+ 0x0a7, 0x000, 0x298, 0x347, 0x033, 0x028, 0x000, 0x3fa,
+ 0x347, 0x034, 0x028, 0x000, 0x3f5, 0x111, 0x009, 0x3ff,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x040,
+ 0x103, 0x051, 0x042, 0x2a7, 0x001, 0x00c, 0x040, 0x280,
+ 0x051, 0x1a7, 0x101, 0x050, 0x225, 0x042, 0x280, 0x045,
+ 0x287, 0x045, 0x364, 0x005, 0x008, 0x009, 0x3fd, 0x347,
+ 0x01b, 0x00a, 0x347, 0x01c, 0x00b, 0x3c7, 0x041, 0x00c,
+ 0x020, 0x260, 0x000, 0x364, 0x005, 0x008, 0x009, 0x3fd,
+ 0x3d5, 0x100, 0x00c, 0x3c7, 0x042, 0x00c, 0x044, 0x260,
+ 0x000, 0x364, 0x005, 0x010, 0x001, 0x3fd, 0x307, 0x005,
+ 0x384, 0x007, 0x262, 0x004, 0x009, 0x009, 0x347, 0x002,
+ 0x01b, 0x347, 0x003, 0x01c, 0x045, 0x262, 0x006, 0x009,
+ 0x3ef, 0x141, 0x020, 0x0c0, 0x01b, 0x0c0, 0x01c, 0x045,
+ 0x347, 0x019, 0x00d, 0x1c7, 0x00d, 0x345, 0x018, 0x00d,
+ 0x3d5, 0x200, 0x00c, 0x364, 0x005, 0x008, 0x001, 0x3fd,
+ 0x3d4, 0x1ff, 0x00c, 0x045, 0x364, 0x005, 0x008, 0x009,
+ 0x3fd, 0x0c0, 0x00c, 0x020, 0x260, 0x000, 0x054, 0x347,
+ 0x032, 0x031, 0x307, 0x031, 0x044, 0x2ad, 0x000, 0x141,
+ 0x031, 0x307, 0x030, 0x302, 0x031, 0x00b, 0x004, 0x0c0,
+ 0x031, 0x2a7, 0x001, 0x00a, 0x347, 0x031, 0x032, 0x347,
+ 0x031, 0x070, 0x000, 0x00b, 0x307, 0x032, 0x302, 0x031,
+ 0x009, 0x3e2, 0x3c7, 0x3ff, 0x070, 0x055, 0x3d4, 0x2ff,
+ 0x00e, 0x0a3, 0x020, 0x091, 0x000, 0x044, 0x1b9, 0x000,
+ 0x362, 0x01b, 0x001, 0x001, 0x009, 0x362, 0x01b, 0x016,
+ 0x001, 0x004, 0x000, 0x00b, 0x0cf, 0x019, 0x044, 0x233,
+ 0x000, 0x307, 0x01b, 0x003, 0x004, 0x0a0, 0x045, 0x304,
+ 0x035, 0x227, 0x1bc, 0x2e2, 0x003, 0x001, 0x093, 0x2e2,
+ 0x002, 0x001, 0x046, 0x2e2, 0x001, 0x001, 0x063, 0x227,
+ 0x001, 0x03d, 0x2e4, 0x001, 0x009, 0x010, 0x2e4, 0x002,
+ 0x009, 0x019, 0x2e4, 0x004, 0x009, 0x029, 0x2e4, 0x008,
+ 0x009, 0x029, 0x000, 0x02b, 0x041, 0x0a2, 0x1a7, 0x3b5,
+ 0x03a, 0x044, 0x1b1, 0x001, 0x043, 0x3a4, 0x3fe, 0x000,
+ 0x3e1, 0x041, 0x327, 0x018, 0x041, 0x044, 0x08d, 0x001,
+ 0x247, 0x01e, 0x043, 0x2c7, 0x018, 0x044, 0x0fb, 0x001,
+ 0x043, 0x3a4, 0x3fd, 0x000, 0x3cd, 0x3a4, 0x3fb, 0x000,
+ 0x3c9, 0x3a4, 0x3f7, 0x000, 0x3c5, 0x0a1, 0x045, 0x227,
+ 0x3a4, 0x0ff, 0x197, 0x384, 0x01f, 0x001, 0x010, 0x262,
+ 0x001, 0x001, 0x00e, 0x262, 0x002, 0x001, 0x00f, 0x262,
+ 0x003, 0x001, 0x00d, 0x000, 0x3ea, 0x000, 0x3e8, 0x044,
+ 0x079, 0x002, 0x000, 0x3e3, 0x000, 0x3e1, 0x000, 0x3df,
+ 0x0bf, 0x1aa, 0x2a8, 0x224, 0x19a, 0x384, 0x003, 0x001,
+ 0x00c, 0x262, 0x001, 0x001, 0x017, 0x262, 0x002, 0x001,
+ 0x015, 0x000, 0x015, 0x0c5, 0x019, 0x247, 0x01b, 0x0a2,
+ 0x1a7, 0x3b5, 0x03a, 0x2c7, 0x01c, 0x044, 0x222, 0x000,
+ 0x000, 0x3bd, 0x000, 0x3bb, 0x000, 0x3b9, 0x000, 0x3b7,
+ 0x000, 0x3b5, 0x054, 0x307, 0x066, 0x191, 0x384, 0x07f,
+ 0x3c7, 0x0fe, 0x018, 0x0c2, 0x019, 0x044, 0x233, 0x000,
+ 0x347, 0x01b, 0x064, 0x0a7, 0x044, 0x0bf, 0x003, 0x055,
+ 0x0a4, 0x020, 0x091, 0x000, 0x186, 0x285, 0x181, 0x0a7,
+ 0x1ab, 0x324, 0x004, 0x285, 0x055, 0x364, 0x010, 0x002,
+ 0x009, 0x026, 0x364, 0x011, 0x001, 0x009, 0x029, 0x247,
+ 0x064, 0x044, 0x0cd, 0x003, 0x3c4, 0x2ff, 0x065, 0x04f,
+ 0x04e, 0x055, 0x364, 0x065, 0x100, 0x001, 0x3fa, 0x327,
+ 0x065, 0x3a4, 0x007, 0x364, 0x065, 0x040, 0x001, 0x007,
+ 0x287, 0x302, 0x04e, 0x009, 0x3ec, 0x045, 0x364, 0x011,
+ 0x001, 0x001, 0x005, 0x3c4, 0x3fd, 0x010, 0x054, 0x04e,
+ 0x000, 0x3cc, 0x309, 0x023, 0x003, 0x024, 0x347, 0x023,
+ 0x04f, 0x00b, 0x01f, 0x3d5, 0x010, 0x00e, 0x08f, 0x187,
+ 0x395, 0x0a0, 0x247, 0x017, 0x080, 0x247, 0x016, 0x0c0,
+ 0x015, 0x3c5, 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4,
+ 0x3fe, 0x014, 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e,
+ 0x347, 0x04f, 0x009, 0x020, 0x20b, 0x002, 0x307, 0x04f,
+ 0x00b, 0x028, 0x3c5, 0x002, 0x03d, 0x3d5, 0x010, 0x00e,
+ 0x387, 0x0b2, 0x187, 0x385, 0x080, 0x247, 0x017, 0x08e,
+ 0x187, 0x385, 0x0e6, 0x247, 0x016, 0x0c0, 0x015, 0x3c5,
+ 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4, 0x3fe, 0x014,
+ 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e, 0x000, 0x006,
+ 0x082, 0x208, 0x244, 0x03d, 0x0df, 0x02c, 0x0b3, 0x020,
+ 0x091, 0x000, 0x307, 0x018, 0x3a7, 0x051, 0x044, 0x061,
+ 0x001, 0x2c7, 0x061, 0x04c, 0x324, 0x061, 0x001, 0x020,
+ 0x040, 0x3b7, 0x126, 0x08b, 0x187, 0x044, 0x1c4, 0x001,
+ 0x009, 0x005, 0x3c7, 0x022, 0x01b, 0x042, 0x364, 0x01b,
+ 0x022, 0x009, 0x027, 0x04c, 0x348, 0x061, 0x061, 0x324,
+ 0x061, 0x04d, 0x151, 0x050, 0x000, 0x01c, 0x327, 0x04f,
+ 0x3a4, 0x0ff, 0x322, 0x050, 0x00f, 0x014, 0x040, 0x0aa,
+ 0x084, 0x044, 0x1c4, 0x001, 0x042, 0x362, 0x01b, 0x00a,
+ 0x009, 0x008, 0x04c, 0x325, 0x061, 0x04d, 0x141, 0x050,
+ 0x045, 0x384, 0x0ff, 0x040, 0x193, 0x220, 0x042, 0x384,
+ 0x00f, 0x041, 0x0a1, 0x111, 0x003, 0x005, 0x2a9, 0x000,
+ 0x3fc, 0x042, 0x045, 0x0a0, 0x0d0, 0x061, 0x04d, 0x101,
+ 0x151, 0x061, 0x009, 0x3fc, 0x045, 0x0c0, 0x04f, 0x0c0,
+ 0x050, 0x387, 0x051, 0x044, 0x073, 0x001, 0x044, 0x209,
+ 0x000, 0x051, 0x101, 0x051, 0x045, 0x307, 0x03f, 0x001,
+ 0x006, 0x247, 0x018, 0x000, 0x01f, 0x0a0, 0x287, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x012, 0x001, 0x012, 0x362,
+ 0x01b, 0x011, 0x009, 0x005, 0x347, 0x018, 0x040, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3eb, 0x0a0, 0x045, 0x347,
+ 0x018, 0x03f, 0x0c0, 0x019, 0x3c7, 0x070, 0x01b, 0x3c5,
+ 0x001, 0x01b, 0x044, 0x222, 0x000, 0x3d4, 0x3fe, 0x01b,
+ 0x044, 0x222, 0x000, 0x307, 0x03e, 0x044, 0x205, 0x000,
+ 0x3c5, 0x002, 0x01b, 0x044, 0x222, 0x000, 0x307, 0x03e,
+ 0x194, 0x044, 0x205, 0x000, 0x0c1, 0x019, 0x044, 0x233,
+ 0x000, 0x327, 0x01b, 0x0c0, 0x019, 0x3c5, 0x001, 0x01b,
+ 0x044, 0x222, 0x000, 0x3b5, 0x200, 0x045, 0x2c7, 0x01e,
+ 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001, 0x007, 0x041,
+ 0x044, 0x0fb, 0x001, 0x043, 0x121, 0x307, 0x030, 0x282,
+ 0x00b, 0x3f1, 0x045, 0x3b7, 0x03b, 0x307, 0x01e, 0x000,
+ 0x0b2, 0x091, 0x020, 0x362, 0x002, 0x054, 0x327, 0x06f,
+ 0x009, 0x009, 0x044, 0x08d, 0x001, 0x001, 0x080, 0x000,
+ 0x00d, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x012, 0x009,
+ 0x076, 0x044, 0x0b2, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x287, 0x001, 0x06b, 0x2c7, 0x01d, 0x327, 0x040,
+ 0x3a4, 0x0ff, 0x2c7, 0x018, 0x0c9, 0x01a, 0x044, 0x2d7,
+ 0x001, 0x044, 0x206, 0x001, 0x324, 0x035, 0x0c0, 0x01a,
+ 0x087, 0x187, 0x385, 0x06c, 0x282, 0x00b, 0x015, 0x0c1,
+ 0x01a, 0x087, 0x187, 0x385, 0x0bc, 0x282, 0x00b, 0x00c,
+ 0x0c2, 0x01a, 0x088, 0x187, 0x08c, 0x282, 0x00b, 0x004,
+ 0x0c3, 0x01a, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x010, 0x041, 0x307, 0x01a, 0x18b, 0x305, 0x042, 0x385,
+ 0x0e0, 0x3b7, 0x33b, 0x044, 0x1b1, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3e8, 0x307, 0x03e, 0x044,
+ 0x205, 0x000, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x00b, 0x041, 0x307, 0x01d, 0x3b7, 0x03b, 0x044, 0x1b1,
+ 0x001, 0x043, 0x121, 0x307, 0x030, 0x282, 0x00b, 0x3ed,
+ 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387, 0x3ff, 0x3c7,
+ 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296, 0x001, 0x347,
+ 0x023, 0x043, 0x092, 0x020, 0x365, 0x002, 0x054, 0x044,
+ 0x1de, 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296,
+ 0x001, 0x040, 0x044, 0x1c9, 0x001, 0x001, 0x00c, 0x042,
+ 0x0c5, 0x019, 0x2c7, 0x01c, 0x247, 0x01b, 0x020, 0x222,
+ 0x000, 0x042, 0x080, 0x045, 0x044, 0x1b1, 0x001, 0x001,
+ 0x3fb, 0x041, 0x3a7, 0x0ff, 0x0c6, 0x019, 0x131, 0x001,
+ 0x00c, 0x044, 0x233, 0x000, 0x30a, 0x01c, 0x002, 0x3f8,
+ 0x081, 0x000, 0x003, 0x080, 0x043, 0x045, 0x307, 0x040,
+ 0x384, 0x0ff, 0x009, 0x017, 0x0a0, 0x287, 0x044, 0x1b9,
+ 0x000, 0x362, 0x01b, 0x011, 0x001, 0x00a, 0x121, 0x307,
+ 0x030, 0x282, 0x00b, 0x3f3, 0x0a0, 0x045, 0x347, 0x018,
+ 0x040, 0x0c0, 0x01a, 0x327, 0x040, 0x044, 0x2c3, 0x001,
+ 0x324, 0x034, 0x009, 0x3f2, 0x227, 0x045, 0x247, 0x01c,
+ 0x2c7, 0x01b, 0x280, 0x001, 0x3e9, 0x0c3, 0x019, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222,
+ 0x000, 0x0c2, 0x01b, 0x044, 0x222, 0x000, 0x347, 0x01e,
+ 0x019, 0x3d0, 0x041, 0x019, 0x307, 0x041, 0x044, 0x205,
+ 0x000, 0x044, 0x233, 0x000, 0x327, 0x01b, 0x00b, 0x3f6,
+ 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x287,
+ 0x045, 0x2c7, 0x01e, 0x0a0, 0x287, 0x044, 0x300, 0x001,
+ 0x001, 0x007, 0x041, 0x044, 0x24e, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3f1, 0x045, 0x09f, 0x18a,
+ 0x0a1, 0x1aa, 0x324, 0x01e, 0x001, 0x006, 0x305, 0x01e,
+ 0x000, 0x005, 0x208, 0x304, 0x01e, 0x300, 0x043, 0x00b,
+ 0x005, 0x080, 0x000, 0x008, 0x272, 0x3e8, 0x00b, 0x004,
+ 0x397, 0x3e8, 0x305, 0x033, 0x3b7, 0x13b, 0x000, 0x343,
+ 0x327, 0x06e, 0x0c0, 0x01a, 0x044, 0x2c3, 0x001, 0x0a9,
+ 0x305, 0x033, 0x247, 0x007, 0x344, 0x035, 0x007, 0x3d4,
+ 0x37f, 0x00e, 0x020, 0x091, 0x000, 0x327, 0x06f, 0x0c0,
+ 0x01a, 0x044, 0x2c3, 0x001, 0x324, 0x034, 0x009, 0x005,
+ 0x347, 0x06f, 0x040, 0x0ad, 0x0c1, 0x008, 0x247, 0x009,
+ 0x345, 0x033, 0x008, 0x344, 0x035, 0x008, 0x020, 0x091,
+ 0x000, 0x327, 0x06e, 0x054, 0x055, 0x0ab, 0x000, 0x3d2,
+ 0x327, 0x06f, 0x054, 0x055, 0x0af, 0x000, 0x3e7, 0x327,
+ 0x06e, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001, 0x0aa, 0x000,
+ 0x3c1, 0x327, 0x06f, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001,
+ 0x0ae, 0x000, 0x3d3, 0x044, 0x2f8, 0x001, 0x362, 0x01b,
+ 0x011, 0x009, 0x00a, 0x044, 0x2d7, 0x001, 0x044, 0x206,
+ 0x001, 0x055, 0x045, 0x307, 0x034, 0x000, 0x3fc, 0x080,
+ 0x1bb, 0x2e2, 0x008, 0x00b, 0x00d, 0x0a1, 0x0c0, 0x01e,
+ 0x340, 0x01a, 0x01e, 0x001, 0x014, 0x1a8, 0x000, 0x011,
+ 0x320, 0x01a, 0x2c7, 0x01e, 0x2c7, 0x01a, 0x0a1, 0x151,
+ 0x01a, 0x003, 0x006, 0x2a9, 0x20b, 0x000, 0x3fa, 0x045,
+ 0x054, 0x287, 0x384, 0x0ff, 0x111, 0x020, 0x1b9, 0x000,
+ 0x044, 0x1b9, 0x000, 0x362, 0x01b, 0x016, 0x001, 0x007,
+ 0x362, 0x01b, 0x001, 0x009, 0x023, 0x3d7, 0x0fe, 0x019,
+ 0x044, 0x233, 0x000, 0x387, 0x0bb, 0x187, 0x385, 0x031,
+ 0x302, 0x01b, 0x001, 0x014, 0x387, 0x0b9, 0x187, 0x385,
+ 0x031, 0x302, 0x01b, 0x001, 0x00b, 0x387, 0x0b6, 0x187,
+ 0x385, 0x031, 0x302, 0x01b, 0x009, 0x003, 0x080, 0x045,
+ 0x327, 0x06e, 0x044, 0x340, 0x001, 0x0a8, 0x000, 0x342,
+ 0x327, 0x06f, 0x044, 0x340, 0x001, 0x0ac, 0x000, 0x356,
+ 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x00b, 0x009, 0x38d,
+ 0x0c0, 0x01c, 0x041, 0x1bb, 0x3a4, 0x007, 0x001, 0x047,
+ 0x131, 0x001, 0x02f, 0x131, 0x001, 0x017, 0x083, 0x3a7,
+ 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0, 0x044, 0x3f6,
+ 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x081, 0x044, 0x3f6,
+ 0x001, 0x000, 0x03e, 0x082, 0x3a7, 0x3ff, 0x044, 0x3f6,
+ 0x001, 0x084, 0x0a0, 0x044, 0x3f6, 0x001, 0x083, 0x044,
+ 0x3f6, 0x001, 0x081, 0x044, 0x3f6, 0x001, 0x000, 0x029,
+ 0x081, 0x3a7, 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0,
+ 0x044, 0x3f6, 0x001, 0x083, 0x044, 0x3f6, 0x001, 0x082,
+ 0x044, 0x3f6, 0x001, 0x000, 0x014, 0x081, 0x3a7, 0x3ff,
+ 0x044, 0x3f6, 0x001, 0x084, 0x044, 0x3f6, 0x001, 0x083,
+ 0x044, 0x3f6, 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x0c6,
+ 0x019, 0x0c0, 0x01b, 0x044, 0x222, 0x000, 0x0c5, 0x019,
+ 0x387, 0x0ff, 0x3a7, 0x3ff, 0x322, 0x04c, 0x303, 0x04d,
+ 0x2c7, 0x01b, 0x247, 0x01c, 0x044, 0x222, 0x000, 0x0c0,
+ 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x397, 0x3ff,
+ 0x044, 0x205, 0x000, 0x044, 0x233, 0x000, 0x3c4, 0x001,
+ 0x01b, 0x009, 0x3f5, 0x0c6, 0x019, 0x044, 0x233, 0x000,
+ 0x307, 0x01b, 0x043, 0x2a7, 0x003, 0x016, 0x0c0, 0x01a,
+ 0x397, 0x044, 0x04c, 0x322, 0x01b, 0x00b, 0x00b, 0x272,
+ 0x04b, 0x001, 0x007, 0x101, 0x141, 0x01a, 0x000, 0x3f4,
+ 0x307, 0x01a, 0x305, 0x033, 0x055, 0x045, 0x247, 0x019,
+ 0x2c7, 0x01b, 0x020, 0x222, 0x000, 0x090, 0x020, 0x362,
+ 0x002, 0x327, 0x06f, 0x287, 0x384, 0x0ff, 0x262, 0x0df,
+ 0x00b, 0x00a, 0x262, 0x0e7, 0x003, 0x006, 0x247, 0x018,
+ 0x000, 0x00a, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x001,
+ 0x009, 0x057, 0x0c7, 0x019, 0x0d1, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0d0, 0x01b, 0x044, 0x222, 0x000,
+ 0x0c0, 0x019, 0x0c0, 0x01b, 0x347, 0x034, 0x01c, 0x044,
+ 0x222, 0x000, 0x0ca, 0x019, 0x081, 0x189, 0x247, 0x012,
+ 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x133, 0x347, 0x013,
+ 0x01b, 0x347, 0x013, 0x01c, 0x044, 0x222, 0x000, 0x132,
+ 0x007, 0x3f6, 0x0c0, 0x019, 0x0c0, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0cb, 0x019, 0x0cc, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c7, 0x019, 0x0c2, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0c8, 0x019, 0x0c0, 0x01b, 0x044,
+ 0x222, 0x000, 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b0, 0x020, 0x296,
+ 0x001, 0x0c1, 0x019, 0x0c0, 0x01b, 0x0c1, 0x01c, 0x1cc,
+ 0x01c, 0x044, 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x054,
+ 0x01c, 0x044, 0x222, 0x000, 0x345, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x01d, 0x102, 0x050, 0x282, 0x00f,
+ 0x08f, 0x307, 0x01d, 0x104, 0x1a0, 0x280, 0x051, 0x1a7,
+ 0x101, 0x050, 0x285, 0x300, 0x01d, 0x051, 0x1a7, 0x2c7,
+ 0x01d, 0x101, 0x051, 0x2c5, 0x01d, 0x101, 0x051, 0x1a7,
+ 0x2c7, 0x01e, 0x101, 0x051, 0x2c5, 0x01e, 0x347, 0x034,
+ 0x01c, 0x227, 0x044, 0x222, 0x000, 0x287, 0x044, 0x11b,
+ 0x002, 0x044, 0x11b, 0x002, 0x347, 0x01d, 0x01b, 0x347,
+ 0x034, 0x01c, 0x044, 0x222, 0x000, 0x327, 0x01e, 0x001,
+ 0x057, 0x0ca, 0x019, 0x101, 0x052, 0x01b, 0x101, 0x051,
+ 0x041, 0x1a9, 0x2c5, 0x01b, 0x043, 0x1b5, 0x2c7, 0x01c,
+ 0x101, 0x051, 0x1a3, 0x325, 0x037, 0x2c5, 0x01c, 0x044,
+ 0x222, 0x000, 0x153, 0x01e, 0x007, 0x3e7, 0x0c0, 0x019,
+ 0x0c0, 0x01c, 0x044, 0x222, 0x000, 0x0ca, 0x01b, 0x0c5,
+ 0x019, 0x090, 0x187, 0x395, 0x03a, 0x247, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x020, 0x01b, 0x0c0,
+ 0x01c, 0x044, 0x222, 0x000, 0x347, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x045, 0x101, 0x051, 0x1a7, 0x2c7, 0x01b,
+ 0x101, 0x051, 0x2c5, 0x01b, 0x101, 0x051, 0x325, 0x037,
+ 0x2c7, 0x01c, 0x044, 0x222, 0x000, 0x045, 0x0c5, 0x01b,
+ 0x000, 0x3cf, 0x347, 0x000, 0x022, 0x347, 0x001, 0x023,
+ 0x0c0, 0x008, 0x0c0, 0x009, 0x307, 0x022, 0x384, 0x03f,
+ 0x229, 0x220, 0x3b0, 0x14b, 0x287, 0x199, 0x390, 0x002,
+ 0x041, 0x040, 0x045, 0x020, 0x21a, 0x002, 0x020, 0x223,
+ 0x002, 0x020, 0x22c, 0x002, 0x020, 0x351, 0x002, 0x020,
+ 0x281, 0x002, 0x020, 0x304, 0x002, 0x020, 0x309, 0x002,
+ 0x020, 0x30e, 0x002, 0x020, 0x313, 0x002, 0x020, 0x318,
+ 0x002, 0x020, 0x31d, 0x002, 0x020, 0x322, 0x002, 0x020,
+ 0x327, 0x002, 0x020, 0x32c, 0x002, 0x020, 0x331, 0x002,
+ 0x020, 0x336, 0x002, 0x020, 0x339, 0x002, 0x020, 0x33f,
+ 0x002, 0x020, 0x344, 0x002, 0x020, 0x348, 0x002, 0x020,
+ 0x354, 0x002, 0x020, 0x358, 0x002, 0x020, 0x361, 0x002,
+ 0x020, 0x36a, 0x002, 0x020, 0x36d, 0x002, 0x020, 0x370,
+ 0x002, 0x020, 0x2f1, 0x002, 0x020, 0x373, 0x002, 0x020,
+ 0x384, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x38e, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x3fd, 0x001, 0x020, 0x101,
+ 0x001, 0x020, 0x197, 0x001, 0x020, 0x3ba, 0x000, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x0c1, 0x008, 0x345, 0x033, 0x008,
+ 0x344, 0x035, 0x008, 0x04e, 0x047, 0x3c7, 0x3ff, 0x008,
+ 0x000, 0x3f8, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7,
+ 0x009, 0x000, 0x3ea, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001,
+ 0x2c7, 0x009, 0x000, 0x3e1, 0x080, 0x3b7, 0x003, 0x1a9,
+ 0x3b5, 0x105, 0x2c7, 0x023, 0x0c0, 0x028, 0x0c0, 0x029,
+ 0x052, 0x022, 0x044, 0x2d6, 0x002, 0x001, 0x005, 0x101,
+ 0x000, 0x3f8, 0x101, 0x052, 0x022, 0x346, 0x028, 0x022,
+ 0x3c4, 0x0ff, 0x022, 0x362, 0x022, 0x0ef, 0x009, 0x02e,
+ 0x101, 0x052, 0x022, 0x327, 0x028, 0x1b7, 0x2c6, 0x022,
+ 0x362, 0x022, 0x0be, 0x009, 0x021, 0x101, 0x052, 0x022,
+ 0x346, 0x029, 0x022, 0x3c4, 0x0ff, 0x022, 0x362, 0x022,
+ 0x0ad, 0x009, 0x013, 0x101, 0x052, 0x022, 0x327, 0x029,
+ 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0de, 0x009, 0x006,
+ 0x0c1, 0x009, 0x000, 0x391, 0x3c7, 0x3ff, 0x009, 0x000,
+ 0x38c, 0x081, 0x189, 0x247, 0x012, 0x0c0, 0x028, 0x0c0,
+ 0x029, 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x2c7, 0x023,
+ 0x247, 0x012, 0x080, 0x347, 0x013, 0x022, 0x044, 0x2d6,
+ 0x002, 0x001, 0x005, 0x101, 0x000, 0x3f7, 0x347, 0x013,
+ 0x022, 0x346, 0x028, 0x022, 0x3c4, 0x0ff, 0x022, 0x362,
+ 0x022, 0x0ef, 0x009, 0x3d2, 0x347, 0x013, 0x022, 0x327,
+ 0x028, 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0be, 0x009,
+ 0x3c5, 0x347, 0x013, 0x022, 0x346, 0x029, 0x022, 0x3c4,
+ 0x0ff, 0x022, 0x362, 0x022, 0x0ad, 0x009, 0x3b7, 0x347,
+ 0x013, 0x022, 0x327, 0x029, 0x1b7, 0x2c6, 0x022, 0x362,
+ 0x022, 0x0de, 0x009, 0x3aa, 0x000, 0x3a4, 0x340, 0x022,
+ 0x028, 0x3c1, 0x000, 0x029, 0x3c6, 0x0d8, 0x028, 0x349,
+ 0x028, 0x028, 0x34b, 0x029, 0x029, 0x00a, 0x008, 0x3c0,
+ 0x0e5, 0x028, 0x3c1, 0x000, 0x029, 0x327, 0x023, 0x222,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x051,
+ 0x1a7, 0x2c7, 0x023, 0x101, 0x051, 0x2c5, 0x023, 0x111,
+ 0x240, 0x023, 0x000, 0x332, 0x347, 0x023, 0x026, 0x000,
+ 0x304, 0x347, 0x023, 0x027, 0x000, 0x2ff, 0x347, 0x023,
+ 0x028, 0x000, 0x2fa, 0x347, 0x023, 0x029, 0x000, 0x2f5,
+ 0x347, 0x023, 0x02a, 0x000, 0x2f0, 0x347, 0x023, 0x02b,
+ 0x000, 0x2eb, 0x347, 0x028, 0x009, 0x000, 0x2e6, 0x347,
+ 0x029, 0x009, 0x000, 0x2e1, 0x347, 0x02a, 0x009, 0x000,
+ 0x2dc, 0x347, 0x02b, 0x009, 0x000, 0x2d7, 0x347, 0x023,
+ 0x02d, 0x055, 0x347, 0x023, 0x02c, 0x000, 0x2ce, 0x347,
+ 0x023, 0x021, 0x000, 0x2c9, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x327, 0x023, 0x04d, 0x000,
+ 0x2bc, 0x347, 0x023, 0x021, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x04c, 0x2c7, 0x009, 0x000,
+ 0x2ac, 0x08c, 0x347, 0x023, 0x06f, 0x055, 0x247, 0x02c,
+ 0x000, 0x2ab, 0x08d, 0x000, 0x3f7, 0x08e, 0x000, 0x3f4,
+ 0x08f, 0x000, 0x3f1, 0x307, 0x022, 0x183, 0x19b, 0x262,
+ 0x009, 0x00b, 0x007, 0x347, 0x037, 0x008, 0x000, 0x292,
+ 0x390, 0x044, 0x000, 0x3ca, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x009, 0x000, 0x27f, 0x000, 0x27d,
+ 0x364, 0x004, 0x002, 0x009, 0x090, 0x364, 0x004, 0x008,
+ 0x009, 0x165, 0x364, 0x004, 0x004, 0x009, 0x08e, 0x0c0,
+ 0x007, 0x3d5, 0x080, 0x00e, 0x347, 0x006, 0x062, 0x0c0,
+ 0x064, 0x307, 0x062, 0x197, 0x384, 0x00f, 0x229, 0x220,
+ 0x3b0, 0x3ca, 0x287, 0x199, 0x390, 0x002, 0x041, 0x040,
+ 0x045, 0x345, 0x033, 0x064, 0x347, 0x064, 0x007, 0x344,
+ 0x035, 0x007, 0x3d4, 0x37f, 0x00e, 0x3c5, 0x020, 0x00e,
+ 0x04e, 0x047, 0x020, 0x3ff, 0x002, 0x020, 0x00a, 0x003,
+ 0x020, 0x011, 0x003, 0x020, 0x01a, 0x003, 0x020, 0x01d,
+ 0x003, 0x020, 0x020, 0x003, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002,
+ 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa,
+ 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x345, 0x037, 0x064, 0x000, 0x3bf, 0x3b7,
+ 0x010, 0x1a7, 0x3a5, 0x013, 0x324, 0x038, 0x2c7, 0x064,
+ 0x000, 0x3b1, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001, 0x000,
+ 0x3f5, 0x088, 0x055, 0x247, 0x02c, 0x347, 0x062, 0x06e,
+ 0x000, 0x3ad, 0x089, 0x000, 0x3f7, 0x08a, 0x000, 0x3f4,
+ 0x08b, 0x000, 0x3f1, 0x055, 0x0c3, 0x02c, 0x3d5, 0x100,
+ 0x00e, 0x000, 0x39c, 0x347, 0x011, 0x066, 0x3c4, 0x3fc,
+ 0x066, 0x364, 0x065, 0x040, 0x009, 0x0a3, 0x044, 0x085,
+ 0x003, 0x001, 0x06f, 0x044, 0x06b, 0x003, 0x001, 0x005,
+ 0x055, 0x000, 0x384, 0x044, 0x074, 0x003, 0x3c4, 0x37f,
+ 0x065, 0x1b5, 0x3a4, 0x038, 0x3c4, 0x3c7, 0x065, 0x2c5,
+ 0x065, 0x1b2, 0x2e2, 0x007, 0x001, 0x047, 0x2e2, 0x006,
+ 0x001, 0x043, 0x2e2, 0x002, 0x001, 0x047, 0x2e2, 0x001,
+ 0x001, 0x031, 0x3d7, 0x07f, 0x064, 0x0a6, 0x044, 0x0bf,
+ 0x003, 0x000, 0x35c, 0x364, 0x010, 0x002, 0x009, 0x005,
+ 0x364, 0x011, 0x001, 0x045, 0x3d5, 0x040, 0x00e, 0x3c5,
+ 0x001, 0x010, 0x3c4, 0x3bf, 0x065, 0x327, 0x066, 0x00b,
+ 0x005, 0x3c5, 0x040, 0x065, 0x045, 0x327, 0x004, 0x307,
+ 0x066, 0x198, 0x384, 0x038, 0x1b8, 0x3a4, 0x038, 0x222,
+ 0x045, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7, 0x064,
+ 0x0a7, 0x000, 0x3cd, 0x347, 0x066, 0x064, 0x044, 0x0cd,
+ 0x003, 0x000, 0x324, 0x055, 0x0c4, 0x02c, 0x000, 0x31f,
+ 0x044, 0x074, 0x003, 0x3c5, 0x080, 0x065, 0x3c4, 0x3f8,
+ 0x065, 0x3a4, 0x1fc, 0x081, 0x245, 0x065, 0x1b1, 0x390,
+ 0x066, 0x04d, 0x3c5, 0x100, 0x065, 0x000, 0x308, 0x3c4,
+ 0x07f, 0x064, 0x1c1, 0x064, 0x307, 0x066, 0x198, 0x384,
+ 0x038, 0x285, 0x188, 0x245, 0x064, 0x3c4, 0x003, 0x010,
+ 0x345, 0x064, 0x010, 0x3c5, 0x002, 0x010, 0x045, 0x044,
+ 0x085, 0x003, 0x009, 0x009, 0x307, 0x065, 0x364, 0x065,
+ 0x080, 0x009, 0x00c, 0x044, 0x06b, 0x003, 0x009, 0x35a,
+ 0x044, 0x074, 0x003, 0x000, 0x3b0, 0x044, 0x074, 0x003,
+ 0x384, 0x007, 0x262, 0x007, 0x001, 0x2d1, 0x141, 0x065,
+ 0x101, 0x324, 0x035, 0x000, 0x3bb, 0x3d5, 0x010, 0x00e,
+ 0x055, 0x0d3, 0x02c, 0x000, 0x2c2, 0x04e, 0x062, 0x0b1,
+ 0x0e7, 0x079
+
+}; /* end fm10000_sbus_master_code_prd */
+
+/* Spico SerDes code (production version 2)
+ * source file: serdes.0x2055/serdes.0x2055_0045.rom
+ * KR support
+ */
+const uint32_t fm10000_serdes_spico_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_spico_code_size_prd2 = 12167;
+
+const uint16_t fm10000_serdes_spico_code_prd2[] = {
+ 0x035, 0x006, 0x00a, 0x000, 0x1a7, 0x006, 0x020, 0x055,
+ 0x000, 0x045, 0x3c7, 0x03f, 0x000, 0x0c0, 0x200, 0x307,
+ 0x200, 0x304, 0x02b, 0x009, 0x00b, 0x0a0, 0x397, 0x3bb,
+ 0x04d, 0x111, 0x272, 0x1ff, 0x009, 0x3fc, 0x3c8, 0x000,
+ 0x21e, 0x3ce, 0x3ff, 0x21f, 0x348, 0x21f, 0x21d, 0x34e,
+ 0x21d, 0x221, 0x34e, 0x221, 0x21c, 0x34e, 0x21c, 0x220,
+ 0x3c7, 0x3fd, 0x23d, 0x3c7, 0x020, 0x23e, 0x3c7, 0x077,
+ 0x22d, 0x0c0, 0x202, 0x044, 0x36d, 0x003, 0x044, 0x360,
+ 0x002, 0x3d7, 0x303, 0x030, 0x347, 0x032, 0x236, 0x3b7,
+ 0x213, 0x090, 0x044, 0x1a9, 0x000, 0x3b7, 0x204, 0x387,
+ 0x07c, 0x044, 0x1a9, 0x000, 0x0a4, 0x397, 0x27c, 0x044,
+ 0x1a9, 0x000, 0x3d5, 0x230, 0x232, 0x345, 0x21d, 0x232,
+ 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000, 0x3b7,
+ 0x080, 0x044, 0x192, 0x000, 0x385, 0x00c, 0x384, 0x3cf,
+ 0x044, 0x1bd, 0x000, 0x3c5, 0x040, 0x021, 0x3b7, 0x020,
+ 0x387, 0x021, 0x189, 0x044, 0x1a9, 0x000, 0x0c0, 0x008,
+ 0x0c0, 0x005, 0x0c0, 0x00b, 0x0c0, 0x000, 0x3c5, 0x020,
+ 0x027, 0x055, 0x307, 0x221, 0x304, 0x233, 0x009, 0x012,
+ 0x374, 0x027, 0x200, 0x001, 0x00d, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x009, 0x006, 0x3b8, 0x200, 0x2c4, 0x027,
+ 0x054, 0x364, 0x200, 0x004, 0x029, 0x3de, 0x003, 0x364,
+ 0x33b, 0x010, 0x029, 0x3de, 0x003, 0x054, 0x364, 0x200,
+ 0x040, 0x029, 0x034, 0x008, 0x364, 0x2cf, 0x040, 0x009,
+ 0x01c, 0x307, 0x026, 0x384, 0x003, 0x266, 0x003, 0x009,
+ 0x014, 0x362, 0x26b, 0x0e0, 0x009, 0x00f, 0x364, 0x234,
+ 0x002, 0x001, 0x00a, 0x3c5, 0x060, 0x2cf, 0x0c0, 0x26b,
+ 0x3c5, 0x040, 0x200, 0x054, 0x364, 0x200, 0x100, 0x029,
+ 0x22f, 0x003, 0x054, 0x044, 0x141, 0x001, 0x055, 0x364,
+ 0x23f, 0x003, 0x001, 0x012, 0x044, 0x34c, 0x001, 0x029,
+ 0x0af, 0x006, 0x044, 0x340, 0x001, 0x227, 0x326, 0x249,
+ 0x1ba, 0x029, 0x0af, 0x006, 0x3c4, 0x3df, 0x000, 0x364,
+ 0x33c, 0x020, 0x001, 0x017, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x055, 0x364, 0x200, 0x080, 0x001, 0x00d, 0x3c4,
+ 0x3fd, 0x26f, 0x3c4, 0x3fe, 0x200, 0x366, 0x200, 0x080,
+ 0x001, 0x00f, 0x140, 0x200, 0x009, 0x365, 0x364, 0x23f,
+ 0x003, 0x009, 0x360, 0x04f, 0x04e, 0x000, 0x35c, 0x3c5,
+ 0x030, 0x000, 0x347, 0x244, 0x054, 0x374, 0x000, 0x2c0,
+ 0x009, 0x34c, 0x387, 0x0c1, 0x187, 0x245, 0x054, 0x04f,
+ 0x055, 0x141, 0x26c, 0x347, 0x245, 0x200, 0x347, 0x245,
+ 0x26d, 0x0c0, 0x245, 0x364, 0x026, 0x002, 0x001, 0x336,
+ 0x364, 0x020, 0x005, 0x009, 0x007, 0x0c2, 0x26f, 0x044,
+ 0x1b8, 0x001, 0x055, 0x364, 0x020, 0x005, 0x009, 0x326,
+ 0x362, 0x26b, 0x0e0, 0x009, 0x321, 0x364, 0x234, 0x002,
+ 0x001, 0x31c, 0x3c5, 0x060, 0x2cf, 0x3c5, 0x040, 0x200,
+ 0x000, 0x314, 0x044, 0x290, 0x001, 0x3c5, 0x080, 0x200,
+ 0x347, 0x2cf, 0x26b, 0x3c4, 0x0e0, 0x26b, 0x3c4, 0x3bf,
+ 0x2cf, 0x045, 0x387, 0x080, 0x020, 0x2f8, 0x001, 0x387,
+ 0x02f, 0x111, 0x009, 0x3ff, 0x045, 0x054, 0x111, 0x009,
+ 0x3fe, 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x032, 0x054,
+ 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x033, 0x054, 0x045,
+ 0x055, 0x2c7, 0x030, 0x364, 0x035, 0x001, 0x009, 0x3fd,
+ 0x045, 0x044, 0x1a0, 0x000, 0x305, 0x033, 0x000, 0x012,
+ 0x044, 0x1a0, 0x000, 0x306, 0x033, 0x000, 0x00b, 0x208,
+ 0x044, 0x1a0, 0x000, 0x304, 0x033, 0x055, 0x2c7, 0x030,
+ 0x247, 0x031, 0x345, 0x21d, 0x030, 0x140, 0x035, 0x009,
+ 0x3fe, 0x346, 0x21d, 0x030, 0x054, 0x045, 0x040, 0x140,
+ 0x036, 0x00b, 0x020, 0x044, 0x206, 0x000, 0x344, 0x21f,
+ 0x036, 0x3c4, 0x3ef, 0x036, 0x09f, 0x044, 0x189, 0x000,
+ 0x000, 0x010, 0x040, 0x140, 0x036, 0x003, 0x00c, 0x044,
+ 0x206, 0x000, 0x345, 0x21d, 0x036, 0x3c4, 0x3ef, 0x036,
+ 0x054, 0x327, 0x22b, 0x3a4, 0x003, 0x133, 0x009, 0x00e,
+ 0x0a1, 0x1aa, 0x2a8, 0x2c4, 0x021, 0x08f, 0x044, 0x189,
+ 0x000, 0x2a8, 0x2c5, 0x021, 0x042, 0x045, 0x055, 0x088,
+ 0x3c5, 0x010, 0x036, 0x000, 0x37e, 0x3c4, 0x3fe, 0x026,
+ 0x327, 0x22c, 0x3a4, 0x007, 0x2e6, 0x001, 0x001, 0x01b,
+ 0x044, 0x1e2, 0x000, 0x364, 0x234, 0x001, 0x009, 0x013,
+ 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x3b6, 0x003,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x066, 0x004,
+ 0x054, 0x045, 0x364, 0x233, 0x002, 0x009, 0x00d, 0x347,
+ 0x271, 0x270, 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x044,
+ 0x02b, 0x008, 0x374, 0x232, 0x200, 0x001, 0x0a5, 0x3d7,
+ 0x200, 0x27b, 0x044, 0x29c, 0x001, 0x044, 0x2f6, 0x002,
+ 0x009, 0x005, 0x044, 0x028, 0x008, 0x3b7, 0x221, 0x364,
+ 0x22b, 0x003, 0x009, 0x007, 0x083, 0x18b, 0x044, 0x1a9,
+ 0x000, 0x398, 0x200, 0x244, 0x232, 0x3b7, 0x221, 0x091,
+ 0x189, 0x385, 0x128, 0x044, 0x1a9, 0x000, 0x083, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x220, 0x044, 0x192, 0x000, 0x197,
+ 0x264, 0x007, 0x009, 0x005, 0x091, 0x000, 0x003, 0x081,
+ 0x3b7, 0x221, 0x189, 0x044, 0x1b7, 0x000, 0x364, 0x246,
+ 0x010, 0x009, 0x002, 0x3b7, 0x221, 0x081, 0x18d, 0x044,
+ 0x1a9, 0x000, 0x081, 0x189, 0x044, 0x1a9, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x364, 0x22b, 0x003, 0x009,
+ 0x009, 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000,
+ 0x364, 0x02b, 0x020, 0x009, 0x020, 0x0a3, 0x364, 0x22b,
+ 0x003, 0x001, 0x006, 0x044, 0x025, 0x008, 0x0a0, 0x307,
+ 0x251, 0x197, 0x384, 0x00f, 0x220, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x327, 0x251, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x044, 0x1ce, 0x000, 0x364, 0x232,
+ 0x100, 0x001, 0x005, 0x044, 0x0a9, 0x001, 0x3c4, 0x3f0,
+ 0x22d, 0x3c5, 0x001, 0x026, 0x364, 0x026, 0x002, 0x001,
+ 0x1c5, 0x044, 0x283, 0x001, 0x3b7, 0x300, 0x387, 0x020,
+ 0x000, 0x2c1, 0x307, 0x036, 0x00b, 0x3ea, 0x044, 0x1ce,
+ 0x000, 0x000, 0x3e5, 0x387, 0x3ff, 0x327, 0x22c, 0x3a4,
+ 0x007, 0x131, 0x001, 0x01a, 0x3d5, 0x200, 0x232, 0x000,
+ 0x006, 0x080, 0x131, 0x001, 0x011, 0x3b7, 0x205, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x222, 0x044, 0x1bd, 0x000, 0x3b7,
+ 0x251, 0x044, 0x1bd, 0x000, 0x3b7, 0x214, 0x100, 0x009,
+ 0x00a, 0x044, 0x1bd, 0x000, 0x3c4, 0x37b, 0x021, 0x000,
+ 0x29e, 0x3c5, 0x084, 0x021, 0x000, 0x299, 0x364, 0x233,
+ 0x002, 0x009, 0x00d, 0x347, 0x272, 0x270, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x02e, 0x008, 0x307, 0x22d,
+ 0x384, 0x070, 0x386, 0x010, 0x001, 0x04a, 0x364, 0x02b,
+ 0x020, 0x009, 0x045, 0x3b7, 0x071, 0x387, 0x128, 0x044,
+ 0x1a9, 0x000, 0x387, 0x368, 0x044, 0x1b8, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x0a1, 0x307, 0x22b, 0x384,
+ 0x030, 0x001, 0x003, 0x0ac, 0x307, 0x256, 0x384, 0x01f,
+ 0x220, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x080, 0x327,
+ 0x22b, 0x3a4, 0x030, 0x3a2, 0x030, 0x009, 0x00e, 0x0a1,
+ 0x1ab, 0x2a8, 0x2c4, 0x024, 0x08f, 0x044, 0x189, 0x000,
+ 0x2a8, 0x2c5, 0x024, 0x327, 0x256, 0x1b7, 0x3a4, 0x07f,
+ 0x220, 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x044, 0x2f6,
+ 0x002, 0x009, 0x007, 0x0c0, 0x27b, 0x044, 0x29c, 0x001,
+ 0x044, 0x0e6, 0x001, 0x3c4, 0x30f, 0x22d, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x031, 0x008, 0x345, 0x21d,
+ 0x01c, 0x307, 0x21d, 0x304, 0x233, 0x001, 0x007, 0x3a7,
+ 0x071, 0x044, 0x1a9, 0x000, 0x3c5, 0x002, 0x026, 0x044,
+ 0x141, 0x001, 0x020, 0x283, 0x001, 0x055, 0x3c4, 0x3ef,
+ 0x027, 0x3c4, 0x3fd, 0x026, 0x364, 0x26f, 0x002, 0x001,
+ 0x008, 0x3c4, 0x3fd, 0x26f, 0x044, 0x290, 0x001, 0x344,
+ 0x21f, 0x01c, 0x3c4, 0x3bf, 0x200, 0x3c4, 0x39f, 0x2cf,
+ 0x0c0, 0x2d8, 0x0c0, 0x26b, 0x364, 0x234, 0x002, 0x009,
+ 0x012, 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x066,
+ 0x004, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x3b6,
+ 0x003, 0x054, 0x0a0, 0x081, 0x189, 0x020, 0x1a9, 0x000,
+ 0x044, 0x3fd, 0x000, 0x387, 0x3ff, 0x044, 0x02b, 0x001,
+ 0x3a7, 0x022, 0x020, 0x1bd, 0x000, 0x0a0, 0x081, 0x189,
+ 0x020, 0x1a9, 0x000, 0x080, 0x044, 0x02b, 0x001, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x364, 0x024,
+ 0x002, 0x001, 0x006, 0x399, 0x305, 0x000, 0x004, 0x397,
+ 0x200, 0x044, 0x322, 0x001, 0x3a7, 0x022, 0x044, 0x1bd,
+ 0x000, 0x364, 0x232, 0x002, 0x001, 0x005, 0x044, 0x382,
+ 0x001, 0x000, 0x0c9, 0x3a7, 0x061, 0x044, 0x1bd, 0x000,
+ 0x0a5, 0x044, 0x1bd, 0x000, 0x3a7, 0x072, 0x044, 0x1bd,
+ 0x000, 0x3a7, 0x0f1, 0x044, 0x1bd, 0x000, 0x385, 0x004,
+ 0x0b1, 0x020, 0x1bd, 0x000, 0x364, 0x22c, 0x002, 0x009,
+ 0x2ac, 0x364, 0x234, 0x001, 0x001, 0x058, 0x020, 0x232,
+ 0x000, 0x3c4, 0x3fe, 0x027, 0x364, 0x234, 0x002, 0x001,
+ 0x04d, 0x000, 0x2cd, 0x3b7, 0x211, 0x044, 0x199, 0x000,
+ 0x384, 0x00f, 0x140, 0x239, 0x00b, 0x004, 0x385, 0x010,
+ 0x140, 0x237, 0x00b, 0x004, 0x385, 0x020, 0x044, 0x1bd,
+ 0x000, 0x307, 0x239, 0x00b, 0x004, 0x208, 0x101, 0x384,
+ 0x01f, 0x183, 0x327, 0x237, 0x00b, 0x004, 0x2a8, 0x121,
+ 0x3a4, 0x00f, 0x285, 0x184, 0x3c4, 0x01f, 0x238, 0x362,
+ 0x238, 0x017, 0x00b, 0x005, 0x0a1, 0x1ad, 0x285, 0x305,
+ 0x238, 0x32a, 0x238, 0x3a4, 0x008, 0x285, 0x3b7, 0x210,
+ 0x044, 0x1bd, 0x000, 0x347, 0x237, 0x23a, 0x347, 0x238,
+ 0x23c, 0x347, 0x239, 0x23b, 0x045, 0x345, 0x21c, 0x021,
+ 0x045, 0x308, 0x21c, 0x244, 0x021, 0x0c0, 0x240, 0x307,
+ 0x022, 0x384, 0x001, 0x245, 0x240, 0x142, 0x240, 0x364,
+ 0x240, 0x100, 0x009, 0x3eb, 0x0a1, 0x044, 0x0d9, 0x001,
+ 0x307, 0x240, 0x327, 0x022, 0x286, 0x384, 0x001, 0x001,
+ 0x3ee, 0x0a2, 0x307, 0x22b, 0x384, 0x003, 0x001, 0x008,
+ 0x0aa, 0x262, 0x003, 0x001, 0x003, 0x0a5, 0x289, 0x240,
+ 0x240, 0x387, 0x084, 0x020, 0x2f8, 0x001, 0x094, 0x020,
+ 0x2f2, 0x001, 0x090, 0x020, 0x2f2, 0x001, 0x0a0, 0x082,
+ 0x18b, 0x304, 0x020, 0x009, 0x3b9, 0x081, 0x189, 0x020,
+ 0x1b7, 0x000, 0x0a1, 0x1a9, 0x364, 0x234, 0x002, 0x001,
+ 0x018, 0x364, 0x22c, 0x020, 0x001, 0x016, 0x364, 0x22c,
+ 0x010, 0x001, 0x009, 0x364, 0x232, 0x080, 0x001, 0x00c,
+ 0x000, 0x007, 0x364, 0x232, 0x040, 0x001, 0x005, 0x2c5,
+ 0x024, 0x045, 0x309, 0x024, 0x003, 0x3fb, 0x2a8, 0x2c4,
+ 0x024, 0x045, 0x3b7, 0x213, 0x364, 0x234, 0x004, 0x009,
+ 0x017, 0x307, 0x233, 0x182, 0x00a, 0x007, 0x3c5, 0x040,
+ 0x021, 0x000, 0x005, 0x3c4, 0x3bf, 0x021, 0x082, 0x044,
+ 0x1b7, 0x000, 0x088, 0x020, 0x1a9, 0x000, 0x082, 0x044,
+ 0x1a9, 0x000, 0x3c4, 0x3bf, 0x021, 0x088, 0x020, 0x1b7,
+ 0x000, 0x055, 0x364, 0x026, 0x002, 0x001, 0x031, 0x364,
+ 0x024, 0x002, 0x009, 0x007, 0x364, 0x025, 0x004, 0x009,
+ 0x027, 0x309, 0x01c, 0x003, 0x004, 0x00a, 0x021, 0x364,
+ 0x2cf, 0x011, 0x009, 0x01c, 0x364, 0x33b, 0x007, 0x001,
+ 0x007, 0x364, 0x33b, 0x002, 0x001, 0x012, 0x3c5, 0x010,
+ 0x027, 0x364, 0x234, 0x040, 0x001, 0x005, 0x3c4, 0x37f,
+ 0x024, 0x3c5, 0x008, 0x025, 0x000, 0x010, 0x364, 0x234,
+ 0x040, 0x001, 0x005, 0x3c5, 0x080, 0x024, 0x3c4, 0x3ef,
+ 0x027, 0x3c5, 0x010, 0x026, 0x054, 0x045, 0x364, 0x026,
+ 0x001, 0x001, 0x008, 0x3c5, 0x004, 0x232, 0x044, 0x20d,
+ 0x000, 0x09f, 0x18a, 0x208, 0x244, 0x243, 0x244, 0x250,
+ 0x327, 0x243, 0x326, 0x250, 0x001, 0x308, 0x347, 0x243,
+ 0x250, 0x087, 0x187, 0x208, 0x244, 0x020, 0x364, 0x026,
+ 0x002, 0x001, 0x2fb, 0x3c5, 0x008, 0x232, 0x020, 0x3b5,
+ 0x000, 0x364, 0x233, 0x008, 0x001, 0x004, 0x307, 0x220,
+ 0x3c5, 0x001, 0x00c, 0x247, 0x01b, 0x083, 0x18d, 0x244,
+ 0x01c, 0x044, 0x1c9, 0x001, 0x0a3, 0x1a8, 0x2c5, 0x01f,
+ 0x045, 0x3c4, 0x0ff, 0x01f, 0x085, 0x044, 0x189, 0x000,
+ 0x3d5, 0x100, 0x01f, 0x3b7, 0x300, 0x397, 0x200, 0x044,
+ 0x1a9, 0x000, 0x3c7, 0x07f, 0x252, 0x055, 0x151, 0x252,
+ 0x001, 0x006, 0x140, 0x01e, 0x00b, 0x3fa, 0x397, 0x200,
+ 0x044, 0x1b7, 0x000, 0x3c4, 0x2ff, 0x01f, 0x045, 0x327,
+ 0x22c, 0x307, 0x22d, 0x2e4, 0x004, 0x009, 0x00d, 0x286,
+ 0x384, 0x007, 0x001, 0x00f, 0x2e4, 0x007, 0x001, 0x032,
+ 0x000, 0x039, 0x384, 0x007, 0x3c4, 0x3f8, 0x22c, 0x245,
+ 0x22c, 0x327, 0x22c, 0x307, 0x22d, 0x2e4, 0x040, 0x009,
+ 0x016, 0x286, 0x264, 0x070, 0x001, 0x018, 0x3a4, 0x070,
+ 0x001, 0x029, 0x1aa, 0x002, 0x02e, 0x044, 0x3b5, 0x000,
+ 0x044, 0x3fd, 0x000, 0x000, 0x009, 0x384, 0x070, 0x3c4,
+ 0x38f, 0x22c, 0x245, 0x22c, 0x347, 0x22c, 0x22d, 0x045,
+ 0x227, 0x044, 0x301, 0x000, 0x044, 0x232, 0x000, 0x000,
+ 0x3d2, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x000,
+ 0x3ca, 0x044, 0x003, 0x001, 0x044, 0x326, 0x000, 0x000,
+ 0x3e5, 0x044, 0x0f2, 0x001, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x000, 0x3da, 0x3a7, 0x080, 0x384, 0x001,
+ 0x001, 0x011, 0x082, 0x044, 0x1a9, 0x000, 0x081, 0x044,
+ 0x1b7, 0x000, 0x3c5, 0x004, 0x024, 0x3c5, 0x008, 0x025,
+ 0x045, 0x366, 0x22e, 0x020, 0x001, 0x010, 0x082, 0x044,
+ 0x1b7, 0x000, 0x081, 0x044, 0x1a9, 0x000, 0x3c4, 0x3fb,
+ 0x024, 0x3c4, 0x3f7, 0x025, 0x045, 0x308, 0x248, 0x19e,
+ 0x020, 0x254, 0x001, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x020, 0x1b6, 0x001,
+ 0x3c5, 0x001, 0x00c, 0x0c0, 0x01b, 0x044, 0x1c9, 0x001,
+ 0x3c4, 0x3fe, 0x00c, 0x045, 0x0a1, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x247, 0x24f, 0x397, 0x3e0, 0x044, 0x1a9,
+ 0x000, 0x0a0, 0x325, 0x27b, 0x387, 0x140, 0x044, 0x1a9,
+ 0x000, 0x044, 0x187, 0x000, 0x084, 0x044, 0x1a9, 0x000,
+ 0x044, 0x187, 0x000, 0x387, 0x040, 0x044, 0x1b7, 0x000,
+ 0x387, 0x106, 0x044, 0x189, 0x000, 0x387, 0x100, 0x044,
+ 0x1b7, 0x000, 0x387, 0x099, 0x044, 0x189, 0x000, 0x0a1,
+ 0x325, 0x27b, 0x307, 0x24f, 0x044, 0x1bd, 0x000, 0x0a0,
+ 0x325, 0x27b, 0x084, 0x044, 0x1b7, 0x000, 0x387, 0x099,
+ 0x044, 0x2f6, 0x002, 0x009, 0x004, 0x399, 0x3ff, 0x020,
+ 0x189, 0x000, 0x041, 0x327, 0x236, 0x3a4, 0x00c, 0x134,
+ 0x043, 0x045, 0x055, 0x3d7, 0x071, 0x030, 0x000, 0x006,
+ 0x055, 0x3d7, 0x221, 0x030, 0x347, 0x034, 0x258, 0x0c0,
+ 0x034, 0x140, 0x035, 0x009, 0x3fe, 0x347, 0x032, 0x031,
+ 0x345, 0x21d, 0x030, 0x246, 0x031, 0x374, 0x030, 0x200,
+ 0x001, 0x007, 0x246, 0x031, 0x044, 0x18c, 0x000, 0x131,
+ 0x009, 0x3f3, 0x346, 0x21d, 0x030, 0x347, 0x258, 0x034,
+ 0x054, 0x045, 0x364, 0x233, 0x040, 0x001, 0x006, 0x3b7,
+ 0x200, 0x2a8, 0x284, 0x045, 0x227, 0x055, 0x040, 0x397,
+ 0x203, 0x300, 0x202, 0x04d, 0x141, 0x202, 0x362, 0x202,
+ 0x018, 0x009, 0x004, 0x0c0, 0x202, 0x042, 0x054, 0x045,
+ 0x055, 0x3c7, 0x0f2, 0x030, 0x364, 0x035, 0x001, 0x009,
+ 0x3fd, 0x307, 0x032, 0x045, 0x0a1, 0x1aa, 0x324, 0x233,
+ 0x045, 0x327, 0x222, 0x3a4, 0x07f, 0x001, 0x02c, 0x055,
+ 0x307, 0x034, 0x0c0, 0x034, 0x040, 0x140, 0x035, 0x009,
+ 0x3fe, 0x3c7, 0x071, 0x030, 0x387, 0x0d7, 0x347, 0x032,
+ 0x031, 0x345, 0x21d, 0x030, 0x141, 0x23d, 0x009, 0x007,
+ 0x153, 0x23d, 0x346, 0x21c, 0x031, 0x246, 0x031, 0x131,
+ 0x009, 0x3f4, 0x346, 0x21d, 0x030, 0x042, 0x247, 0x034,
+ 0x054, 0x045, 0x083, 0x18d, 0x345, 0x21d, 0x233, 0x364,
+ 0x2cf, 0x004, 0x001, 0x005, 0x245, 0x305, 0x045, 0x3a7,
+ 0x071, 0x020, 0x1a9, 0x000, 0x083, 0x18d, 0x364, 0x2cf,
+ 0x004, 0x001, 0x006, 0x208, 0x244, 0x305, 0x045, 0x328,
+ 0x21d, 0x2c4, 0x233, 0x3a7, 0x071, 0x020, 0x1b7, 0x000,
+ 0x120, 0x001, 0x027, 0x041, 0x2f4, 0x200, 0x001, 0x00a,
+ 0x3a7, 0x073, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x009,
+ 0x3a7, 0x073, 0x387, 0x3fd, 0x044, 0x1b8, 0x000, 0x043,
+ 0x2e4, 0x180, 0x001, 0x016, 0x2e4, 0x080, 0x009, 0x017,
+ 0x3c5, 0x002, 0x232, 0x044, 0x382, 0x001, 0x000, 0x012,
+ 0x3c4, 0x3fd, 0x232, 0x044, 0x394, 0x001, 0x000, 0x00a,
+ 0x044, 0x353, 0x001, 0x000, 0x005, 0x081, 0x043, 0x040,
+ 0x020, 0x181, 0x006, 0x3c4, 0x3fe, 0x03a, 0x247, 0x25e,
+ 0x0c5, 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263,
+ 0x3c4, 0x3bf, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03a,
+ 0x3c5, 0x040, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042,
+ 0x100, 0x00b, 0x008, 0x247, 0x03c, 0x042, 0x247, 0x03b,
+ 0x042, 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03a,
+ 0x041, 0x307, 0x25d, 0x040, 0x152, 0x03a, 0x345, 0x263,
+ 0x03a, 0x045, 0x3c4, 0x3fe, 0x03d, 0x247, 0x25f, 0x0c5,
+ 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263, 0x3c4,
+ 0x37f, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03d, 0x3c5,
+ 0x080, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042, 0x100,
+ 0x00b, 0x008, 0x247, 0x03f, 0x042, 0x247, 0x03e, 0x042,
+ 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03d, 0x041,
+ 0x307, 0x25d, 0x040, 0x152, 0x03d, 0x345, 0x263, 0x03d,
+ 0x045, 0x3c4, 0x3fe, 0x040, 0x0c1, 0x041, 0x3c4, 0x3cf,
+ 0x00c, 0x151, 0x041, 0x0c0, 0x263, 0x247, 0x262, 0x0c0,
+ 0x040, 0x384, 0x003, 0x227, 0x1a0, 0x285, 0x384, 0x3fd,
+ 0x187, 0x245, 0x040, 0x245, 0x041, 0x245, 0x263, 0x3c5,
+ 0x030, 0x00c, 0x3c7, 0x3ff, 0x039, 0x142, 0x040, 0x152,
+ 0x040, 0x144, 0x040, 0x364, 0x262, 0x100, 0x001, 0x004,
+ 0x148, 0x040, 0x042, 0x043, 0x247, 0x261, 0x2c7, 0x260,
+ 0x044, 0x0de, 0x002, 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x04a,
+ 0x307, 0x262, 0x384, 0x030, 0x187, 0x245, 0x04a, 0x0a1,
+ 0x307, 0x262, 0x384, 0x030, 0x193, 0x262, 0x000, 0x001,
+ 0x008, 0x262, 0x001, 0x001, 0x007, 0x000, 0x006, 0x0a0,
+ 0x000, 0x003, 0x0a2, 0x1ad, 0x2c5, 0x263, 0x0a3, 0x307,
+ 0x262, 0x384, 0x00c, 0x191, 0x262, 0x001, 0x001, 0x008,
+ 0x262, 0x002, 0x001, 0x007, 0x000, 0x006, 0x0a1, 0x000,
+ 0x003, 0x0a2, 0x1a5, 0x2c5, 0x263, 0x374, 0x262, 0x200,
+ 0x001, 0x00e, 0x307, 0x261, 0x327, 0x260, 0x041, 0x040,
+ 0x347, 0x263, 0x041, 0x141, 0x040, 0x045, 0x347, 0x263,
+ 0x041, 0x141, 0x040, 0x020, 0x089, 0x000, 0x042, 0x043,
+ 0x247, 0x25c, 0x042, 0x264, 0x3ff, 0x00b, 0x008, 0x247,
+ 0x043, 0x042, 0x247, 0x042, 0x042, 0x247, 0x039, 0x042,
+ 0x247, 0x038, 0x142, 0x040, 0x152, 0x040, 0x307, 0x25c,
+ 0x041, 0x040, 0x045, 0x055, 0x140, 0x259, 0x001, 0x009,
+ 0x322, 0x259, 0x001, 0x007, 0x0a1, 0x054, 0x045, 0x2c7,
+ 0x259, 0x264, 0x008, 0x009, 0x06e, 0x0c0, 0x012, 0x043,
+ 0x2c7, 0x25a, 0x043, 0x2c7, 0x25b, 0x227, 0x274, 0x200,
+ 0x009, 0x022, 0x3a4, 0x006, 0x2c7, 0x00d, 0x227, 0x3c4,
+ 0x3fd, 0x00c, 0x3c4, 0x3fc, 0x018, 0x384, 0x001, 0x245,
+ 0x018, 0x142, 0x00c, 0x042, 0x247, 0x014, 0x042, 0x247,
+ 0x013, 0x120, 0x00b, 0x008, 0x042, 0x247, 0x011, 0x042,
+ 0x247, 0x010, 0x3c7, 0x050, 0x26e, 0x142, 0x012, 0x141,
+ 0x00d, 0x080, 0x2e4, 0x100, 0x001, 0x003, 0x108, 0x151,
+ 0x26e, 0x001, 0x007, 0x364, 0x012, 0x040, 0x009, 0x3f9,
+ 0x152, 0x012, 0x151, 0x00d, 0x2e4, 0x080, 0x001, 0x003,
+ 0x104, 0x245, 0x012, 0x307, 0x25b, 0x040, 0x307, 0x25a,
+ 0x040, 0x2a9, 0x003, 0x017, 0x151, 0x26e, 0x003, 0x007,
+ 0x364, 0x012, 0x080, 0x009, 0x3f9, 0x364, 0x026, 0x002,
+ 0x001, 0x007, 0x364, 0x025, 0x004, 0x009, 0x007, 0x141,
+ 0x012, 0x0a0, 0x054, 0x045, 0x364, 0x2cf, 0x007, 0x001,
+ 0x3f8, 0x3c4, 0x3fc, 0x2cf, 0x3d5, 0x200, 0x2cf, 0x020,
+ 0x33a, 0x002, 0x247, 0x017, 0x287, 0x00b, 0x006, 0x3a7,
+ 0x3ff, 0x000, 0x003, 0x1bb, 0x384, 0x0ff, 0x264, 0x080,
+ 0x001, 0x004, 0x385, 0x300, 0x380, 0x011, 0x00b, 0x018,
+ 0x041, 0x0a0, 0x131, 0x380, 0x014, 0x003, 0x3fd, 0x041,
+ 0x0bf, 0x1a6, 0x2a8, 0x2c4, 0x018, 0x186, 0x245, 0x018,
+ 0x083, 0x18d, 0x245, 0x017, 0x042, 0x043, 0x103, 0x121,
+ 0x18a, 0x1a7, 0x285, 0x247, 0x01a, 0x387, 0x3ff, 0x247,
+ 0x019, 0x18b, 0x245, 0x018, 0x307, 0x017, 0x19d, 0x262,
+ 0x003, 0x001, 0x00e, 0x262, 0x001, 0x001, 0x013, 0x262,
+ 0x002, 0x009, 0x037, 0x3a7, 0x0aa, 0x000, 0x00d, 0x0c1,
+ 0x019, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x000, 0x039,
+ 0x3a7, 0x055, 0x041, 0x3a7, 0x071, 0x044, 0x192, 0x000,
+ 0x00b, 0x010, 0x040, 0x3a7, 0x0f2, 0x044, 0x192, 0x000,
+ 0x198, 0x043, 0x3a4, 0x001, 0x009, 0x004, 0x386, 0x001,
+ 0x043, 0x264, 0x001, 0x001, 0x004, 0x3a6, 0x0ff, 0x287,
+ 0x187, 0x285, 0x247, 0x019, 0x18b, 0x103, 0x244, 0x018,
+ 0x364, 0x24d, 0x030, 0x009, 0x00c, 0x08f, 0x18b, 0x208,
+ 0x244, 0x018, 0x397, 0x1fe, 0x182, 0x244, 0x019, 0x327,
+ 0x017, 0x1ba, 0x3a4, 0x007, 0x001, 0x019, 0x2e2, 0x003,
+ 0x001, 0x015, 0x041, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x043, 0x384, 0x008, 0x001, 0x00a, 0x2e2, 0x003, 0x003,
+ 0x004, 0x3a6, 0x002, 0x3a6, 0x001, 0x387, 0x002, 0x189,
+ 0x395, 0x23f, 0x280, 0x050, 0x245, 0x01a, 0x045, 0x0ff,
+ 0x00c, 0x030, 0x03c, 0x00f, 0x0f0, 0x004, 0x020, 0x307,
+ 0x26f, 0x384, 0x003, 0x229, 0x220, 0x3b0, 0x04b, 0x287,
+ 0x199, 0x390, 0x008, 0x041, 0x040, 0x045, 0x055, 0x329,
+ 0x01f, 0x00b, 0x08b, 0x364, 0x234, 0x080, 0x001, 0x00e,
+ 0x364, 0x024, 0x002, 0x009, 0x009, 0x364, 0x025, 0x004,
+ 0x001, 0x004, 0x000, 0x012, 0x327, 0x01d, 0x009, 0x00a,
+ 0x081, 0x18d, 0x245, 0x020, 0x327, 0x01d, 0x246, 0x020,
+ 0x3a2, 0x020, 0x003, 0x02b, 0x309, 0x01c, 0x003, 0x022,
+ 0x3c4, 0x3ef, 0x027, 0x3c5, 0x010, 0x026, 0x0a0, 0x387,
+ 0x3df, 0x044, 0x1b8, 0x000, 0x081, 0x189, 0x044, 0x1a9,
+ 0x000, 0x140, 0x01c, 0x00b, 0x00d, 0x0a1, 0x044, 0x192,
+ 0x000, 0x247, 0x27f, 0x395, 0x3ff, 0x044, 0x1bd, 0x000,
+ 0x344, 0x21f, 0x01c, 0x000, 0x02a, 0x307, 0x01c, 0x003,
+ 0x026, 0x345, 0x21d, 0x01c, 0x209, 0x003, 0x020, 0x307,
+ 0x020, 0x181, 0x003, 0x01b, 0x364, 0x025, 0x004, 0x009,
+ 0x005, 0x3c5, 0x008, 0x025, 0x0a0, 0x081, 0x189, 0x044,
+ 0x1b7, 0x000, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1,
+ 0x307, 0x27f, 0x044, 0x1bd, 0x000, 0x055, 0x364, 0x234,
+ 0x002, 0x001, 0x00f, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x044, 0x1b6, 0x001,
+ 0x054, 0x020, 0x0a3, 0x006, 0x346, 0x21d, 0x232, 0x02b,
+ 0x181, 0x006, 0x364, 0x233, 0x001, 0x021, 0x08d, 0x006,
+ 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4, 0x000, 0x327, 0x21d,
+ 0x324, 0x020, 0x045, 0x307, 0x28d, 0x384, 0x07f, 0x22a,
+ 0x286, 0x040, 0x0b0, 0x044, 0x192, 0x000, 0x384, 0x380,
+ 0x043, 0x285, 0x0b0, 0x020, 0x1bd, 0x000, 0x0a2, 0x141,
+ 0x041, 0x141, 0x2f1, 0x397, 0x200, 0x151, 0x041, 0x044,
+ 0x0fb, 0x002, 0x055, 0x364, 0x200, 0x080, 0x009, 0x00e,
+ 0x364, 0x234, 0x002, 0x001, 0x009, 0x054, 0x364, 0x012,
+ 0x040, 0x001, 0x3f1, 0x045, 0x364, 0x2cf, 0x007, 0x009,
+ 0x3fc, 0x044, 0x33f, 0x002, 0x3c4, 0x080, 0x2cf, 0x3c4,
+ 0x2bf, 0x200, 0x080, 0x046, 0x020, 0x0d3, 0x000, 0x0c0,
+ 0x013, 0x0c0, 0x014, 0x0c0, 0x259, 0x045, 0x088, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x364, 0x259, 0x002, 0x009, 0x007,
+ 0x042, 0x042, 0x020, 0x0d3, 0x000, 0x387, 0x085, 0x187,
+ 0x385, 0x080, 0x247, 0x041, 0x3c5, 0x020, 0x00c, 0x045,
+ 0x387, 0x07f, 0x247, 0x2a1, 0x247, 0x2a2, 0x247, 0x2a3,
+ 0x247, 0x2a4, 0x247, 0x2a7, 0x247, 0x2a8, 0x247, 0x2a5,
+ 0x247, 0x2a6, 0x044, 0x150, 0x003, 0x0c3, 0x2e0, 0x364,
+ 0x02b, 0x010, 0x009, 0x011, 0x044, 0x38c, 0x002, 0x044,
+ 0x39f, 0x002, 0x347, 0x290, 0x2c6, 0x347, 0x28f, 0x2c5,
+ 0x347, 0x28e, 0x2c4, 0x045, 0x0c0, 0x2ce, 0x0c8, 0x2d5,
+ 0x0cf, 0x2ca, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x0c5, 0x2cc,
+ 0x0cf, 0x2d1, 0x0c0, 0x2d2, 0x0dd, 0x2cd, 0x045, 0x0c0,
+ 0x2d8, 0x044, 0x3b3, 0x002, 0x3c7, 0x038, 0x290, 0x0cc,
+ 0x28f, 0x0c0, 0x28e, 0x0cd, 0x291, 0x044, 0x034, 0x003,
+ 0x020, 0x06e, 0x003, 0x364, 0x02b, 0x010, 0x009, 0x013,
+ 0x0ca, 0x293, 0x0a0, 0x397, 0x294, 0x04d, 0x101, 0x272,
+ 0x2a0, 0x009, 0x3fc, 0x044, 0x06e, 0x003, 0x044, 0x096,
+ 0x003, 0x090, 0x3a7, 0x022, 0x044, 0x1a9, 0x000, 0x347,
+ 0x2aa, 0x2a1, 0x347, 0x2ab, 0x2a2, 0x347, 0x2ac, 0x2a3,
+ 0x347, 0x2ad, 0x2a4, 0x020, 0x3f0, 0x002, 0x044, 0x034,
+ 0x003, 0x020, 0x096, 0x003, 0x3a7, 0x020, 0x084, 0x020,
+ 0x1b7, 0x000, 0x3a7, 0x020, 0x084, 0x020, 0x1a9, 0x000,
+ 0x307, 0x2a1, 0x187, 0x327, 0x2a3, 0x00b, 0x004, 0x141,
+ 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x024, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a2, 0x187, 0x327, 0x2a4, 0x00b,
+ 0x004, 0x141, 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7,
+ 0x026, 0x044, 0x1bd, 0x000, 0x307, 0x2a7, 0x187, 0x327,
+ 0x2a5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x025, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a8, 0x187, 0x327, 0x2a6, 0x3a4,
+ 0x0ff, 0x285, 0x208, 0x3a7, 0x027, 0x020, 0x1bd, 0x000,
+ 0x0c0, 0x28f, 0x000, 0x00d, 0x140, 0x28f, 0x003, 0x3fa,
+ 0x362, 0x28f, 0x00f, 0x00b, 0x004, 0x0cf, 0x28f, 0x140,
+ 0x28e, 0x003, 0x00b, 0x362, 0x28e, 0x00f, 0x00b, 0x008,
+ 0x0cf, 0x28e, 0x000, 0x004, 0x0c0, 0x28e, 0x307, 0x28f,
+ 0x183, 0x305, 0x28e, 0x22a, 0x3a4, 0x3f7, 0x286, 0x327,
+ 0x290, 0x1a7, 0x285, 0x3a7, 0x02b, 0x044, 0x1bd, 0x000,
+ 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x0c2, 0x2fa, 0x045,
+ 0x045, 0x347, 0x2ca, 0x293, 0x000, 0x011, 0x307, 0x2ca,
+ 0x302, 0x293, 0x003, 0x3f7, 0x327, 0x2c9, 0x322, 0x293,
+ 0x003, 0x005, 0x347, 0x2c9, 0x293, 0x307, 0x291, 0x044,
+ 0x106, 0x003, 0x0a1, 0x364, 0x236, 0x100, 0x009, 0x003,
+ 0x0a5, 0x1a3, 0x285, 0x183, 0x305, 0x292, 0x183, 0x305,
+ 0x293, 0x3a7, 0x02c, 0x020, 0x1bd, 0x000, 0x307, 0x296,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x295, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x294, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x028, 0x044,
+ 0x1bd, 0x000, 0x307, 0x299, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x298, 0x044, 0x106, 0x003, 0x040, 0x307, 0x297,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x029, 0x044, 0x1bd, 0x000, 0x307, 0x29f,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x29e, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x29d, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x031, 0x044,
+ 0x1bd, 0x000, 0x307, 0x29c, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x29b, 0x044, 0x106, 0x003, 0x040, 0x307, 0x29a,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x02a, 0x020, 0x1bd, 0x000, 0x100, 0x003,
+ 0x005, 0x22a, 0x286, 0x045, 0x228, 0x121, 0x28a, 0x286,
+ 0x385, 0x010, 0x045, 0x083, 0x182, 0x0a1, 0x182, 0x385,
+ 0x001, 0x181, 0x285, 0x181, 0x385, 0x002, 0x181, 0x285,
+ 0x3a7, 0x030, 0x020, 0x1bd, 0x000, 0x044, 0x15b, 0x003,
+ 0x382, 0x081, 0x240, 0x2a5, 0x240, 0x2a6, 0x044, 0x134,
+ 0x003, 0x020, 0x3f0, 0x002, 0x364, 0x2a5, 0x300, 0x001,
+ 0x007, 0x003, 0x010, 0x3c7, 0x0ff, 0x2a5, 0x364, 0x2a6,
+ 0x300, 0x001, 0x007, 0x003, 0x00a, 0x3c7, 0x0ff, 0x2a6,
+ 0x045, 0x0c0, 0x2a5, 0x000, 0x3f3, 0x0c0, 0x2a6, 0x045,
+ 0x397, 0x2a1, 0x04c, 0x109, 0x04d, 0x118, 0x272, 0x2a9,
+ 0x009, 0x3fa, 0x045, 0x347, 0x2ae, 0x2a5, 0x347, 0x2af,
+ 0x2a6, 0x045, 0x342, 0x2fe, 0x00f, 0x003, 0x03f, 0x045,
+ 0x302, 0x2fe, 0x003, 0x019, 0x042, 0x042, 0x045, 0x327,
+ 0x00e, 0x309, 0x00f, 0x002, 0x005, 0x2a8, 0x208, 0x121,
+ 0x20e, 0x040, 0x041, 0x322, 0x2fd, 0x001, 0x3eb, 0x303,
+ 0x2fe, 0x00b, 0x3eb, 0x042, 0x247, 0x2fd, 0x042, 0x247,
+ 0x2fe, 0x000, 0x02e, 0x0c0, 0x2ff, 0x044, 0x30e, 0x002,
+ 0x364, 0x017, 0x004, 0x009, 0x3dc, 0x340, 0x046, 0x047,
+ 0x001, 0x021, 0x342, 0x2fd, 0x00e, 0x001, 0x3c5, 0x343,
+ 0x2fe, 0x00f, 0x00b, 0x017, 0x347, 0x00e, 0x2fd, 0x347,
+ 0x00f, 0x2fe, 0x009, 0x00d, 0x140, 0x2fd, 0x003, 0x009,
+ 0x342, 0x300, 0x2fd, 0x00b, 0x004, 0x0c0, 0x2fd, 0x0c1,
+ 0x2ff, 0x045, 0x30e, 0x21d, 0x102, 0x000, 0x004, 0x30e,
+ 0x21d, 0x327, 0x2c7, 0x041, 0x327, 0x2c8, 0x041, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x045, 0x366, 0x200, 0x080, 0x009,
+ 0x019, 0x364, 0x026, 0x002, 0x001, 0x014, 0x054, 0x140,
+ 0x2fa, 0x00b, 0x3f3, 0x3c7, 0x3ff, 0x2fa, 0x360, 0x2fa,
+ 0x00a, 0x003, 0x007, 0x054, 0x140, 0x01c, 0x00b, 0x3f8,
+ 0x045, 0x0c0, 0x301, 0x3ce, 0x3ff, 0x2fe, 0x040, 0x044,
+ 0x3f0, 0x002, 0x044, 0x18b, 0x003, 0x042, 0x04c, 0x140,
+ 0x301, 0x001, 0x010, 0x346, 0x301, 0x00f, 0x003, 0x029,
+ 0x320, 0x303, 0x2e4, 0x100, 0x009, 0x02a, 0x04d, 0x000,
+ 0x3e7, 0x0c1, 0x303, 0x347, 0x00f, 0x301, 0x00b, 0x004,
+ 0x152, 0x303, 0x009, 0x004, 0x0c1, 0x301, 0x040, 0x3a7,
+ 0x060, 0x044, 0x192, 0x000, 0x264, 0x008, 0x001, 0x007,
+ 0x348, 0x303, 0x303, 0x141, 0x303, 0x000, 0x3d0, 0x140,
+ 0x2ff, 0x009, 0x005, 0x322, 0x303, 0x04d, 0x045, 0x364,
+ 0x2cf, 0x080, 0x009, 0x019, 0x044, 0x346, 0x002, 0x0a0,
+ 0x397, 0x294, 0x04d, 0x101, 0x272, 0x2a0, 0x009, 0x3fc,
+ 0x044, 0x096, 0x003, 0x0c0, 0x293, 0x044, 0x06e, 0x003,
+ 0x044, 0x2b1, 0x003, 0x140, 0x2d3, 0x009, 0x002, 0x3c4,
+ 0x2fd, 0x00d, 0x044, 0x33f, 0x002, 0x3c4, 0x2ff, 0x200,
+ 0x020, 0x089, 0x000, 0x0cd, 0x2ce, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x307, 0x2e3, 0x140, 0x304, 0x009,
+ 0x004, 0x208, 0x101, 0x240, 0x2a6, 0x240, 0x2a5, 0x044,
+ 0x134, 0x003, 0x044, 0x1ba, 0x003, 0x0c0, 0x309, 0x387,
+ 0x071, 0x140, 0x304, 0x001, 0x003, 0x108, 0x364, 0x309,
+ 0x001, 0x001, 0x004, 0x380, 0x040, 0x187, 0x106, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x397, 0x2a5, 0x300, 0x309, 0x044,
+ 0x1e9, 0x003, 0x141, 0x309, 0x362, 0x309, 0x002, 0x009,
+ 0x3e0, 0x044, 0x3f0, 0x002, 0x347, 0x2a5, 0x2e3, 0x342,
+ 0x2ae, 0x2e3, 0x00b, 0x007, 0x348, 0x2e3, 0x2e3, 0x141,
+ 0x2e3, 0x140, 0x307, 0x009, 0x005, 0x347, 0x2e3, 0x307,
+ 0x045, 0x364, 0x2cf, 0x080, 0x009, 0x0ac, 0x0c1, 0x2ce,
+ 0x055, 0x3c5, 0x004, 0x2cf, 0x3a7, 0x071, 0x044, 0x192,
+ 0x000, 0x247, 0x305, 0x054, 0x080, 0x044, 0x1bd, 0x000,
+ 0x3a7, 0x02c, 0x387, 0x30f, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x022, 0x08a, 0x044, 0x1bd, 0x000, 0x081, 0x189, 0x0a0,
+ 0x044, 0x1a9, 0x000, 0x30e, 0x21d, 0x102, 0x0b0, 0x041,
+ 0x0a0, 0x041, 0x0a2, 0x044, 0x0fb, 0x002, 0x080, 0x0a0,
+ 0x262, 0x001, 0x00b, 0x004, 0x3a7, 0x0ff, 0x040, 0x397,
+ 0x2a1, 0x04d, 0x101, 0x04d, 0x101, 0x04d, 0x101, 0x04d,
+ 0x113, 0x043, 0x280, 0x3a7, 0x080, 0x04d, 0x040, 0x392,
+ 0x2a1, 0x262, 0x005, 0x00b, 0x007, 0x040, 0x044, 0x3e4,
+ 0x002, 0x042, 0x227, 0x3a4, 0x001, 0x121, 0x1a5, 0x191,
+ 0x285, 0x187, 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x042,
+ 0x044, 0x1e9, 0x003, 0x041, 0x101, 0x392, 0x2a1, 0x264,
+ 0x008, 0x001, 0x3c6, 0x390, 0x2a1, 0x111, 0x043, 0x04d,
+ 0x272, 0x2a1, 0x003, 0x3fb, 0x044, 0x150, 0x003, 0x044,
+ 0x3ea, 0x002, 0x044, 0x06e, 0x003, 0x364, 0x292, 0x008,
+ 0x009, 0x008, 0x080, 0x3a7, 0x022, 0x044, 0x1bd, 0x000,
+ 0x307, 0x020, 0x192, 0x002, 0x013, 0x19a, 0x002, 0x010,
+ 0x081, 0x189, 0x0a0, 0x044, 0x1b7, 0x000, 0x055, 0x3a7,
+ 0x071, 0x307, 0x305, 0x044, 0x1bd, 0x000, 0x044, 0x3f0,
+ 0x002, 0x3c4, 0x3fb, 0x2cf, 0x3c5, 0x080, 0x2cf, 0x054,
+ 0x045, 0x397, 0x36a, 0x247, 0x369, 0x0a0, 0x101, 0x04d,
+ 0x272, 0x38e, 0x009, 0x3fc, 0x045, 0x0c3, 0x34e, 0x1cb,
+ 0x34e, 0x3c5, 0x03f, 0x34e, 0x0c3, 0x362, 0x0c4, 0x363,
+ 0x0c0, 0x33c, 0x0c0, 0x356, 0x0c0, 0x357, 0x3d7, 0x2f9,
+ 0x352, 0x1c2, 0x352, 0x3d7, 0x254, 0x353, 0x1c0, 0x353,
+ 0x08d, 0x18b, 0x385, 0x05e, 0x247, 0x354, 0x3c7, 0x0b2,
+ 0x355, 0x345, 0x21d, 0x355, 0x0c4, 0x341, 0x0c2, 0x33f,
+ 0x0d2, 0x340, 0x0d3, 0x342, 0x0ce, 0x344, 0x0de, 0x343,
+ 0x0c0, 0x348, 0x045, 0x055, 0x3c4, 0x008, 0x33b, 0x144,
+ 0x33b, 0x0c6, 0x33d, 0x3c4, 0x3fb, 0x200, 0x307, 0x027,
+ 0x384, 0x324, 0x101, 0x247, 0x027, 0x045, 0x055, 0x3c4,
+ 0x3ec, 0x052, 0x044, 0x05b, 0x001, 0x3c4, 0x3fb, 0x00c,
+ 0x0c0, 0x359, 0x3c4, 0x1ff, 0x35b, 0x045, 0x055, 0x0c0,
+ 0x33d, 0x3c4, 0x3fe, 0x03d, 0x0df, 0x053, 0x3c4, 0x373,
+ 0x00c, 0x0c0, 0x349, 0x0c0, 0x35a, 0x3c4, 0x330, 0x027,
+ 0x0c0, 0x33b, 0x344, 0x21f, 0x33c, 0x045, 0x044, 0x050,
+ 0x005, 0x307, 0x33c, 0x00b, 0x018, 0x362, 0x33d, 0x003,
+ 0x007, 0x013, 0x009, 0x005, 0x044, 0x073, 0x004, 0x055,
+ 0x364, 0x33c, 0x040, 0x001, 0x008, 0x3c4, 0x3fb, 0x200,
+ 0x044, 0x103, 0x004, 0x044, 0x1bb, 0x005, 0x020, 0x0ad,
+ 0x000, 0x364, 0x33c, 0x002, 0x001, 0x055, 0x3c5, 0x008,
+ 0x00c, 0x364, 0x33c, 0x080, 0x009, 0x023, 0x0c0, 0x392,
+ 0x347, 0x354, 0x03e, 0x347, 0x355, 0x03f, 0x347, 0x352,
+ 0x038, 0x347, 0x353, 0x039, 0x307, 0x352, 0x040, 0x307,
+ 0x353, 0x040, 0x307, 0x354, 0x040, 0x307, 0x355, 0x305,
+ 0x21d, 0x040, 0x397, 0x2a0, 0x044, 0x01a, 0x002, 0x0c0,
+ 0x290, 0x044, 0x034, 0x003, 0x044, 0x361, 0x003, 0x0c7,
+ 0x2d1, 0x0c3, 0x2cc, 0x0c0, 0x2c8, 0x0c1, 0x2c7, 0x1cd,
+ 0x2c7, 0x0c1, 0x33d, 0x142, 0x027, 0x0c0, 0x33e, 0x0c0,
+ 0x365, 0x020, 0x04c, 0x004, 0x085, 0x364, 0x33b, 0x020,
+ 0x001, 0x003, 0x102, 0x18a, 0x385, 0x11f, 0x247, 0x053,
+ 0x045, 0x0c2, 0x33b, 0x0c5, 0x33d, 0x3c5, 0x010, 0x027,
+ 0x0df, 0x053, 0x045, 0x044, 0x3b6, 0x003, 0x044, 0x3c6,
+ 0x003, 0x044, 0x38c, 0x002, 0x044, 0x39f, 0x002, 0x3c4,
+ 0x3fb, 0x200, 0x045, 0x055, 0x364, 0x33c, 0x020, 0x009,
+ 0x007, 0x0c4, 0x33d, 0x044, 0x103, 0x004, 0x0c5, 0x33d,
+ 0x3c4, 0x3fe, 0x03d, 0x3c5, 0x00a, 0x33b, 0x307, 0x027,
+ 0x384, 0x334, 0x305, 0x392, 0x247, 0x027, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x377, 0x00c, 0x020, 0x38c, 0x002, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x044, 0x006,
+ 0x005, 0x364, 0x33b, 0x010, 0x3c4, 0x3bf, 0x33c, 0x364,
+ 0x33c, 0x020, 0x009, 0x007, 0x364, 0x33c, 0x002, 0x001,
+ 0x015, 0x3c5, 0x004, 0x00c, 0x364, 0x33c, 0x020, 0x001,
+ 0x005, 0x3c5, 0x080, 0x058, 0x0c0, 0x350, 0x044, 0x0c5,
+ 0x004, 0x3c5, 0x001, 0x052, 0x045, 0x0c0, 0x04c, 0x347,
+ 0x357, 0x04d, 0x347, 0x356, 0x04e, 0x347, 0x357, 0x364,
+ 0x347, 0x356, 0x365, 0x387, 0x067, 0x187, 0x385, 0x080,
+ 0x3c4, 0x04c, 0x052, 0x245, 0x052, 0x309, 0x026, 0x003,
+ 0x00c, 0x0c0, 0x029, 0x3c7, 0x022, 0x029, 0x345, 0x21d,
+ 0x029, 0x000, 0x00f, 0x347, 0x362, 0x02c, 0x399, 0x380,
+ 0x208, 0x244, 0x029, 0x307, 0x363, 0x187, 0x245, 0x029,
+ 0x081, 0x18c, 0x245, 0x053, 0x347, 0x021, 0x35c, 0x3c4,
+ 0x3cf, 0x021, 0x045, 0x055, 0x364, 0x052, 0x001, 0x001,
+ 0x02a, 0x142, 0x052, 0x086, 0x187, 0x385, 0x072, 0x364,
+ 0x33c, 0x020, 0x001, 0x008, 0x044, 0x18d, 0x000, 0x055,
+ 0x000, 0x005, 0x044, 0x189, 0x000, 0x347, 0x35c, 0x021,
+ 0x3c7, 0x080, 0x052, 0x3c4, 0x3fb, 0x00c, 0x364, 0x33c,
+ 0x200, 0x009, 0x008, 0x3c5, 0x020, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x045, 0x055, 0x140, 0x33d, 0x001, 0x3fc, 0x055,
+ 0x307, 0x368, 0x304, 0x34e, 0x0c0, 0x351, 0x227, 0x1a2,
+ 0x002, 0x071, 0x00b, 0x08b, 0x044, 0x03b, 0x005, 0x044,
+ 0x006, 0x005, 0x141, 0x374, 0x140, 0x346, 0x001, 0x00b,
+ 0x342, 0x342, 0x346, 0x00b, 0x019, 0x0c4, 0x350, 0x000,
+ 0x019, 0x340, 0x347, 0x345, 0x001, 0x00a, 0x342, 0x347,
+ 0x345, 0x364, 0x33c, 0x004, 0x001, 0x3ec, 0x0cc, 0x350,
+ 0x141, 0x384, 0x000, 0x009, 0x0c8, 0x350, 0x141, 0x383,
+ 0x340, 0x342, 0x346, 0x140, 0x345, 0x001, 0x00c, 0x342,
+ 0x344, 0x345, 0x00b, 0x00e, 0x3c5, 0x001, 0x350, 0x000,
+ 0x00e, 0x3c5, 0x003, 0x350, 0x141, 0x37c, 0x000, 0x00a,
+ 0x3c5, 0x002, 0x350, 0x141, 0x37b, 0x340, 0x344, 0x345,
+ 0x140, 0x347, 0x001, 0x00c, 0x342, 0x343, 0x347, 0x00b,
+ 0x00f, 0x3c5, 0x010, 0x350, 0x000, 0x00f, 0x3c5, 0x030,
+ 0x350, 0x141, 0x38c, 0x020, 0x3fa, 0x004, 0x3c5, 0x020,
+ 0x350, 0x141, 0x38b, 0x340, 0x343, 0x347, 0x020, 0x3fa,
+ 0x004, 0x044, 0x03b, 0x005, 0x141, 0x375, 0x0c0, 0x346,
+ 0x0c0, 0x347, 0x0c0, 0x345, 0x044, 0x01e, 0x005, 0x3c7,
+ 0x03f, 0x350, 0x141, 0x37c, 0x141, 0x384, 0x141, 0x38c,
+ 0x0c0, 0x348, 0x020, 0x3fa, 0x004, 0x205, 0x001, 0x182,
+ 0x044, 0x03b, 0x005, 0x227, 0x3a4, 0x00c, 0x1b1, 0x001,
+ 0x013, 0x364, 0x350, 0x00c, 0x009, 0x05c, 0x364, 0x04d,
+ 0x00c, 0x009, 0x057, 0x132, 0x003, 0x00b, 0x001, 0x027,
+ 0x000, 0x050, 0x3c4, 0x3f3, 0x350, 0x000, 0x04b, 0x140,
+ 0x351, 0x009, 0x004, 0x141, 0x37f, 0x140, 0x346, 0x001,
+ 0x00b, 0x151, 0x348, 0x151, 0x346, 0x3c5, 0x004, 0x350,
+ 0x000, 0x038, 0x3c5, 0x00c, 0x350, 0x140, 0x351, 0x001,
+ 0x031, 0x141, 0x384, 0x000, 0x02d, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x380, 0x342, 0x23e, 0x348, 0x001, 0x016,
+ 0x342, 0x342, 0x346, 0x001, 0x00e, 0x141, 0x348, 0x141,
+ 0x346, 0x340, 0x342, 0x346, 0x3c5, 0x004, 0x350, 0x000,
+ 0x00e, 0x340, 0x342, 0x346, 0x3c5, 0x008, 0x350, 0x140,
+ 0x351, 0x001, 0x004, 0x141, 0x383, 0x340, 0x23e, 0x348,
+ 0x227, 0x3a4, 0x003, 0x001, 0x013, 0x364, 0x350, 0x003,
+ 0x009, 0x071, 0x364, 0x04d, 0x003, 0x009, 0x06c, 0x132,
+ 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065, 0x3c4, 0x3fc,
+ 0x350, 0x000, 0x060, 0x140, 0x351, 0x009, 0x004, 0x141,
+ 0x377, 0x140, 0x345, 0x009, 0x00d, 0x3c5, 0x003, 0x350,
+ 0x140, 0x351, 0x001, 0x04f, 0x141, 0x37c, 0x000, 0x04b,
+ 0x342, 0x342, 0x346, 0x009, 0x007, 0x340, 0x342, 0x346,
+ 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151, 0x348, 0x141,
+ 0x346, 0x152, 0x345, 0x3c5, 0x001, 0x350, 0x000, 0x033,
+ 0x140, 0x351, 0x009, 0x004, 0x141, 0x378, 0x342, 0x23e,
+ 0x348, 0x001, 0x01c, 0x342, 0x344, 0x345, 0x001, 0x014,
+ 0x140, 0x346, 0x001, 0x010, 0x141, 0x348, 0x151, 0x346,
+ 0x142, 0x345, 0x340, 0x344, 0x345, 0x3c5, 0x001, 0x350,
+ 0x000, 0x00e, 0x340, 0x344, 0x345, 0x3c5, 0x002, 0x350,
+ 0x140, 0x351, 0x001, 0x004, 0x141, 0x37b, 0x340, 0x23e,
+ 0x348, 0x227, 0x1b3, 0x3a4, 0x003, 0x001, 0x013, 0x364,
+ 0x350, 0x030, 0x009, 0x071, 0x364, 0x04d, 0x030, 0x009,
+ 0x06c, 0x132, 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065,
+ 0x3c4, 0x3cf, 0x350, 0x000, 0x060, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x387, 0x140, 0x347, 0x009, 0x00d, 0x3c5,
+ 0x030, 0x350, 0x140, 0x351, 0x001, 0x04f, 0x141, 0x38c,
+ 0x000, 0x04b, 0x342, 0x342, 0x346, 0x009, 0x007, 0x340,
+ 0x342, 0x346, 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151,
+ 0x348, 0x141, 0x346, 0x152, 0x347, 0x3c5, 0x010, 0x350,
+ 0x000, 0x033, 0x140, 0x351, 0x009, 0x004, 0x141, 0x388,
+ 0x342, 0x23e, 0x348, 0x001, 0x01c, 0x342, 0x343, 0x347,
+ 0x001, 0x014, 0x140, 0x346, 0x001, 0x010, 0x141, 0x348,
+ 0x151, 0x346, 0x142, 0x347, 0x340, 0x343, 0x347, 0x3c5,
+ 0x010, 0x350, 0x000, 0x00e, 0x340, 0x343, 0x347, 0x3c5,
+ 0x020, 0x350, 0x140, 0x351, 0x001, 0x004, 0x141, 0x38b,
+ 0x340, 0x23e, 0x348, 0x328, 0x04d, 0x324, 0x350, 0x2e4,
+ 0x02a, 0x001, 0x026, 0x140, 0x351, 0x009, 0x022, 0x141,
+ 0x351, 0x2e4, 0x020, 0x001, 0x005, 0x3c4, 0x3cf, 0x350,
+ 0x2e4, 0x008, 0x001, 0x005, 0x3c4, 0x3f3, 0x350, 0x2e4,
+ 0x002, 0x001, 0x28a, 0x3c4, 0x3fc, 0x350, 0x000, 0x285,
+ 0x0c0, 0x350, 0x3c4, 0x3bf, 0x027, 0x000, 0x0a8, 0x328,
+ 0x04d, 0x324, 0x350, 0x287, 0x3a4, 0x003, 0x131, 0x009,
+ 0x029, 0x140, 0x345, 0x001, 0x01c, 0x342, 0x344, 0x345,
+ 0x044, 0x030, 0x005, 0x003, 0x006, 0x140, 0x345, 0x009,
+ 0x00b, 0x364, 0x368, 0x002, 0x001, 0x006, 0x141, 0x350,
+ 0x141, 0x37b, 0x340, 0x344, 0x345, 0x000, 0x00b, 0x364,
+ 0x368, 0x001, 0x001, 0x006, 0x142, 0x350, 0x141, 0x37c,
+ 0x227, 0x3a4, 0x00c, 0x134, 0x009, 0x039, 0x140, 0x346,
+ 0x001, 0x01c, 0x342, 0x342, 0x346, 0x044, 0x030, 0x005,
+ 0x003, 0x006, 0x140, 0x346, 0x009, 0x00b, 0x364, 0x368,
+ 0x008, 0x001, 0x006, 0x144, 0x350, 0x141, 0x383, 0x340,
+ 0x342, 0x346, 0x000, 0x01b, 0x364, 0x368, 0x008, 0x009,
+ 0x3e3, 0x364, 0x368, 0x004, 0x001, 0x011, 0x327, 0x345,
+ 0x320, 0x347, 0x001, 0x007, 0x364, 0x33c, 0x004, 0x001,
+ 0x006, 0x148, 0x350, 0x141, 0x384, 0x227, 0x3a4, 0x030,
+ 0x3a2, 0x010, 0x009, 0x02b, 0x140, 0x347, 0x001, 0x01d,
+ 0x342, 0x343, 0x347, 0x044, 0x030, 0x005, 0x003, 0x006,
+ 0x140, 0x347, 0x009, 0x00c, 0x364, 0x368, 0x020, 0x001,
+ 0x007, 0x3c0, 0x010, 0x350, 0x141, 0x38b, 0x340, 0x343,
+ 0x347, 0x000, 0x00c, 0x364, 0x368, 0x010, 0x001, 0x007,
+ 0x3c0, 0x020, 0x350, 0x141, 0x38c, 0x044, 0x3fa, 0x004,
+ 0x000, 0x026, 0x3c5, 0x040, 0x027, 0x307, 0x04d, 0x384,
+ 0x200, 0x305, 0x350, 0x247, 0x04d, 0x045, 0x08f, 0x3b7,
+ 0x211, 0x044, 0x1b8, 0x000, 0x347, 0x33f, 0x346, 0x347,
+ 0x340, 0x347, 0x347, 0x341, 0x345, 0x347, 0x347, 0x348,
+ 0x340, 0x345, 0x348, 0x340, 0x346, 0x348, 0x307, 0x347,
+ 0x183, 0x305, 0x345, 0x184, 0x305, 0x346, 0x32a, 0x346,
+ 0x3a4, 0x008, 0x285, 0x3b7, 0x210, 0x020, 0x1bd, 0x000,
+ 0x364, 0x33c, 0x008, 0x001, 0x007, 0x327, 0x23e, 0x322,
+ 0x348, 0x131, 0x045, 0x040, 0x227, 0x306, 0x373, 0x001,
+ 0x00f, 0x307, 0x369, 0x101, 0x272, 0x372, 0x009, 0x003,
+ 0x118, 0x04d, 0x247, 0x369, 0x2c7, 0x373, 0x042, 0x045,
+ 0x140, 0x35a, 0x00b, 0x0b0, 0x344, 0x21f, 0x35a, 0x0a3,
+ 0x1ad, 0x2a8, 0x2c4, 0x027, 0x055, 0x044, 0x340, 0x001,
+ 0x054, 0x264, 0x002, 0x001, 0x005, 0x0c0, 0x33d, 0x045,
+ 0x083, 0x18b, 0x304, 0x050, 0x305, 0x221, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3f4, 0x308, 0x221, 0x244, 0x027, 0x387,
+ 0x030, 0x304, 0x050, 0x187, 0x044, 0x0d3, 0x005, 0x001,
+ 0x3e6, 0x08c, 0x304, 0x050, 0x189, 0x044, 0x0d3, 0x005,
+ 0x001, 0x3dd, 0x083, 0x304, 0x050, 0x18b, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3d4, 0x083, 0x18b, 0x304, 0x365, 0x044,
+ 0x0d3, 0x005, 0x001, 0x3cb, 0x387, 0x030, 0x304, 0x365,
+ 0x187, 0x044, 0x0d3, 0x005, 0x001, 0x3c1, 0x08c, 0x304,
+ 0x365, 0x189, 0x044, 0x0d3, 0x005, 0x001, 0x3b8, 0x083,
+ 0x304, 0x365, 0x18b, 0x044, 0x0d3, 0x005, 0x001, 0x3af,
+ 0x307, 0x21d, 0x304, 0x364, 0x192, 0x364, 0x052, 0x020,
+ 0x001, 0x00d, 0x327, 0x21d, 0x324, 0x04f, 0x001, 0x005,
+ 0x345, 0x21d, 0x33c, 0x1b1, 0x285, 0x044, 0x0d3, 0x005,
+ 0x001, 0x395, 0x045, 0x364, 0x33b, 0x001, 0x001, 0x02c,
+ 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x027, 0x245, 0x027, 0x345,
+ 0x21d, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x01d, 0x044,
+ 0x103, 0x005, 0x364, 0x35b, 0x002, 0x001, 0x3f5, 0x344,
+ 0x21f, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x00d, 0x044,
+ 0x103, 0x005, 0x307, 0x35b, 0x264, 0x002, 0x009, 0x3f4,
+ 0x264, 0x001, 0x045, 0x055, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x044, 0x340, 0x001, 0x247, 0x35b, 0x374, 0x027,
+ 0x200, 0x001, 0x00a, 0x264, 0x080, 0x009, 0x006, 0x3b8,
+ 0x200, 0x2c4, 0x027, 0x054, 0x045, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x001, 0x088, 0x3b8, 0x300, 0x2c4, 0x027,
+ 0x264, 0x040, 0x001, 0x00f, 0x140, 0x359, 0x001, 0x00b,
+ 0x0c0, 0x33d, 0x0c0, 0x359, 0x3d5, 0x200, 0x027, 0x000,
+ 0x060, 0x384, 0x030, 0x327, 0x359, 0x2e2, 0x008, 0x003,
+ 0x3f1, 0x2a9, 0x3b0, 0x15a, 0x041, 0x1b9, 0x3b0, 0x005,
+ 0x041, 0x045, 0x000, 0x012, 0x000, 0x015, 0x000, 0x015,
+ 0x000, 0x016, 0x000, 0x017, 0x000, 0x01a, 0x000, 0x01c,
+ 0x000, 0x01f, 0x000, 0x022, 0x187, 0x247, 0x367, 0x000,
+ 0x033, 0x000, 0x02f, 0x191, 0x000, 0x02c, 0x193, 0x000,
+ 0x029, 0x187, 0x247, 0x366, 0x000, 0x026, 0x245, 0x366,
+ 0x000, 0x022, 0x191, 0x245, 0x366, 0x000, 0x01d, 0x193,
+ 0x245, 0x366, 0x000, 0x018, 0x229, 0x3a4, 0x040, 0x2c5,
+ 0x33c, 0x329, 0x04d, 0x18b, 0x2cc, 0x04d, 0x347, 0x367,
+ 0x368, 0x347, 0x366, 0x04e, 0x0c0, 0x359, 0x000, 0x006,
+ 0x245, 0x367, 0x141, 0x359, 0x3d5, 0x300, 0x027, 0x140,
+ 0x359, 0x009, 0x011, 0x140, 0x04d, 0x00b, 0x00a, 0x364,
+ 0x33c, 0x040, 0x001, 0x005, 0x020, 0x103, 0x004, 0x044,
+ 0x137, 0x004, 0x045, 0x362, 0x33e, 0x007, 0x009, 0x003,
+ 0x045, 0x055, 0x362, 0x33d, 0x000, 0x006, 0x077, 0x362,
+ 0x33d, 0x005, 0x00f, 0x072, 0x140, 0x34d, 0x009, 0x007,
+ 0x044, 0x23e, 0x005, 0x000, 0x069, 0x308, 0x356, 0x304,
+ 0x365, 0x001, 0x007, 0x044, 0x36a, 0x005, 0x000, 0x05e,
+ 0x140, 0x349, 0x001, 0x017, 0x364, 0x04f, 0x03f, 0x009,
+ 0x055, 0x0c0, 0x349, 0x362, 0x34f, 0x000, 0x006, 0x04e,
+ 0x307, 0x34f, 0x044, 0x000, 0x006, 0x0c0, 0x34f, 0x000,
+ 0x045, 0x140, 0x33e, 0x009, 0x007, 0x044, 0x23e, 0x005,
+ 0x000, 0x03c, 0x362, 0x33e, 0x001, 0x009, 0x007, 0x044,
+ 0x269, 0x005, 0x000, 0x032, 0x362, 0x33e, 0x002, 0x009,
+ 0x007, 0x044, 0x26f, 0x005, 0x000, 0x028, 0x362, 0x33e,
+ 0x003, 0x009, 0x007, 0x044, 0x27f, 0x005, 0x000, 0x01e,
+ 0x362, 0x33e, 0x004, 0x009, 0x007, 0x044, 0x29a, 0x005,
+ 0x000, 0x014, 0x362, 0x33e, 0x005, 0x009, 0x007, 0x044,
+ 0x2b7, 0x005, 0x000, 0x00a, 0x362, 0x33e, 0x006, 0x009,
+ 0x005, 0x044, 0x34f, 0x005, 0x054, 0x045, 0x081, 0x189,
+ 0x304, 0x053, 0x009, 0x00c, 0x0c1, 0x33d, 0x083, 0x189,
+ 0x395, 0x3f7, 0x244, 0x027, 0x000, 0x01c, 0x0c2, 0x33d,
+ 0x0c1, 0x34d, 0x3c5, 0x008, 0x027, 0x0c0, 0x34b, 0x0c0,
+ 0x34c, 0x140, 0x35d, 0x009, 0x00b, 0x0c0, 0x34f, 0x088,
+ 0x189, 0x245, 0x365, 0x044, 0x3c2, 0x005, 0x0c1, 0x33e,
+ 0x045, 0x347, 0x35e, 0x34a, 0x0c2, 0x33e, 0x045, 0x307,
+ 0x34b, 0x302, 0x360, 0x006, 0x009, 0x141, 0x34b, 0x044,
+ 0x3f3, 0x005, 0x000, 0x004, 0x0c3, 0x33e, 0x045, 0x307,
+ 0x34c, 0x302, 0x361, 0x006, 0x014, 0x141, 0x34c, 0x088,
+ 0x3c7, 0x020, 0x34f, 0x3c7, 0x028, 0x372, 0x141, 0x38a,
+ 0x141, 0x382, 0x044, 0x000, 0x006, 0x000, 0x004, 0x0c4,
+ 0x33e, 0x045, 0x0c5, 0x33e, 0x0c0, 0x2d3, 0x364, 0x02b,
+ 0x080, 0x009, 0x015, 0x3c5, 0x010, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x362, 0x201, 0x002, 0x001, 0x00a, 0x0c1, 0x244,
+ 0x3c5, 0x004, 0x245, 0x3c4, 0x3fb, 0x200, 0x045, 0x364,
+ 0x2cf, 0x010, 0x009, 0x094, 0x364, 0x34a, 0x001, 0x001,
+ 0x009, 0x0c6, 0x33e, 0x044, 0x33f, 0x002, 0x000, 0x088,
+ 0x140, 0x259, 0x009, 0x009, 0x044, 0x346, 0x002, 0x0c0,
+ 0x330, 0x0c0, 0x331, 0x044, 0x012, 0x006, 0x140, 0x330,
+ 0x009, 0x00c, 0x140, 0x331, 0x009, 0x008, 0x347, 0x32e,
+ 0x330, 0x347, 0x32f, 0x331, 0x362, 0x331, 0x000, 0x00f,
+ 0x00c, 0x362, 0x32f, 0x000, 0x00f, 0x007, 0x044, 0x3f3,
+ 0x005, 0x000, 0x057, 0x362, 0x331, 0x000, 0x007, 0x00c,
+ 0x362, 0x32f, 0x000, 0x007, 0x007, 0x044, 0x3e6, 0x005,
+ 0x000, 0x048, 0x044, 0x33f, 0x002, 0x0c6, 0x33e, 0x362,
+ 0x331, 0x000, 0x00f, 0x021, 0x308, 0x330, 0x348, 0x331,
+ 0x336, 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x330, 0x347,
+ 0x336, 0x331, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302,
+ 0x32e, 0x343, 0x32f, 0x336, 0x002, 0x024, 0x044, 0x3e6,
+ 0x005, 0x000, 0x01f, 0x308, 0x32e, 0x348, 0x32f, 0x336,
+ 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x32e, 0x347, 0x336,
+ 0x32f, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302, 0x32e,
+ 0x343, 0x32f, 0x336, 0x002, 0x005, 0x044, 0x3f3, 0x005,
+ 0x347, 0x32e, 0x330, 0x347, 0x32f, 0x331, 0x045, 0x3c5,
+ 0x008, 0x33b, 0x0c3, 0x33d, 0x3c5, 0x004, 0x027, 0x0c7,
+ 0x33e, 0x364, 0x33c, 0x020, 0x001, 0x00a, 0x345, 0x21d,
+ 0x364, 0x347, 0x21d, 0x35a, 0x000, 0x005, 0x345, 0x21d,
+ 0x04d, 0x045, 0x364, 0x04f, 0x03f, 0x001, 0x054, 0x307,
+ 0x04f, 0x384, 0x003, 0x113, 0x009, 0x007, 0x3c5, 0x001,
+ 0x34a, 0x141, 0x37e, 0x307, 0x04f, 0x384, 0x003, 0x112,
+ 0x009, 0x007, 0x3c5, 0x001, 0x34a, 0x141, 0x37d, 0x307,
+ 0x04f, 0x384, 0x00c, 0x11c, 0x009, 0x007, 0x3c5, 0x002,
+ 0x34a, 0x141, 0x386, 0x307, 0x04f, 0x384, 0x00c, 0x118,
+ 0x009, 0x007, 0x3c5, 0x002, 0x34a, 0x141, 0x385, 0x307,
+ 0x04f, 0x384, 0x030, 0x382, 0x030, 0x009, 0x007, 0x3c5,
+ 0x004, 0x34a, 0x141, 0x38e, 0x307, 0x04f, 0x384, 0x030,
+ 0x382, 0x020, 0x009, 0x007, 0x3c5, 0x004, 0x34a, 0x141,
+ 0x38d, 0x347, 0x356, 0x365, 0x0c1, 0x349, 0x044, 0x3c2,
+ 0x005, 0x045, 0x088, 0x189, 0x304, 0x365, 0x001, 0x004,
+ 0x141, 0x376, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x347,
+ 0x21d, 0x35a, 0x000, 0x013, 0x347, 0x365, 0x04e, 0x307,
+ 0x04e, 0x306, 0x356, 0x001, 0x007, 0x3c5, 0x080, 0x027,
+ 0x000, 0x005, 0x3c4, 0x37f, 0x027, 0x045, 0x081, 0x0c4,
+ 0x34f, 0x0c5, 0x372, 0x141, 0x379, 0x141, 0x381, 0x044,
+ 0x000, 0x006, 0x045, 0x088, 0x0c2, 0x34f, 0x0ca, 0x372,
+ 0x141, 0x37a, 0x141, 0x382, 0x044, 0x000, 0x006, 0x045,
+ 0x364, 0x33c, 0x010, 0x001, 0x009, 0x22a, 0x3a4, 0x015,
+ 0x209, 0x384, 0x02a, 0x285, 0x245, 0x365, 0x044, 0x3c2,
+ 0x005, 0x045, 0x0c2, 0x2ce, 0x0c0, 0x304, 0x044, 0x25b,
+ 0x003, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x086, 0x189, 0x395,
+ 0x104, 0x0a2, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003,
+ 0x3c5, 0x100, 0x00d, 0x044, 0x30e, 0x002, 0x347, 0x00e,
+ 0x32e, 0x347, 0x00f, 0x32f, 0x045, 0x307, 0x000, 0x183,
+ 0x247, 0x21b, 0x003, 0x00b, 0x347, 0x21e, 0x008, 0x307,
+ 0x006, 0x327, 0x007, 0x000, 0x009, 0x347, 0x21e, 0x00b,
+ 0x307, 0x009, 0x327, 0x00a, 0x020, 0x1bb, 0x006, 0x347,
+ 0x059, 0x24b, 0x347, 0x248, 0x24a, 0x044, 0x340, 0x001,
+ 0x247, 0x248, 0x054, 0x364, 0x001, 0x080, 0x009, 0x059,
+ 0x347, 0x24a, 0x248, 0x364, 0x001, 0x001, 0x009, 0x171,
+ 0x364, 0x001, 0x002, 0x009, 0x171, 0x364, 0x001, 0x004,
+ 0x009, 0x17c, 0x364, 0x001, 0x008, 0x009, 0x1a4, 0x364,
+ 0x001, 0x010, 0x029, 0x042, 0x008, 0x364, 0x001, 0x020,
+ 0x029, 0x01f, 0x008, 0x364, 0x001, 0x040, 0x009, 0x1b1,
+ 0x364, 0x001, 0x100, 0x009, 0x1ac, 0x327, 0x21c, 0x324,
+ 0x020, 0x029, 0x0ab, 0x006, 0x364, 0x23f, 0x003, 0x001,
+ 0x00c, 0x044, 0x340, 0x001, 0x227, 0x1ba, 0x326, 0x248,
+ 0x054, 0x009, 0x026, 0x307, 0x24b, 0x306, 0x059, 0x384,
+ 0x080, 0x009, 0x01b, 0x055, 0x142, 0x000, 0x047, 0x3c5,
+ 0x020, 0x000, 0x3c5, 0x001, 0x233, 0x000, 0x012, 0x364,
+ 0x233, 0x001, 0x001, 0x00a, 0x3c5, 0x020, 0x000, 0x347,
+ 0x24a, 0x248, 0x000, 0x3e9, 0x044, 0x340, 0x001, 0x247,
+ 0x248, 0x347, 0x059, 0x24b, 0x044, 0x34c, 0x001, 0x009,
+ 0x012, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x364, 0x24b,
+ 0x080, 0x009, 0x008, 0x306, 0x249, 0x19a, 0x001, 0x3cd,
+ 0x054, 0x364, 0x23f, 0x002, 0x001, 0x052, 0x307, 0x248,
+ 0x19e, 0x00a, 0x01c, 0x3c5, 0x084, 0x021, 0x3c4, 0x3fe,
+ 0x026, 0x307, 0x248, 0x306, 0x249, 0x209, 0x00b, 0x022,
+ 0x3b7, 0x210, 0x044, 0x192, 0x000, 0x247, 0x257, 0x080,
+ 0x044, 0x1bd, 0x000, 0x000, 0x015, 0x364, 0x026, 0x001,
+ 0x009, 0x010, 0x3c4, 0x37b, 0x021, 0x364, 0x234, 0x001,
+ 0x001, 0x008, 0x3c5, 0x001, 0x026, 0x044, 0x342, 0x006,
+ 0x307, 0x248, 0x19d, 0x00a, 0x007, 0x3c4, 0x3fd, 0x026,
+ 0x000, 0x00d, 0x364, 0x234, 0x002, 0x001, 0x008, 0x3c5,
+ 0x002, 0x026, 0x044, 0x141, 0x001, 0x327, 0x248, 0x326,
+ 0x249, 0x00b, 0x005, 0x044, 0x27d, 0x001, 0x364, 0x23f,
+ 0x001, 0x001, 0x013, 0x044, 0x34c, 0x001, 0x001, 0x007,
+ 0x2a8, 0x2c4, 0x233, 0x000, 0x00b, 0x327, 0x248, 0x326,
+ 0x249, 0x1ba, 0x3a4, 0x003, 0x001, 0x01f, 0x327, 0x248,
+ 0x1ba, 0x287, 0x190, 0x002, 0x006, 0x3a4, 0x3fb, 0x000,
+ 0x004, 0x3a5, 0x004, 0x3a4, 0x007, 0x307, 0x234, 0x384,
+ 0x3f8, 0x225, 0x344, 0x21f, 0x232, 0x347, 0x248, 0x249,
+ 0x020, 0x022, 0x008, 0x347, 0x248, 0x249, 0x364, 0x33c,
+ 0x020, 0x001, 0x005, 0x044, 0x12d, 0x005, 0x364, 0x233,
+ 0x001, 0x001, 0x314, 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4,
+ 0x000, 0x328, 0x21b, 0x044, 0x32d, 0x001, 0x043, 0x307,
+ 0x21b, 0x009, 0x00e, 0x2c7, 0x005, 0x144, 0x000, 0x364,
+ 0x233, 0x001, 0x001, 0x2fb, 0x020, 0x0a3, 0x006, 0x00b,
+ 0x009, 0x0c0, 0x21b, 0x2c7, 0x00b, 0x148, 0x000, 0x047,
+ 0x0c0, 0x21b, 0x2c7, 0x008, 0x141, 0x000, 0x047, 0x3c4,
+ 0x3df, 0x000, 0x364, 0x000, 0x100, 0x001, 0x2a2, 0x364,
+ 0x233, 0x001, 0x001, 0x005, 0x3c5, 0x020, 0x000, 0x307,
+ 0x003, 0x327, 0x004, 0x041, 0x044, 0x32c, 0x001, 0x043,
+ 0x044, 0x32d, 0x001, 0x040, 0x209, 0x002, 0x083, 0x003,
+ 0x089, 0x188, 0x199, 0x041, 0x229, 0x280, 0x043, 0x390,
+ 0x057, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x3c4,
+ 0x3f3, 0x012, 0x000, 0x2b3, 0x0af, 0x1a7, 0x2a8, 0x2c4,
+ 0x01f, 0x3c5, 0x001, 0x200, 0x344, 0x21f, 0x232, 0x151,
+ 0x2fa, 0x020, 0x247, 0x002, 0x364, 0x03a, 0x040, 0x009,
+ 0x007, 0x3c4, 0x3f7, 0x03a, 0x000, 0x299, 0x3c4, 0x3f0,
+ 0x03a, 0x364, 0x2cf, 0x001, 0x001, 0x01b, 0x364, 0x2cf,
+ 0x002, 0x009, 0x016, 0x142, 0x03a, 0x141, 0x2e6, 0x3c1,
+ 0x000, 0x2e7, 0x32a, 0x03a, 0x2f4, 0x200, 0x001, 0x3fc,
+ 0x143, 0x03a, 0x374, 0x25e, 0x200, 0x009, 0x002, 0x000,
+ 0x276, 0x364, 0x03d, 0x040, 0x009, 0x007, 0x3c4, 0x3f7,
+ 0x03d, 0x000, 0x26c, 0x3c4, 0x3f0, 0x03d, 0x364, 0x33b,
+ 0x001, 0x001, 0x007, 0x044, 0x3a3, 0x003, 0x000, 0x25f,
+ 0x374, 0x25f, 0x200, 0x009, 0x002, 0x000, 0x258, 0x158,
+ 0x025, 0x044, 0x141, 0x001, 0x020, 0x0a3, 0x006, 0x042,
+ 0x3a7, 0x020, 0x1a7, 0x3a5, 0x055, 0x020, 0x1de, 0x007,
+ 0x042, 0x394, 0x3ff, 0x04d, 0x040, 0x020, 0x181, 0x006,
+ 0x042, 0x394, 0x3ff, 0x04c, 0x020, 0x1de, 0x007, 0x364,
+ 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x042, 0x186, 0x003,
+ 0x06b, 0x002, 0x06d, 0x2e4, 0x001, 0x001, 0x059, 0x287,
+ 0x384, 0x00e, 0x190, 0x3a4, 0x370, 0x285, 0x041, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x043, 0x1b3, 0x3c4, 0x3fd, 0x00c,
+ 0x3c4, 0x3fc, 0x018, 0x287, 0x384, 0x070, 0x382, 0x030,
+ 0x001, 0x007, 0x3a4, 0x007, 0x133, 0x009, 0x007, 0x3c4,
+ 0x3ef, 0x021, 0x141, 0x018, 0x142, 0x00c, 0x0a1, 0x1aa,
+ 0x364, 0x026, 0x004, 0x009, 0x004, 0x2c5, 0x02a, 0x2a8,
+ 0x082, 0x044, 0x189, 0x000, 0x2c4, 0x02a, 0x3c4, 0x3f7,
+ 0x00d, 0x3c5, 0x001, 0x00d, 0x3c4, 0x3fe, 0x00d, 0x140,
+ 0x013, 0x009, 0x00b, 0x140, 0x014, 0x009, 0x007, 0x3c5,
+ 0x008, 0x00d, 0x000, 0x014, 0x0c0, 0x012, 0x0c2, 0x012,
+ 0x0c0, 0x012, 0x0c5, 0x012, 0x000, 0x00a, 0x0c0, 0x00d,
+ 0x044, 0x33f, 0x002, 0x3c4, 0x3fd, 0x00c, 0x0a3, 0x020,
+ 0x1de, 0x007, 0x2c7, 0x013, 0x000, 0x3fa, 0x2c7, 0x014,
+ 0x000, 0x3f6, 0x020, 0x181, 0x006, 0x2e4, 0x100, 0x009,
+ 0x042, 0x041, 0x3a4, 0x007, 0x387, 0x006, 0x189, 0x395,
+ 0x321, 0x280, 0x050, 0x0c3, 0x265, 0x262, 0x005, 0x00b,
+ 0x004, 0x0c7, 0x265, 0x043, 0x1b3, 0x2e2, 0x003, 0x009,
+ 0x00b, 0x041, 0x040, 0x084, 0x0b1, 0x044, 0x1b7, 0x000,
+ 0x042, 0x043, 0x2a9, 0x349, 0x265, 0x265, 0x111, 0x00b,
+ 0x3fb, 0x2aa, 0x34a, 0x265, 0x265, 0x348, 0x265, 0x265,
+ 0x041, 0x3a7, 0x030, 0x044, 0x192, 0x000, 0x043, 0x304,
+ 0x265, 0x285, 0x3a7, 0x030, 0x044, 0x1bd, 0x000, 0x000,
+ 0x007, 0x084, 0x0b1, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x000, 0x006, 0x009, 0x002, 0x004, 0x2c7, 0x22c,
+ 0x2f4, 0x100, 0x001, 0x013, 0x044, 0x342, 0x006, 0x364,
+ 0x23f, 0x002, 0x009, 0x005, 0x020, 0x181, 0x006, 0x345,
+ 0x21d, 0x232, 0x020, 0x2ea, 0x002, 0x044, 0x1ef, 0x001,
+ 0x000, 0x3ef, 0x044, 0x1ef, 0x001, 0x091, 0x044, 0x189,
+ 0x000, 0x3b7, 0x210, 0x307, 0x257, 0x020, 0x1bd, 0x000,
+ 0x1ad, 0x1b9, 0x3c4, 0x3cf, 0x232, 0x2c5, 0x232, 0x020,
+ 0x181, 0x006, 0x287, 0x2e4, 0x100, 0x001, 0x016, 0x041,
+ 0x384, 0x001, 0x182, 0x040, 0x3b7, 0x211, 0x044, 0x1a9,
+ 0x000, 0x042, 0x385, 0x3f3, 0x3b7, 0x211, 0x044, 0x1b8,
+ 0x000, 0x043, 0x287, 0x2f4, 0x200, 0x001, 0x016, 0x384,
+ 0x010, 0x190, 0x227, 0x190, 0x285, 0x040, 0x3a7, 0x060,
+ 0x044, 0x1a9, 0x000, 0x042, 0x385, 0x3f3, 0x3a7, 0x060,
+ 0x044, 0x1b8, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x100,
+ 0x009, 0x059, 0x287, 0x384, 0x0ff, 0x264, 0x080, 0x001,
+ 0x004, 0x385, 0x300, 0x1bd, 0x3a4, 0x003, 0x009, 0x004,
+ 0x247, 0x23a, 0x131, 0x009, 0x006, 0x247, 0x23c, 0x003,
+ 0x036, 0x131, 0x009, 0x004, 0x247, 0x23b, 0x131, 0x009,
+ 0x00a, 0x247, 0x23a, 0x003, 0x02a, 0x247, 0x23c, 0x247,
+ 0x23b, 0x327, 0x23a, 0x00b, 0x004, 0x2a8, 0x121, 0x140,
+ 0x23b, 0x00b, 0x006, 0x322, 0x23b, 0x000, 0x004, 0x320,
+ 0x23b, 0x320, 0x23c, 0x322, 0x23e, 0x007, 0x010, 0x347,
+ 0x23a, 0x237, 0x347, 0x23b, 0x239, 0x347, 0x23c, 0x238,
+ 0x044, 0x05b, 0x001, 0x000, 0x038, 0x347, 0x237, 0x23a,
+ 0x347, 0x239, 0x23b, 0x347, 0x238, 0x23c, 0x020, 0x1dc,
+ 0x007, 0x140, 0x33b, 0x009, 0x015, 0x289, 0x002, 0x00c,
+ 0x003, 0x006, 0x327, 0x237, 0x000, 0x008, 0x327, 0x238,
+ 0x000, 0x004, 0x327, 0x239, 0x042, 0x020, 0x1de, 0x007,
+ 0x289, 0x002, 0x00c, 0x003, 0x006, 0x327, 0x345, 0x000,
+ 0x3f5, 0x327, 0x346, 0x000, 0x3f1, 0x327, 0x347, 0x000,
+ 0x3ed, 0x2c7, 0x01a, 0x020, 0x181, 0x006, 0x327, 0x00c,
+ 0x3c5, 0x002, 0x00c, 0x3c5, 0x001, 0x00d, 0x08f, 0x044,
+ 0x189, 0x000, 0x3c4, 0x3fe, 0x00d, 0x2c7, 0x00c, 0x020,
+ 0x181, 0x006, 0x130, 0x001, 0x013, 0x083, 0x044, 0x189,
+ 0x000, 0x3c6, 0x001, 0x02b, 0x083, 0x044, 0x189, 0x000,
+ 0x3c6, 0x001, 0x02b, 0x131, 0x009, 0x3f1, 0x020, 0x181,
+ 0x006, 0x327, 0x00c, 0x3c5, 0x002, 0x00c, 0x387, 0x07c,
+ 0x187, 0x385, 0x08f, 0x244, 0x02a, 0x088, 0x187, 0x107,
+ 0x245, 0x02a, 0x082, 0x044, 0x189, 0x000, 0x081, 0x18a,
+ 0x208, 0x244, 0x02a, 0x2c7, 0x00c, 0x020, 0x181, 0x006,
+ 0x364, 0x026, 0x002, 0x001, 0x007, 0x364, 0x012, 0x040,
+ 0x001, 0x3f8, 0x020, 0x181, 0x006, 0x042, 0x327, 0x00d,
+ 0x000, 0x16e, 0x042, 0x327, 0x012, 0x000, 0x169, 0x042,
+ 0x197, 0x264, 0x001, 0x009, 0x011, 0x264, 0x002, 0x009,
+ 0x015, 0x264, 0x004, 0x009, 0x017, 0x264, 0x010, 0x009,
+ 0x01b, 0x0a0, 0x000, 0x154, 0x325, 0x21d, 0x2c7, 0x054,
+ 0x384, 0x3fe, 0x000, 0x3e7, 0x0c0, 0x054, 0x384, 0x3fd,
+ 0x000, 0x3e1, 0x324, 0x21f, 0x2c7, 0x054, 0x384, 0x3fb,
+ 0x000, 0x3d9, 0x044, 0x048, 0x008, 0x3c6, 0x100, 0x02b,
+ 0x0c0, 0x000, 0x364, 0x000, 0x020, 0x009, 0x3f5, 0x000,
+ 0x3db, 0x042, 0x040, 0x197, 0x001, 0x00d, 0x111, 0x001,
+ 0x028, 0x111, 0x001, 0x029, 0x111, 0x001, 0x02a, 0x000,
+ 0x041, 0x2c7, 0x254, 0x3b7, 0x241, 0x387, 0x100, 0x044,
+ 0x1a9, 0x000, 0x121, 0x044, 0x192, 0x000, 0x384, 0x030,
+ 0x040, 0x132, 0x044, 0x192, 0x000, 0x3b8, 0x300, 0x284,
+ 0x043, 0x1a3, 0x285, 0x395, 0x080, 0x000, 0x01e, 0x2c7,
+ 0x254, 0x000, 0x01f, 0x2c7, 0x255, 0x000, 0x01b, 0x287,
+ 0x040, 0x3b7, 0x241, 0x044, 0x1bd, 0x000, 0x131, 0x044,
+ 0x192, 0x000, 0x3a7, 0x061, 0x1a6, 0x2a8, 0x284, 0x043,
+ 0x3a4, 0x200, 0x285, 0x3b7, 0x240, 0x044, 0x1bd, 0x000,
+ 0x020, 0x181, 0x006, 0x362, 0x391, 0x001, 0x021, 0x181,
+ 0x006, 0x0c1, 0x391, 0x042, 0x0c0, 0x38f, 0x0c0, 0x390,
+ 0x080, 0x051, 0x2c0, 0x38f, 0x3c1, 0x000, 0x390, 0x3c6,
+ 0x0d8, 0x38f, 0x349, 0x38f, 0x38f, 0x34b, 0x390, 0x390,
+ 0x00a, 0x008, 0x3c0, 0x0e5, 0x38f, 0x3c1, 0x000, 0x390,
+ 0x3b7, 0x00b, 0x1a9, 0x3b5, 0x382, 0x222, 0x001, 0x005,
+ 0x101, 0x000, 0x3e0, 0x101, 0x051, 0x326, 0x38f, 0x3a4,
+ 0x0ff, 0x2e2, 0x0ef, 0x009, 0x029, 0x101, 0x051, 0x040,
+ 0x307, 0x38f, 0x197, 0x226, 0x2e2, 0x0be, 0x009, 0x01e,
+ 0x042, 0x101, 0x051, 0x326, 0x390, 0x3a4, 0x0ff, 0x2e2,
+ 0x0ad, 0x009, 0x013, 0x101, 0x050, 0x327, 0x390, 0x1b7,
+ 0x286, 0x262, 0x0de, 0x009, 0x009, 0x080, 0x040, 0x0c0,
+ 0x391, 0x020, 0x181, 0x006, 0x387, 0x0ff, 0x000, 0x3f8,
+ 0x043, 0x0a0, 0x1a7, 0x3a5, 0x045, 0x000, 0x071, 0x345,
+ 0x221, 0x233, 0x28a, 0x00a, 0x00c, 0x3a4, 0x300, 0x3c4,
+ 0x0ff, 0x027, 0x2c5, 0x027, 0x020, 0x181, 0x006, 0x327,
+ 0x059, 0x1a7, 0x042, 0x000, 0x05b, 0x2e4, 0x020, 0x001,
+ 0x012, 0x081, 0x18c, 0x245, 0x233, 0x2e4, 0x010, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x044, 0x11a, 0x001, 0x000,
+ 0x042, 0x041, 0x091, 0x189, 0x2e4, 0x002, 0x001, 0x015,
+ 0x3c5, 0x040, 0x233, 0x245, 0x024, 0x364, 0x026, 0x002,
+ 0x001, 0x01d, 0x3a7, 0x022, 0x397, 0x200, 0x044, 0x1b7,
+ 0x000, 0x000, 0x014, 0x3c4, 0x3bf, 0x233, 0x364, 0x026,
+ 0x002, 0x001, 0x00c, 0x208, 0x244, 0x024, 0x3a7, 0x022,
+ 0x397, 0x200, 0x044, 0x1a9, 0x000, 0x043, 0x081, 0x189,
+ 0x2e4, 0x001, 0x001, 0x009, 0x3a7, 0x020, 0x044, 0x1a9,
+ 0x000, 0x000, 0x008, 0x3a7, 0x020, 0x208, 0x044, 0x1b8,
+ 0x000, 0x020, 0x181, 0x006, 0x043, 0x0a0, 0x041, 0x020,
+ 0x181, 0x006, 0x2e2, 0x00f, 0x001, 0x060, 0x307, 0x2cf,
+ 0x0c0, 0x2d3, 0x120, 0x001, 0x05f, 0x00b, 0x018, 0x0c1,
+ 0x2d3, 0x3c4, 0x00f, 0x2cf, 0x3c5, 0x100, 0x200, 0x000,
+ 0x048, 0x0c0, 0x2d8, 0x3a4, 0x3f7, 0x001, 0x042, 0x0c1,
+ 0x2d8, 0x2c7, 0x2da, 0x000, 0x03c, 0x2e4, 0x008, 0x009,
+ 0x3f2, 0x2e2, 0x004, 0x009, 0x021, 0x364, 0x026, 0x002,
+ 0x001, 0x3cc, 0x364, 0x027, 0x010, 0x001, 0x3c7, 0x140,
+ 0x2f9, 0x009, 0x3c3, 0x140, 0x2d8, 0x001, 0x3bf, 0x327,
+ 0x2da, 0x364, 0x026, 0x002, 0x001, 0x3b8, 0x364, 0x2cf,
+ 0x073, 0x029, 0x1dc, 0x007, 0x3c4, 0x08f, 0x2cf, 0x2e4,
+ 0x003, 0x001, 0x010, 0x287, 0x1b5, 0x2c7, 0x2d4, 0x384,
+ 0x007, 0x183, 0x245, 0x2cf, 0x3c5, 0x040, 0x200, 0x0c0,
+ 0x2f9, 0x020, 0x181, 0x006, 0x30a, 0x2cf, 0x384, 0x020,
+ 0x000, 0x3f2, 0x347, 0x2cf, 0x2f9, 0x3c4, 0x3df, 0x2cf,
+ 0x000, 0x3f1, 0x042, 0x2c7, 0x269, 0x287, 0x384, 0x0ff,
+ 0x1a1, 0x002, 0x00b, 0x00b, 0x014, 0x247, 0x28d, 0x044,
+ 0x2fb, 0x002, 0x000, 0x005, 0x044, 0x125, 0x003, 0x140,
+ 0x269, 0x00b, 0x01b, 0x0a1, 0x044, 0x30f, 0x002, 0x140,
+ 0x269, 0x00b, 0x013, 0x364, 0x012, 0x040, 0x001, 0x3fd,
+ 0x347, 0x00e, 0x266, 0x347, 0x00f, 0x267, 0x397, 0x200,
+ 0x0a1, 0x044, 0x0fb, 0x002, 0x347, 0x28d, 0x268, 0x307,
+ 0x269, 0x197, 0x384, 0x003, 0x390, 0x266, 0x04c, 0x041,
+ 0x020, 0x181, 0x006, 0x042, 0x247, 0x265, 0x041, 0x397,
+ 0x007, 0x189, 0x395, 0x2ca, 0x1bb, 0x3a4, 0x007, 0x280,
+ 0x050, 0x043, 0x041, 0x1b7, 0x3a4, 0x00f, 0x280, 0x043,
+ 0x364, 0x265, 0x100, 0x009, 0x01b, 0x120, 0x00b, 0x008,
+ 0x3a4, 0x0ff, 0x2a8, 0x121, 0x000, 0x004, 0x3a4, 0x0ff,
+ 0x04d, 0x041, 0x044, 0x3f0, 0x002, 0x044, 0x3de, 0x002,
+ 0x044, 0x06e, 0x003, 0x020, 0x181, 0x006, 0x04c, 0x041,
+ 0x000, 0x3fb, 0x2c4, 0x2a1, 0x28e, 0x293, 0x2b2, 0x2d4,
+ 0x2e4, 0x347, 0x22e, 0x22f, 0x2c7, 0x22e, 0x2e4, 0x040,
+ 0x001, 0x01a, 0x3a7, 0x080, 0x044, 0x192, 0x000, 0x384,
+ 0x3c3, 0x327, 0x22e, 0x1b5, 0x3a4, 0x03c, 0x285, 0x3a7,
+ 0x080, 0x044, 0x1bd, 0x000, 0x347, 0x22f, 0x22e, 0x020,
+ 0x181, 0x006, 0x3c5, 0x004, 0x233, 0x364, 0x22e, 0x001,
+ 0x009, 0x005, 0x3c4, 0x3fb, 0x233, 0x364, 0x22e, 0x020,
+ 0x009, 0x005, 0x080, 0x000, 0x003, 0x081, 0x044, 0x254,
+ 0x001, 0x364, 0x22e, 0x001, 0x001, 0x007, 0x0c8, 0x291,
+ 0x044, 0x06e, 0x003, 0x307, 0x22e, 0x20a, 0x384, 0x001,
+ 0x044, 0x045, 0x008, 0x020, 0x181, 0x006, 0x2c7, 0x223,
+ 0x0c0, 0x224, 0x000, 0x01e, 0x309, 0x223, 0x002, 0x01d,
+ 0x003, 0x020, 0x001, 0x030, 0x362, 0x224, 0x001, 0x003,
+ 0x011, 0x112, 0x001, 0x01f, 0x112, 0x001, 0x057, 0x116,
+ 0x001, 0x06a, 0x114, 0x001, 0x055, 0x112, 0x001, 0x060,
+ 0x020, 0x181, 0x006, 0x20e, 0x384, 0x0ff, 0x000, 0x055,
+ 0x041, 0x229, 0x1b1, 0x042, 0x044, 0x1bd, 0x000, 0x000,
+ 0x3f1, 0x344, 0x21f, 0x02a, 0x083, 0x18b, 0x245, 0x02a,
+ 0x000, 0x005, 0x344, 0x21f, 0x029, 0x3b4, 0x3ff, 0x307,
+ 0x224, 0x262, 0x007, 0x00e, 0x3dd, 0x20a, 0x183, 0x247,
+ 0x02d, 0x30a, 0x224, 0x002, 0x006, 0x2c7, 0x02c, 0x000,
+ 0x009, 0x287, 0x189, 0x245, 0x02c, 0x1b5, 0x2c5, 0x02d,
+ 0x307, 0x223, 0x001, 0x00a, 0x3c5, 0x080, 0x02d, 0x3c4,
+ 0x37f, 0x02d, 0x000, 0x030, 0x3c5, 0x040, 0x02d, 0x3c4,
+ 0x3bf, 0x02d, 0x000, 0x028, 0x397, 0x013, 0x000, 0x004,
+ 0x397, 0x2c7, 0x300, 0x224, 0x140, 0x224, 0x009, 0x005,
+ 0x04d, 0x101, 0x0a0, 0x04d, 0x000, 0x016, 0x2c7, 0x2cb,
+ 0x000, 0x3a0, 0x140, 0x224, 0x009, 0x006, 0x2c7, 0x019,
+ 0x000, 0x00a, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x1ab,
+ 0x2c5, 0x018, 0x141, 0x224, 0x000, 0x38c, 0x042, 0x309,
+ 0x223, 0x002, 0x03e, 0x003, 0x043, 0x116, 0x001, 0x049,
+ 0x112, 0x001, 0x00e, 0x114, 0x029, 0x1dd, 0x007, 0x397,
+ 0x200, 0x300, 0x224, 0x04c, 0x041, 0x000, 0x3e5, 0x307,
+ 0x224, 0x262, 0x007, 0x02e, 0x1dd, 0x007, 0x364, 0x026,
+ 0x004, 0x029, 0x1dd, 0x007, 0x20a, 0x187, 0x0a3, 0x1a7,
+ 0x2a8, 0x2c4, 0x02a, 0x245, 0x02a, 0x327, 0x02e, 0x307,
+ 0x224, 0x20a, 0x00a, 0x009, 0x1b9, 0x307, 0x02f, 0x384,
+ 0x00f, 0x185, 0x225, 0x3b4, 0x3ff, 0x000, 0x022, 0x20a,
+ 0x384, 0x0ff, 0x04c, 0x041, 0x000, 0x344, 0x304, 0x21f,
+ 0x22e, 0x044, 0x192, 0x000, 0x040, 0x000, 0x33b, 0x30a,
+ 0x224, 0x002, 0x011, 0x307, 0x00d, 0x3c4, 0x3f7, 0x00d,
+ 0x347, 0x00f, 0x26a, 0x327, 0x00e, 0x247, 0x00d, 0x041,
+ 0x000, 0x39a, 0x327, 0x26a, 0x000, 0x3fb, 0x04e, 0x020,
+ 0x23b, 0x009, 0x020, 0x28e, 0x00a, 0x020, 0x03a, 0x00a,
+ 0x020, 0x0fe, 0x00a, 0x020, 0x3f3, 0x008, 0x020, 0x013,
+ 0x009, 0x020, 0x075, 0x00a, 0x0c1, 0x244, 0x3c5, 0x040,
+ 0x245, 0x3c4, 0x3bf, 0x200, 0x044, 0x172, 0x000, 0x020,
+ 0x0d3, 0x000, 0x020, 0x2c3, 0x008, 0x020, 0x375, 0x00b,
+ 0x0c1, 0x201, 0x045, 0x020, 0x04a, 0x009, 0x020, 0x04e,
+ 0x009, 0x020, 0x256, 0x002, 0x020, 0x17f, 0x009, 0x020,
+ 0x23f, 0x006, 0x020, 0x285, 0x00a, 0x020, 0x209, 0x00b,
+ 0x020, 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x394,
+ 0x00a, 0x020, 0x048, 0x00b, 0x020, 0x1dc, 0x007, 0x020,
+ 0x27f, 0x00a, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007,
+ 0x020, 0x295, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x277,
+ 0x00b, 0x020, 0x2ba, 0x00b, 0x020, 0x2d5, 0x006, 0x020,
+ 0x326, 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007,
+ 0x020, 0x35a, 0x006, 0x020, 0x0a4, 0x00b, 0x020, 0x38e,
+ 0x006, 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020,
+ 0x31e, 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007,
+ 0x020, 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060,
+ 0x007, 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020,
+ 0x2d1, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252,
+ 0x007, 0x020, 0x293, 0x007, 0x020, 0x2d8, 0x00b, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007,
+ 0x020, 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x30e, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007,
+ 0x020, 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5,
+ 0x008, 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287,
+ 0x264, 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5,
+ 0x020, 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004,
+ 0x3a4, 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009,
+ 0x093, 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe,
+ 0x052, 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de,
+ 0x007, 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c,
+ 0x100, 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c,
+ 0x3d5, 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066,
+ 0x004, 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b,
+ 0x262, 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4,
+ 0x3ef, 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009,
+ 0x007, 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200,
+ 0x004, 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001,
+ 0x028, 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344,
+ 0x21f, 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183,
+ 0x19b, 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001,
+ 0x004, 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5,
+ 0x080, 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044,
+ 0x001, 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5,
+ 0x004, 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044,
+ 0x097, 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d,
+ 0x001, 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390,
+ 0x1d7, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000,
+ 0x021, 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000,
+ 0x025, 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000,
+ 0x041, 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000,
+ 0x031, 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007,
+ 0x044, 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072,
+ 0x2c7, 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a,
+ 0x2c7, 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062,
+ 0x2c7, 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a,
+ 0x2c7, 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052,
+ 0x2c7, 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a,
+ 0x307, 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007,
+ 0x006, 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020,
+ 0x1de, 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3,
+ 0x052, 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c,
+ 0x2c5, 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7,
+ 0x3ff, 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf,
+ 0x354, 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222,
+ 0x355, 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a,
+ 0x006, 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357,
+ 0x020, 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4,
+ 0x00f, 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020,
+ 0x131, 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001,
+ 0x025, 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132,
+ 0x182, 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de,
+ 0x007, 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373,
+ 0x000, 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3,
+ 0x322, 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7,
+ 0x327, 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df,
+ 0x307, 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108,
+ 0x04c, 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b,
+ 0x364, 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f,
+ 0x181, 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5,
+ 0x080, 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7,
+ 0x392, 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5,
+ 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364,
+ 0x33c, 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000,
+ 0x018, 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140,
+ 0x04f, 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3,
+ 0x2c5, 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081,
+ 0x189, 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027,
+ 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053,
+ 0x3c4, 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004,
+ 0x020, 0x08d, 0x006, 0x2c7, 0x231, 0x2e4, 0x100, 0x001,
+ 0x045, 0x2e4, 0x001, 0x001, 0x020, 0x3b7, 0x213, 0x084,
+ 0x044, 0x1a9, 0x000, 0x3b7, 0x250, 0x084, 0x044, 0x1a9,
+ 0x000, 0x399, 0x305, 0x3a7, 0x022, 0x044, 0x1a9, 0x000,
+ 0x044, 0x322, 0x001, 0x3c5, 0x002, 0x024, 0x3c5, 0x008,
+ 0x292, 0x000, 0x020, 0x3c4, 0x3fd, 0x024, 0x364, 0x234,
+ 0x002, 0x001, 0x009, 0x3a7, 0x022, 0x399, 0x305, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x213, 0x084, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x250, 0x084, 0x044, 0x1b7, 0x000, 0x3c4, 0x007,
+ 0x292, 0x044, 0x06e, 0x003, 0x307, 0x231, 0x274, 0x200,
+ 0x001, 0x01b, 0x274, 0x010, 0x001, 0x00b, 0x387, 0x020,
+ 0x245, 0x021, 0x185, 0x245, 0x024, 0x000, 0x00b, 0x387,
+ 0x3df, 0x244, 0x021, 0x185, 0x385, 0x0ff, 0x244, 0x024,
+ 0x044, 0x03a, 0x00a, 0x045, 0x327, 0x230, 0x364, 0x233,
+ 0x004, 0x001, 0x003, 0x045, 0x0c2, 0x22a, 0x347, 0x275,
+ 0x270, 0x0c0, 0x27b, 0x264, 0x010, 0x009, 0x00a, 0x0c1,
+ 0x22a, 0x347, 0x274, 0x270, 0x3d7, 0x200, 0x27b, 0x327,
+ 0x230, 0x264, 0x004, 0x001, 0x012, 0x347, 0x277, 0x270,
+ 0x264, 0x010, 0x009, 0x007, 0x347, 0x276, 0x270, 0x384,
+ 0x00f, 0x112, 0x1b7, 0x000, 0x004, 0x3a4, 0x0ff, 0x2c7,
+ 0x225, 0x384, 0x00f, 0x262, 0x003, 0x001, 0x00b, 0x101,
+ 0x3c2, 0x100, 0x270, 0x34a, 0x225, 0x225, 0x000, 0x3f3,
+ 0x044, 0x237, 0x00a, 0x044, 0x35f, 0x009, 0x364, 0x22a,
+ 0x002, 0x001, 0x006, 0x347, 0x270, 0x272, 0x045, 0x347,
+ 0x270, 0x271, 0x045, 0x044, 0x2f6, 0x002, 0x009, 0x015,
+ 0x307, 0x241, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x3d7, 0x200, 0x27b,
+ 0x044, 0x35f, 0x009, 0x347, 0x241, 0x226, 0x0c1, 0x22a,
+ 0x020, 0x032, 0x009, 0x044, 0x2f6, 0x002, 0x009, 0x014,
+ 0x307, 0x243, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x0c0, 0x27b, 0x044,
+ 0x35f, 0x009, 0x347, 0x243, 0x226, 0x0c2, 0x22a, 0x020,
+ 0x032, 0x009, 0x327, 0x226, 0x2c7, 0x225, 0x3c4, 0x0ff,
+ 0x225, 0x1b7, 0x3a4, 0x007, 0x001, 0x00d, 0x34a, 0x225,
+ 0x225, 0x002, 0x008, 0x131, 0x009, 0x3fa, 0x044, 0x237,
+ 0x00a, 0x045, 0x0c1, 0x26f, 0x000, 0x050, 0x327, 0x01d,
+ 0x2e4, 0x380, 0x009, 0x00a, 0x13a, 0x00b, 0x00a, 0x3c5,
+ 0x008, 0x233, 0x000, 0x005, 0x3c4, 0x3f7, 0x233, 0x327,
+ 0x01d, 0x021, 0x2ed, 0x009, 0x364, 0x020, 0x004, 0x009,
+ 0x01a, 0x309, 0x232, 0x00b, 0x009, 0x347, 0x270, 0x271,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x271, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x04c, 0x347, 0x270, 0x271, 0x000,
+ 0x018, 0x309, 0x232, 0x001, 0x009, 0x347, 0x270, 0x272,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x272, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x089, 0x347, 0x270, 0x272, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x364, 0x26f, 0x002, 0x009,
+ 0x005, 0x3c4, 0x3fe, 0x200, 0x364, 0x020, 0x005, 0x029,
+ 0x2e4, 0x002, 0x364, 0x020, 0x010, 0x009, 0x0e9, 0x364,
+ 0x020, 0x100, 0x009, 0x12b, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3f7, 0x233, 0x364, 0x020, 0x001, 0x009, 0x00c, 0x364,
+ 0x020, 0x004, 0x009, 0x05d, 0x000, 0x3d8, 0x3c5, 0x080,
+ 0x020, 0x364, 0x233, 0x006, 0x001, 0x031, 0x081, 0x189,
+ 0x304, 0x232, 0x009, 0x017, 0x347, 0x271, 0x274, 0x080,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x085, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245, 0x232, 0x000,
+ 0x0c8, 0x347, 0x271, 0x276, 0x081, 0x189, 0x208, 0x244,
+ 0x232, 0x307, 0x22b, 0x197, 0x384, 0x007, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x000, 0x005, 0x044, 0x3f3, 0x008,
+ 0x3b7, 0x200, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x203, 0x387, 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020,
+ 0x385, 0x042, 0x384, 0x3ce, 0x247, 0x020, 0x044, 0x044,
+ 0x001, 0x000, 0x383, 0x0a1, 0x1aa, 0x2c5, 0x020, 0x364,
+ 0x233, 0x006, 0x001, 0x039, 0x081, 0x189, 0x304, 0x232,
+ 0x009, 0x017, 0x347, 0x272, 0x275, 0x080, 0x364, 0x233,
+ 0x004, 0x009, 0x003, 0x095, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x081, 0x189, 0x245, 0x232, 0x000, 0x0ce, 0x347,
+ 0x272, 0x277, 0x081, 0x189, 0x208, 0x244, 0x232, 0x307,
+ 0x22b, 0x197, 0x364, 0x233, 0x002, 0x001, 0x007, 0x193,
+ 0x384, 0x007, 0x385, 0x010, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x000, 0x008, 0x347, 0x272, 0x270, 0x044, 0x013,
+ 0x009, 0x081, 0x18a, 0x208, 0x244, 0x232, 0x0a3, 0x387,
+ 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020, 0x0a4, 0x1a7,
+ 0x3a5, 0x008, 0x285, 0x3b7, 0x304, 0x2a8, 0x284, 0x247,
+ 0x020, 0x0a0, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x044,
+ 0x2f6, 0x002, 0x009, 0x00f, 0x364, 0x2cf, 0x080, 0x009,
+ 0x00a, 0x0c3, 0x26f, 0x044, 0x1b6, 0x001, 0x020, 0x2e4,
+ 0x002, 0x044, 0x051, 0x001, 0x000, 0x308, 0x3c4, 0x3f7,
+ 0x233, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x083, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x3d7,
+ 0x200, 0x27b, 0x0c0, 0x271, 0x3b7, 0x300, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x221, 0x091, 0x189, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x201, 0x387, 0x0a6, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x021, 0x020, 0x3c4, 0x3ed, 0x020, 0x3b7,
+ 0x200, 0x387, 0x020, 0x044, 0x020, 0x00a, 0x009, 0x32a,
+ 0x347, 0x271, 0x270, 0x000, 0x055, 0x3a7, 0x071, 0x307,
+ 0x21d, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x233, 0x044,
+ 0x2f6, 0x002, 0x009, 0x007, 0x0c1, 0x2d3, 0x3c5, 0x100,
+ 0x200, 0x081, 0x18a, 0x245, 0x232, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x093, 0x0c2, 0x22a,
+ 0x044, 0x39c, 0x008, 0x0c0, 0x27b, 0x0c0, 0x272, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1, 0x397,
+ 0x205, 0x247, 0x27f, 0x044, 0x1bd, 0x000, 0x3d5, 0x204,
+ 0x020, 0x3c4, 0x2ff, 0x020, 0x0a0, 0x387, 0x021, 0x184,
+ 0x044, 0x020, 0x00a, 0x009, 0x336, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3fe, 0x200, 0x3c4, 0x3fe,
+ 0x26f, 0x344, 0x21f, 0x232, 0x0c0, 0x020, 0x0a3, 0x1ab,
+ 0x2c5, 0x020, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000,
+ 0x044, 0x3b5, 0x000, 0x044, 0x3f0, 0x000, 0x0a9, 0x1ab,
+ 0x09f, 0x18a, 0x208, 0x304, 0x055, 0x225, 0x044, 0x394,
+ 0x00a, 0x307, 0x055, 0x19b, 0x384, 0x003, 0x227, 0x1a3,
+ 0x225, 0x044, 0x0a4, 0x00b, 0x0a7, 0x020, 0x022, 0x008,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x006, 0x387, 0x07f,
+ 0x000, 0x013, 0x399, 0x240, 0x385, 0x07f, 0x306, 0x270,
+ 0x001, 0x06c, 0x399, 0x240, 0x327, 0x270, 0x3a4, 0x380,
+ 0x226, 0x001, 0x015, 0x327, 0x220, 0x324, 0x232, 0x009,
+ 0x019, 0x364, 0x232, 0x001, 0x001, 0x02b, 0x327, 0x01d,
+ 0x307, 0x01b, 0x190, 0x282, 0x003, 0x041, 0x307, 0x01b,
+ 0x191, 0x380, 0x010, 0x282, 0x003, 0x034, 0x000, 0x019,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x00a, 0x327, 0x270,
+ 0x384, 0x07f, 0x3a4, 0x07f, 0x000, 0x008, 0x327, 0x270,
+ 0x3a4, 0x07f, 0x001, 0x005, 0x286, 0x001, 0x02f, 0x307,
+ 0x270, 0x384, 0x07f, 0x386, 0x07f, 0x009, 0x00a, 0x387,
+ 0x080, 0x382, 0x07f, 0x240, 0x270, 0x000, 0x01c, 0x141,
+ 0x270, 0x364, 0x270, 0x07f, 0x009, 0x015, 0x000, 0x013,
+ 0x3c0, 0x040, 0x270, 0x000, 0x00e, 0x307, 0x270, 0x196,
+ 0x384, 0x00f, 0x266, 0x009, 0x001, 0x008, 0x3c0, 0x080,
+ 0x270, 0x044, 0x35f, 0x009, 0x045, 0x309, 0x232, 0x00b,
+ 0x013, 0x347, 0x271, 0x270, 0x374, 0x27b, 0x200, 0x009,
+ 0x005, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x020,
+ 0x0b7, 0x009, 0x327, 0x232, 0x1a2, 0x023, 0x0b7, 0x009,
+ 0x307, 0x270, 0x196, 0x119, 0x021, 0x0b7, 0x009, 0x307,
+ 0x232, 0x181, 0x023, 0x0b7, 0x009, 0x364, 0x270, 0x080,
+ 0x029, 0x0b7, 0x009, 0x083, 0x18c, 0x245, 0x232, 0x3c4,
+ 0x3f7, 0x233, 0x3c4, 0x380, 0x270, 0x3c0, 0x080, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x09c, 0x009, 0x000, 0x000, 0x000, 0x002, 0x000,
+ 0x00a, 0x000, 0x02a, 0x000, 0x0aa, 0x002, 0x0aa, 0x00a,
+ 0x0aa, 0x02a, 0x0aa, 0x06a, 0x0aa, 0x07a, 0x0aa, 0x07e,
+ 0x0aa, 0x07f, 0x0aa, 0x07f, 0x0ba, 0x07f, 0x0be, 0x07f,
+ 0x0fe, 0x07f, 0x0ff, 0x007, 0x000, 0x017, 0x001, 0x027,
+ 0x007, 0x037, 0x00f, 0x047, 0x03f, 0x080, 0x0ff, 0x364,
+ 0x270, 0x07f, 0x307, 0x270, 0x384, 0x07f, 0x009, 0x00d,
+ 0x089, 0x186, 0x306, 0x270, 0x001, 0x007, 0x3c5, 0x001,
+ 0x232, 0x000, 0x005, 0x3c4, 0x3fe, 0x232, 0x347, 0x270,
+ 0x27c, 0x3c4, 0x007, 0x27c, 0x374, 0x27b, 0x200, 0x001,
+ 0x00a, 0x364, 0x233, 0x008, 0x001, 0x005, 0x044, 0x0fe,
+ 0x00a, 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384,
+ 0x3f8, 0x305, 0x27c, 0x044, 0x1bd, 0x000, 0x307, 0x270,
+ 0x192, 0x384, 0x00f, 0x3b9, 0x200, 0x324, 0x233, 0x001,
+ 0x014, 0x229, 0x387, 0x009, 0x189, 0x395, 0x333, 0x280,
+ 0x051, 0x2c7, 0x27d, 0x1c7, 0x27d, 0x101, 0x051, 0x2c5,
+ 0x27d, 0x000, 0x011, 0x0c0, 0x27d, 0x384, 0x00f, 0x001,
+ 0x00b, 0x349, 0x27d, 0x27d, 0x3c5, 0x001, 0x27d, 0x111,
+ 0x000, 0x3f7, 0x0a2, 0x325, 0x27b, 0x307, 0x27d, 0x044,
+ 0x1bd, 0x000, 0x374, 0x27b, 0x200, 0x001, 0x02c, 0x347,
+ 0x270, 0x27a, 0x3c4, 0x07f, 0x27a, 0x0c0, 0x279, 0x387,
+ 0x009, 0x189, 0x395, 0x353, 0x051, 0x041, 0x101, 0x051,
+ 0x2c7, 0x279, 0x101, 0x043, 0x322, 0x27a, 0x00b, 0x004,
+ 0x000, 0x3f4, 0x1c7, 0x279, 0x0a3, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x384, 0x0ff, 0x305, 0x279, 0x044, 0x1bd,
+ 0x000, 0x327, 0x220, 0x324, 0x232, 0x009, 0x022, 0x0c0,
+ 0x27e, 0x307, 0x270, 0x197, 0x00a, 0x004, 0x000, 0x005,
+ 0x3c5, 0x018, 0x27e, 0x0a4, 0x222, 0x2c5, 0x27e, 0x0a4,
+ 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x3e0, 0x3c4,
+ 0x01f, 0x27e, 0x305, 0x27e, 0x044, 0x1bd, 0x000, 0x045,
+ 0x044, 0x1a9, 0x000, 0x083, 0x18c, 0x208, 0x244, 0x232,
+ 0x3c5, 0x001, 0x00c, 0x0a3, 0x325, 0x27b, 0x088, 0x044,
+ 0x1a9, 0x000, 0x387, 0x3f8, 0x044, 0x1b8, 0x000, 0x020,
+ 0x2f6, 0x002, 0x3c7, 0x020, 0x280, 0x3b7, 0x215, 0x044,
+ 0x192, 0x000, 0x247, 0x247, 0x151, 0x280, 0x001, 0x016,
+ 0x0a1, 0x044, 0x182, 0x000, 0x3b7, 0x215, 0x044, 0x192,
+ 0x000, 0x306, 0x247, 0x001, 0x3f1, 0x0a2, 0x364, 0x22b,
+ 0x003, 0x001, 0x003, 0x0a5, 0x020, 0x182, 0x000, 0x1f6,
+ 0x185, 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122,
+ 0x0ff, 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194,
+ 0x0ce, 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020,
+ 0x009, 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9,
+ 0x286, 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x05f,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051,
+ 0x2c7, 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f,
+ 0x040, 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b,
+ 0x005, 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4,
+ 0x00f, 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288,
+ 0x286, 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b,
+ 0x3fc, 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285,
+ 0x307, 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd,
+ 0x000, 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2,
+ 0x12c, 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002,
+ 0x005, 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b,
+ 0x1f0, 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081,
+ 0x138, 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f,
+ 0x080, 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d,
+ 0x1e0, 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b,
+ 0x347, 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044,
+ 0x1bc, 0x00a, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364,
+ 0x233, 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff,
+ 0x242, 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289,
+ 0x397, 0x00a, 0x189, 0x395, 0x0ca, 0x044, 0x2ea, 0x001,
+ 0x009, 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9,
+ 0x121, 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f,
+ 0x282, 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000,
+ 0x3ed, 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233,
+ 0x002, 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff,
+ 0x281, 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366,
+ 0x282, 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282,
+ 0x044, 0x1bc, 0x00a, 0x001, 0x00d, 0x141, 0x28b, 0x362,
+ 0x28b, 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5,
+ 0x362, 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397,
+ 0x00a, 0x189, 0x395, 0x0d8, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005,
+ 0x105, 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f,
+ 0x121, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042,
+ 0x003, 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387,
+ 0x00a, 0x189, 0x395, 0x0d3, 0x300, 0x28a, 0x050, 0x247,
+ 0x28a, 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a,
+ 0x044, 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282,
+ 0x281, 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045,
+ 0x000, 0x301, 0x103, 0x307, 0x006, 0x10e, 0x21e, 0x33e,
+ 0x13c, 0x038, 0x078, 0x1f8, 0x0f0, 0x000, 0x001, 0x003,
+ 0x002, 0x006, 0x187, 0x227, 0x387, 0x00a, 0x189, 0x395,
+ 0x1c8, 0x300, 0x227, 0x111, 0x050, 0x247, 0x24c, 0x1d7,
+ 0x24c, 0x384, 0x0ff, 0x285, 0x044, 0x226, 0x00a, 0x364,
+ 0x22a, 0x001, 0x001, 0x00e, 0x3d5, 0x200, 0x232, 0x040,
+ 0x1aa, 0x285, 0x3b7, 0x220, 0x044, 0x1bd, 0x000, 0x042,
+ 0x364, 0x22a, 0x002, 0x001, 0x022, 0x040, 0x3a7, 0x073,
+ 0x044, 0x192, 0x000, 0x384, 0x31f, 0x044, 0x226, 0x00a,
+ 0x1a4, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000, 0x3a7,
+ 0x070, 0x044, 0x192, 0x000, 0x19a, 0x18a, 0x043, 0x285,
+ 0x3a7, 0x070, 0x044, 0x1bd, 0x000, 0x045, 0x040, 0x387,
+ 0x00a, 0x189, 0x395, 0x1d5, 0x300, 0x278, 0x051, 0x042,
+ 0x045, 0x00a, 0x058, 0x026, 0x015, 0x044, 0x032, 0x0c0,
+ 0x278, 0x0a0, 0x397, 0x00a, 0x189, 0x395, 0x231, 0x280,
+ 0x050, 0x247, 0x228, 0x3c4, 0x00f, 0x228, 0x347, 0x228,
+ 0x229, 0x040, 0x307, 0x278, 0x001, 0x008, 0x340, 0x229,
+ 0x228, 0x111, 0x000, 0x3fa, 0x042, 0x193, 0x0c0, 0x227,
+ 0x347, 0x225, 0x226, 0x141, 0x227, 0x366, 0x227, 0x00e,
+ 0x001, 0x00b, 0x342, 0x228, 0x226, 0x007, 0x3f6, 0x140,
+ 0x226, 0x001, 0x011, 0x121, 0x2e2, 0x006, 0x009, 0x3cc,
+ 0x141, 0x278, 0x366, 0x278, 0x005, 0x009, 0x3c4, 0x0a1,
+ 0x000, 0x006, 0x044, 0x1da, 0x00a, 0x0a0, 0x045, 0x044,
+ 0x333, 0x008, 0x020, 0x181, 0x006, 0x345, 0x21d, 0x232,
+ 0x364, 0x23f, 0x001, 0x029, 0x2e4, 0x002, 0x2c7, 0x234,
+ 0x399, 0x200, 0x245, 0x233, 0x364, 0x234, 0x020, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x083, 0x18c, 0x208, 0x244,
+ 0x036, 0x2e4, 0x008, 0x001, 0x007, 0x345, 0x220, 0x232,
+ 0x000, 0x007, 0x081, 0x18b, 0x208, 0x244, 0x232, 0x307,
+ 0x235, 0x226, 0x3a4, 0x001, 0x001, 0x010, 0x327, 0x234,
+ 0x2e4, 0x001, 0x009, 0x041, 0x041, 0x044, 0x20d, 0x000,
+ 0x044, 0x2f3, 0x000, 0x043, 0x0a3, 0x1ad, 0x3a5, 0x039,
+ 0x326, 0x236, 0x001, 0x02f, 0x3a7, 0x0bf, 0x1a7, 0x3a5,
+ 0x039, 0x326, 0x236, 0x001, 0x026, 0x327, 0x234, 0x307,
+ 0x235, 0x286, 0x384, 0x002, 0x001, 0x01d, 0x2e4, 0x002,
+ 0x009, 0x051, 0x041, 0x0a1, 0x1ad, 0x2c5, 0x024, 0x307,
+ 0x024, 0x19a, 0x002, 0x005, 0x2a8, 0x2c4, 0x024, 0x044,
+ 0x3b5, 0x000, 0x044, 0x0f2, 0x001, 0x044, 0x3f0, 0x000,
+ 0x043, 0x000, 0x067, 0x364, 0x026, 0x001, 0x009, 0x3c6,
+ 0x041, 0x0a0, 0x044, 0x301, 0x000, 0x364, 0x26f, 0x002,
+ 0x021, 0x310, 0x00a, 0x044, 0x290, 0x001, 0x0c1, 0x26f,
+ 0x043, 0x307, 0x020, 0x193, 0x384, 0x007, 0x110, 0x001,
+ 0x00c, 0x364, 0x232, 0x010, 0x009, 0x007, 0x044, 0x044,
+ 0x001, 0x000, 0x3a3, 0x3c4, 0x3fe, 0x26f, 0x3c5, 0x001,
+ 0x200, 0x3c5, 0x010, 0x020, 0x3c4, 0x31f, 0x020, 0x000,
+ 0x395, 0x364, 0x026, 0x002, 0x009, 0x3c5, 0x041, 0x044,
+ 0x003, 0x001, 0x043, 0x307, 0x020, 0x197, 0x384, 0x007,
+ 0x110, 0x001, 0x00f, 0x364, 0x232, 0x020, 0x009, 0x00a,
+ 0x044, 0x051, 0x001, 0x3c5, 0x001, 0x027, 0x000, 0x3ab,
+ 0x3c4, 0x3fc, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x100,
+ 0x020, 0x087, 0x188, 0x208, 0x244, 0x020, 0x000, 0x39b,
+ 0x044, 0x11a, 0x001, 0x307, 0x234, 0x306, 0x235, 0x384,
+ 0x003, 0x021, 0x2e4, 0x002, 0x347, 0x234, 0x235, 0x366,
+ 0x26f, 0x002, 0x021, 0x2e4, 0x002, 0x364, 0x234, 0x003,
+ 0x021, 0x2e4, 0x002, 0x327, 0x21c, 0x324, 0x020, 0x021,
+ 0x385, 0x00a, 0x020, 0x247, 0x002, 0x364, 0x23f, 0x001,
+ 0x021, 0x38e, 0x00a, 0x020, 0x247, 0x002, 0x345, 0x21d,
+ 0x232, 0x020, 0x247, 0x002, 0x2c7, 0x241, 0x307, 0x242,
+ 0x286, 0x001, 0x092, 0x0c1, 0x22a, 0x327, 0x242, 0x326,
+ 0x241, 0x3a4, 0x0ff, 0x001, 0x005, 0x3c4, 0x38f, 0x020,
+ 0x347, 0x241, 0x242, 0x347, 0x241, 0x226, 0x307, 0x21d,
+ 0x304, 0x241, 0x001, 0x00a, 0x0c3, 0x22a, 0x347, 0x241,
+ 0x243, 0x044, 0x39f, 0x002, 0x327, 0x241, 0x3a4, 0x0ff,
+ 0x2c7, 0x225, 0x3c4, 0x300, 0x230, 0x2c5, 0x230, 0x044,
+ 0x099, 0x00b, 0x081, 0x18b, 0x304, 0x226, 0x009, 0x00d,
+ 0x364, 0x232, 0x100, 0x009, 0x008, 0x081, 0x18a, 0x208,
+ 0x044, 0x1b8, 0x000, 0x044, 0x186, 0x001, 0x327, 0x226,
+ 0x003, 0x032, 0x044, 0x2f6, 0x002, 0x001, 0x019, 0x044,
+ 0x237, 0x00a, 0x347, 0x241, 0x226, 0x081, 0x306, 0x22a,
+ 0x384, 0x003, 0x001, 0x005, 0x347, 0x243, 0x226, 0x044,
+ 0x032, 0x009, 0x002, 0x02d, 0x000, 0x031, 0x044, 0x237,
+ 0x00a, 0x009, 0x01c, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x001, 0x023, 0x364, 0x233, 0x001, 0x000,
+ 0x01e, 0x045, 0x044, 0x186, 0x001, 0x3c5, 0x003, 0x22a,
+ 0x347, 0x241, 0x243, 0x000, 0x3c7, 0x3a4, 0x3ff, 0x001,
+ 0x00e, 0x327, 0x21c, 0x324, 0x020, 0x009, 0x006, 0x043,
+ 0x3a7, 0x0ff, 0x041, 0x0c0, 0x270, 0x044, 0x082, 0x00b,
+ 0x3c4, 0x3fc, 0x22a, 0x327, 0x21c, 0x324, 0x020, 0x009,
+ 0x010, 0x307, 0x270, 0x003, 0x005, 0x020, 0x181, 0x006,
+ 0x043, 0x3a7, 0x0ff, 0x041, 0x020, 0x181, 0x006, 0x045,
+ 0x2c7, 0x243, 0x307, 0x250, 0x286, 0x001, 0x3de, 0x0c2,
+ 0x22a, 0x041, 0x044, 0x39f, 0x002, 0x043, 0x2c7, 0x226,
+ 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x0ff, 0x230, 0x1a7,
+ 0x2c5, 0x230, 0x307, 0x21d, 0x304, 0x243, 0x001, 0x013,
+ 0x3c4, 0x0ff, 0x243, 0x347, 0x243, 0x241, 0x347, 0x243,
+ 0x242, 0x3c5, 0x003, 0x22a, 0x044, 0x186, 0x001, 0x000,
+ 0x005, 0x044, 0x191, 0x001, 0x327, 0x243, 0x003, 0x394,
+ 0x000, 0x362, 0x364, 0x232, 0x004, 0x001, 0x008, 0x044,
+ 0x232, 0x000, 0x3c4, 0x3fb, 0x232, 0x364, 0x232, 0x008,
+ 0x001, 0x008, 0x044, 0x326, 0x000, 0x3c4, 0x3f7, 0x232,
+ 0x045, 0x3b7, 0x221, 0x083, 0x189, 0x044, 0x1a9, 0x000,
+ 0x083, 0x020, 0x1b7, 0x000, 0x2c7, 0x24d, 0x364, 0x233,
+ 0x002, 0x001, 0x019, 0x3b9, 0x381, 0x121, 0x081, 0x18a,
+ 0x225, 0x304, 0x24d, 0x001, 0x005, 0x044, 0x1ff, 0x00b,
+ 0x140, 0x24d, 0x00b, 0x008, 0x1a3, 0x044, 0x1ff, 0x00b,
+ 0x000, 0x012, 0x307, 0x220, 0x284, 0x001, 0x00d, 0x3c4,
+ 0x33f, 0x232, 0x1b6, 0x3a4, 0x0c0, 0x2c5, 0x232, 0x020,
+ 0x18a, 0x00b, 0x327, 0x24d, 0x364, 0x020, 0x070, 0x001,
+ 0x00a, 0x399, 0x383, 0x101, 0x326, 0x22b, 0x234, 0x001,
+ 0x02e, 0x3c5, 0x080, 0x233, 0x3d5, 0x200, 0x232, 0x364,
+ 0x026, 0x001, 0x001, 0x00a, 0x3c5, 0x004, 0x232, 0x044,
+ 0x20d, 0x000, 0x307, 0x24d, 0x307, 0x24d, 0x384, 0x007,
+ 0x001, 0x09a, 0x113, 0x001, 0x0a8, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3f8, 0x103, 0x044, 0x1bd, 0x000,
+ 0x3c4, 0x3f7, 0x021, 0x000, 0x0af, 0x0a7, 0x1a7, 0x324,
+ 0x020, 0x001, 0x014, 0x327, 0x24d, 0x326, 0x22b, 0x397,
+ 0x370, 0x364, 0x233, 0x002, 0x001, 0x006, 0x087, 0x18b,
+ 0x385, 0x070, 0x234, 0x001, 0x02b, 0x3c5, 0x100, 0x233,
+ 0x364, 0x026, 0x002, 0x001, 0x008, 0x3c5, 0x008, 0x232,
+ 0x044, 0x3b5, 0x000, 0x307, 0x24d, 0x384, 0x070, 0x001,
+ 0x091, 0x382, 0x030, 0x001, 0x0a6, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x018, 0x044, 0x1bd,
+ 0x000, 0x3c4, 0x3f7, 0x024, 0x000, 0x0a4, 0x347, 0x24d,
+ 0x22b, 0x364, 0x233, 0x006, 0x001, 0x033, 0x307, 0x22b,
+ 0x364, 0x233, 0x002, 0x001, 0x023, 0x364, 0x233, 0x080,
+ 0x001, 0x00a, 0x307, 0x24d, 0x197, 0x384, 0x007, 0x044,
+ 0x3a4, 0x008, 0x364, 0x233, 0x100, 0x001, 0x00c, 0x307,
+ 0x24d, 0x19b, 0x384, 0x007, 0x385, 0x010, 0x044, 0x3a4,
+ 0x008, 0x3c4, 0x27f, 0x233, 0x000, 0x00b, 0x3c5, 0x003,
+ 0x22a, 0x197, 0x044, 0x39c, 0x008, 0x003, 0x298, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x021, 0x181,
+ 0x006, 0x045, 0x3b7, 0x301, 0x087, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7,
+ 0x021, 0x000, 0x36c, 0x3b7, 0x301, 0x044, 0x192, 0x000,
+ 0x384, 0x3f8, 0x105, 0x044, 0x1bd, 0x000, 0x3b7, 0x211,
+ 0x082, 0x044, 0x1a9, 0x000, 0x3c5, 0x008, 0x021, 0x344,
+ 0x21f, 0x021, 0x3b7, 0x250, 0x088, 0x044, 0x1a9, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x347,
+ 0x3b7, 0x301, 0x387, 0x3c7, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x060, 0x083, 0x044, 0x1b7, 0x000, 0x3b7, 0x070, 0x083,
+ 0x18a, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x024, 0x000,
+ 0x36f, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x028, 0x044, 0x1bd, 0x000, 0x3c5, 0x048, 0x024,
+ 0x3a7, 0x060, 0x083, 0x044, 0x1a9, 0x000, 0x3b7, 0x070,
+ 0x083, 0x18a, 0x044, 0x1a9, 0x000, 0x000, 0x351, 0x307,
+ 0x22b, 0x284, 0x2a8, 0x2c4, 0x24d, 0x245, 0x24d, 0x2a8,
+ 0x045, 0x2f4, 0x200, 0x001, 0x008, 0x364, 0x2cf, 0x007,
+ 0x029, 0x1dc, 0x007, 0x287, 0x385, 0x300, 0x208, 0x001,
+ 0x04b, 0x364, 0x33b, 0x010, 0x029, 0x1dc, 0x007, 0x3c5,
+ 0x002, 0x00c, 0x287, 0x384, 0x0ff, 0x305, 0x21d, 0x1b7,
+ 0x2e4, 0x001, 0x001, 0x01d, 0x040, 0x384, 0x007, 0x117,
+ 0x001, 0x00c, 0x3c5, 0x040, 0x02d, 0x081, 0x044, 0x189,
+ 0x000, 0x3c4, 0x3bf, 0x02d, 0x042, 0x247, 0x029, 0x364,
+ 0x018, 0x001, 0x009, 0x005, 0x3c5, 0x010, 0x021, 0x2e4,
+ 0x002, 0x001, 0x017, 0x264, 0x080, 0x001, 0x005, 0x0a4,
+ 0x1a7, 0x285, 0x040, 0x3c5, 0x080, 0x02d, 0x081, 0x044,
+ 0x189, 0x000, 0x3c4, 0x37f, 0x02d, 0x042, 0x247, 0x02a,
+ 0x000, 0x014, 0x38e, 0x3ff, 0x1b8, 0x00a, 0x00a, 0x244,
+ 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4, 0x3fd, 0x00c, 0x120,
+ 0x001, 0x004, 0x244, 0x02a, 0x020, 0x181, 0x006, 0x289,
+ 0x00a, 0x004, 0x2c7, 0x251, 0x22e, 0x364, 0x026, 0x001,
+ 0x021, 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x002, 0x009,
+ 0x01b, 0x3c4, 0x2ff, 0x232, 0x120, 0x001, 0x012, 0x3d5,
+ 0x300, 0x232, 0x044, 0x099, 0x00b, 0x32a, 0x026, 0x00a,
+ 0x008, 0x044, 0x20d, 0x000, 0x044, 0x232, 0x000, 0x020,
+ 0x181, 0x006, 0x042, 0x307, 0x240, 0x20a, 0x040, 0x020,
+ 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x256, 0x22e,
+ 0x364, 0x026, 0x002, 0x021, 0x181, 0x006, 0x041, 0x3a4,
+ 0x01f, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x043, 0x1b7,
+ 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x020, 0x181, 0x006,
+ 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044, 0x340, 0x001,
+ 0x247, 0x249, 0x054, 0x347, 0x249, 0x248, 0x2c7, 0x23f,
+ 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f, 0x001, 0x001,
+ 0x006, 0x245, 0x057, 0x000, 0x005, 0x208, 0x244, 0x057,
+ 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f, 0x002, 0x001,
+ 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001, 0x000, 0x005,
+ 0x208, 0x244, 0x057, 0x020, 0x181, 0x006, 0x2c7, 0x246,
+ 0x3b7, 0x300, 0x044, 0x192, 0x000, 0x384, 0x33f, 0x327,
+ 0x246, 0x3a4, 0x0c0, 0x285, 0x3b7, 0x300, 0x044, 0x1bd,
+ 0x000, 0x3b7, 0x200, 0x044, 0x192, 0x000, 0x0a3, 0x1a9,
+ 0x2a8, 0x284, 0x040, 0x307, 0x246, 0x384, 0x030, 0x264,
+ 0x010, 0x001, 0x012, 0x040, 0x041, 0x3b7, 0x201, 0x044,
+ 0x192, 0x000, 0x384, 0x3e0, 0x385, 0x006, 0x044, 0x1bd,
+ 0x000, 0x043, 0x042, 0x185, 0x2a8, 0x284, 0x227, 0x042,
+ 0x285, 0x3b7, 0x200, 0x044, 0x1bd, 0x000, 0x307, 0x246,
+ 0x384, 0x007, 0x044, 0x375, 0x00b, 0x3a7, 0x073, 0x307,
+ 0x246, 0x184, 0x00a, 0x014, 0x044, 0x192, 0x000, 0x384,
+ 0x3e2, 0x327, 0x246, 0x1b5, 0x3a4, 0x01c, 0x285, 0x3a7,
+ 0x073, 0x044, 0x1bd, 0x000, 0x000, 0x006, 0x081, 0x044,
+ 0x1a9, 0x000, 0x020, 0x181, 0x006, 0x3c5, 0x001, 0x00c,
+ 0x187, 0x0a7, 0x1a7, 0x2a8, 0x324, 0x00c, 0x225, 0x2c7,
+ 0x00c, 0x045, 0x04e, 0x087, 0x011, 0x054, 0x084
+
+}; /* end fm10000_serdes_spico_code_prd2 */
+
+
+
+/* Spico Swap code (production version 2)
+ * source file: serdes.0x2055_0045.swap
+ * Note: if swap code exists, if must be loaded on top of
+ * the sbus master image.
+ */
+
+const uint32_t fm10000_serdes_swap_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_swap_code_size_prd2 = 7842;
+
+const uint16_t fm10000_serdes_swap_code_prd2[] = {
+ 0x01e, 0x09d, 0x002, 0x000, 0x000, 0x008, 0x00f, 0x07b,
+ 0x020, 0x01e, 0x00f, 0x069, 0x01d, 0x029, 0x00b, 0x01d,
+ 0x02c, 0x382, 0x04e, 0x020, 0x23b, 0x009, 0x020, 0x28e,
+ 0x00a, 0x020, 0x03a, 0x00a, 0x020, 0x0fe, 0x00a, 0x020,
+ 0x3f3, 0x008, 0x020, 0x013, 0x009, 0x020, 0x075, 0x00a,
+ 0x0c1, 0x244, 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200,
+ 0x044, 0x172, 0x000, 0x020, 0x0d3, 0x000, 0x020, 0x2c3,
+ 0x008, 0x020, 0x375, 0x00b, 0x0c1, 0x201, 0x045, 0x020,
+ 0x04a, 0x009, 0x020, 0x04e, 0x009, 0x020, 0x256, 0x002,
+ 0x020, 0x17f, 0x009, 0x020, 0x23f, 0x006, 0x020, 0x285,
+ 0x00a, 0x020, 0x209, 0x00b, 0x020, 0x257, 0x006, 0x020,
+ 0x117, 0x008, 0x020, 0x394, 0x00a, 0x020, 0x048, 0x00b,
+ 0x020, 0x1dc, 0x007, 0x020, 0x27f, 0x00a, 0x020, 0x2d2,
+ 0x006, 0x020, 0x1e2, 0x007, 0x020, 0x295, 0x00b, 0x020,
+ 0x3a8, 0x001, 0x020, 0x277, 0x00b, 0x020, 0x2ba, 0x00b,
+ 0x020, 0x2d5, 0x006, 0x020, 0x326, 0x006, 0x020, 0x350,
+ 0x006, 0x020, 0x1dc, 0x007, 0x020, 0x35a, 0x006, 0x020,
+ 0x0a4, 0x00b, 0x020, 0x38e, 0x006, 0x020, 0x011, 0x007,
+ 0x020, 0x016, 0x007, 0x020, 0x31e, 0x007, 0x020, 0x324,
+ 0x007, 0x020, 0x3b6, 0x007, 0x020, 0x02a, 0x007, 0x020,
+ 0x041, 0x007, 0x020, 0x060, 0x007, 0x020, 0x06d, 0x007,
+ 0x020, 0x072, 0x007, 0x020, 0x2d1, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x252, 0x007, 0x020, 0x293, 0x007,
+ 0x020, 0x2d8, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x16f, 0x007, 0x020, 0x185, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x30e, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x077, 0x007, 0x020, 0x0b1, 0x007, 0x020,
+ 0x103, 0x007, 0x020, 0x1c5, 0x008, 0x020, 0x1dc, 0x007,
+ 0x020, 0x168, 0x007, 0x287, 0x264, 0x004, 0x001, 0x049,
+ 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020, 0x33b, 0x18e, 0x00a,
+ 0x013, 0x044, 0x04c, 0x004, 0x3a4, 0x001, 0x001, 0x014,
+ 0x364, 0x052, 0x001, 0x009, 0x093, 0x044, 0x0bc, 0x004,
+ 0x000, 0x08e, 0x3c4, 0x2ff, 0x053, 0x3c4, 0x030, 0x027,
+ 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052, 0x000, 0x081, 0x043,
+ 0x327, 0x33b, 0x020, 0x1de, 0x007, 0x2e4, 0x080, 0x009,
+ 0x00d, 0x3d5, 0x220, 0x33c, 0x100, 0x009, 0x06e, 0x044,
+ 0x3b6, 0x003, 0x000, 0x06c, 0x3d5, 0x120, 0x33c, 0x100,
+ 0x009, 0x022, 0x044, 0x066, 0x004, 0x000, 0x061, 0x384,
+ 0x003, 0x3c4, 0x3df, 0x33b, 0x262, 0x003, 0x001, 0x3d9,
+ 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef, 0x027, 0x2e4, 0x040,
+ 0x009, 0x3d5, 0x100, 0x009, 0x007, 0x044, 0x063, 0x004,
+ 0x000, 0x046, 0x364, 0x200, 0x004, 0x029, 0x1dc, 0x007,
+ 0x0c0, 0x2d8, 0x111, 0x001, 0x028, 0x3c5, 0x002, 0x33c,
+ 0x3c5, 0x001, 0x33b, 0x344, 0x21f, 0x33c, 0x287, 0x19b,
+ 0x247, 0x361, 0x287, 0x183, 0x19b, 0x247, 0x360, 0x0c0,
+ 0x35e, 0x2e4, 0x020, 0x001, 0x004, 0x0c5, 0x35e, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x289, 0x384,
+ 0x010, 0x245, 0x33c, 0x044, 0x001, 0x004, 0x364, 0x33c,
+ 0x002, 0x001, 0x005, 0x3c5, 0x004, 0x200, 0x364, 0x33c,
+ 0x020, 0x009, 0x005, 0x044, 0x097, 0x004, 0x020, 0x181,
+ 0x006, 0x287, 0x19b, 0x11d, 0x001, 0x002, 0x287, 0x1a3,
+ 0x1b3, 0x19b, 0x180, 0x390, 0x1d7, 0x040, 0x199, 0x390,
+ 0x008, 0x040, 0x045, 0x000, 0x021, 0x000, 0x073, 0x000,
+ 0x05f, 0x000, 0x023, 0x000, 0x025, 0x000, 0x033, 0x000,
+ 0x035, 0x000, 0x037, 0x000, 0x041, 0x000, 0x01f, 0x000,
+ 0x021, 0x000, 0x023, 0x000, 0x031, 0x000, 0x082, 0x000,
+ 0x002, 0x020, 0x1dc, 0x007, 0x044, 0x378, 0x003, 0x3c4,
+ 0x3fd, 0x233, 0x000, 0x072, 0x2c7, 0x363, 0x000, 0x06e,
+ 0x2c7, 0x362, 0x000, 0x06a, 0x2c7, 0x341, 0x000, 0x066,
+ 0x2c7, 0x33f, 0x000, 0x062, 0x2c7, 0x340, 0x000, 0x05e,
+ 0x2c7, 0x344, 0x000, 0x05a, 0x2c7, 0x342, 0x000, 0x056,
+ 0x2c7, 0x343, 0x000, 0x052, 0x2c7, 0x341, 0x2c7, 0x33f,
+ 0x2c7, 0x340, 0x000, 0x04a, 0x307, 0x340, 0x300, 0x341,
+ 0x300, 0x33f, 0x282, 0x007, 0x006, 0x2c7, 0x23e, 0x000,
+ 0x03d, 0x043, 0x0a1, 0x020, 0x1de, 0x007, 0x287, 0x384,
+ 0x003, 0x181, 0x3c4, 0x3f3, 0x052, 0x245, 0x052, 0x3a4,
+ 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5, 0x33c, 0x000, 0x026,
+ 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff, 0x352, 0x3c7, 0x3ff,
+ 0x353, 0x1d3, 0x353, 0x0cf, 0x354, 0x1cb, 0x354, 0x3c5,
+ 0x0c0, 0x354, 0x3c9, 0x222, 0x355, 0x3c5, 0x001, 0x33c,
+ 0x0c0, 0x356, 0x1b0, 0x00a, 0x006, 0x081, 0x18a, 0x247,
+ 0x356, 0x1ab, 0x2c7, 0x357, 0x020, 0x181, 0x006, 0x287,
+ 0x001, 0x025, 0x197, 0x3a4, 0x00f, 0x262, 0x003, 0x001,
+ 0x03d, 0x131, 0x001, 0x020, 0x131, 0x001, 0x021, 0x131,
+ 0x001, 0x022, 0x131, 0x001, 0x025, 0x131, 0x001, 0x026,
+ 0x131, 0x001, 0x027, 0x132, 0x182, 0x390, 0x377, 0x280,
+ 0x04c, 0x042, 0x020, 0x1de, 0x007, 0x044, 0x361, 0x003,
+ 0x000, 0x3d4, 0x327, 0x373, 0x000, 0x3f5, 0x327, 0x372,
+ 0x000, 0x3f1, 0x327, 0x2b3, 0x322, 0x2b2, 0x000, 0x3eb,
+ 0x327, 0x374, 0x000, 0x3e7, 0x327, 0x375, 0x000, 0x3e3,
+ 0x327, 0x376, 0x000, 0x3df, 0x307, 0x369, 0x282, 0x272,
+ 0x369, 0x00e, 0x003, 0x108, 0x04c, 0x000, 0x3d4, 0x364,
+ 0x33c, 0x001, 0x001, 0x01b, 0x364, 0x33b, 0x020, 0x009,
+ 0x016, 0x0a0, 0x307, 0x04f, 0x181, 0x00a, 0x003, 0x0a4,
+ 0x100, 0x00b, 0x004, 0x3a5, 0x080, 0x307, 0x050, 0x194,
+ 0x384, 0x040, 0x225, 0x2c7, 0x392, 0x081, 0x189, 0x314,
+ 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x027, 0x364, 0x33c, 0x020, 0x001, 0x007,
+ 0x347, 0x21d, 0x35a, 0x000, 0x018, 0x347, 0x050, 0x368,
+ 0x3b7, 0x201, 0x1a5, 0x140, 0x04f, 0x00b, 0x00b, 0x364,
+ 0x052, 0x020, 0x001, 0x3f3, 0x2c5, 0x33c, 0x000, 0x005,
+ 0x2a8, 0x2c4, 0x33c, 0x081, 0x189, 0x304, 0x053, 0x001,
+ 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7,
+ 0x027, 0x3c5, 0x080, 0x053, 0x3c4, 0x37f, 0x053, 0x364,
+ 0x33b, 0x020, 0x009, 0x00a, 0x364, 0x33c, 0x020, 0x009,
+ 0x005, 0x044, 0x132, 0x004, 0x020, 0x08d, 0x006, 0x2c7,
+ 0x231, 0x2e4, 0x100, 0x001, 0x045, 0x2e4, 0x001, 0x001,
+ 0x020, 0x3b7, 0x213, 0x084, 0x044, 0x1a9, 0x000, 0x3b7,
+ 0x250, 0x084, 0x044, 0x1a9, 0x000, 0x399, 0x305, 0x3a7,
+ 0x022, 0x044, 0x1a9, 0x000, 0x044, 0x322, 0x001, 0x3c5,
+ 0x002, 0x024, 0x3c5, 0x008, 0x292, 0x000, 0x020, 0x3c4,
+ 0x3fd, 0x024, 0x364, 0x234, 0x002, 0x001, 0x009, 0x3a7,
+ 0x022, 0x399, 0x305, 0x044, 0x1b7, 0x000, 0x3b7, 0x213,
+ 0x084, 0x044, 0x1b7, 0x000, 0x3b7, 0x250, 0x084, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x007, 0x292, 0x044, 0x06e, 0x003,
+ 0x307, 0x231, 0x274, 0x200, 0x001, 0x01b, 0x274, 0x010,
+ 0x001, 0x00b, 0x387, 0x020, 0x245, 0x021, 0x185, 0x245,
+ 0x024, 0x000, 0x00b, 0x387, 0x3df, 0x244, 0x021, 0x185,
+ 0x385, 0x0ff, 0x244, 0x024, 0x044, 0x03a, 0x00a, 0x045,
+ 0x327, 0x230, 0x364, 0x233, 0x004, 0x001, 0x003, 0x045,
+ 0x0c2, 0x22a, 0x347, 0x275, 0x270, 0x0c0, 0x27b, 0x264,
+ 0x010, 0x009, 0x00a, 0x0c1, 0x22a, 0x347, 0x274, 0x270,
+ 0x3d7, 0x200, 0x27b, 0x327, 0x230, 0x264, 0x004, 0x001,
+ 0x012, 0x347, 0x277, 0x270, 0x264, 0x010, 0x009, 0x007,
+ 0x347, 0x276, 0x270, 0x384, 0x00f, 0x112, 0x1b7, 0x000,
+ 0x004, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x384, 0x00f, 0x262,
+ 0x003, 0x001, 0x00b, 0x101, 0x3c2, 0x100, 0x270, 0x34a,
+ 0x225, 0x225, 0x000, 0x3f3, 0x044, 0x237, 0x00a, 0x044,
+ 0x35f, 0x009, 0x364, 0x22a, 0x002, 0x001, 0x006, 0x347,
+ 0x270, 0x272, 0x045, 0x347, 0x270, 0x271, 0x045, 0x044,
+ 0x2f6, 0x002, 0x009, 0x015, 0x307, 0x241, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x3d7, 0x200, 0x27b, 0x044, 0x35f, 0x009, 0x347,
+ 0x241, 0x226, 0x0c1, 0x22a, 0x020, 0x032, 0x009, 0x044,
+ 0x2f6, 0x002, 0x009, 0x014, 0x307, 0x243, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x0c0, 0x27b, 0x044, 0x35f, 0x009, 0x347, 0x243,
+ 0x226, 0x0c2, 0x22a, 0x020, 0x032, 0x009, 0x327, 0x226,
+ 0x2c7, 0x225, 0x3c4, 0x0ff, 0x225, 0x1b7, 0x3a4, 0x007,
+ 0x001, 0x00d, 0x34a, 0x225, 0x225, 0x002, 0x008, 0x131,
+ 0x009, 0x3fa, 0x044, 0x237, 0x00a, 0x045, 0x0c1, 0x26f,
+ 0x000, 0x050, 0x327, 0x01d, 0x2e4, 0x380, 0x009, 0x00a,
+ 0x13a, 0x00b, 0x00a, 0x3c5, 0x008, 0x233, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x233, 0x327, 0x01d, 0x021, 0x2ed, 0x009,
+ 0x364, 0x020, 0x004, 0x009, 0x01a, 0x309, 0x232, 0x00b,
+ 0x009, 0x347, 0x270, 0x271, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x271, 0x270, 0x044, 0x270, 0x009, 0x001, 0x04c,
+ 0x347, 0x270, 0x271, 0x000, 0x018, 0x309, 0x232, 0x001,
+ 0x009, 0x347, 0x270, 0x272, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x272, 0x270, 0x044, 0x270, 0x009, 0x001, 0x089,
+ 0x347, 0x270, 0x272, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x364, 0x26f, 0x002, 0x009, 0x005, 0x3c4, 0x3fe, 0x200,
+ 0x364, 0x020, 0x005, 0x029, 0x2e4, 0x002, 0x364, 0x020,
+ 0x010, 0x009, 0x0e9, 0x364, 0x020, 0x100, 0x009, 0x12b,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3f7, 0x233, 0x364, 0x020,
+ 0x001, 0x009, 0x00c, 0x364, 0x020, 0x004, 0x009, 0x05d,
+ 0x000, 0x3d8, 0x3c5, 0x080, 0x020, 0x364, 0x233, 0x006,
+ 0x001, 0x031, 0x081, 0x189, 0x304, 0x232, 0x009, 0x017,
+ 0x347, 0x271, 0x274, 0x080, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x085, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x081,
+ 0x189, 0x245, 0x232, 0x000, 0x0c8, 0x347, 0x271, 0x276,
+ 0x081, 0x189, 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197,
+ 0x384, 0x007, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x000,
+ 0x005, 0x044, 0x3f3, 0x008, 0x3b7, 0x200, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x203, 0x387, 0x3f7, 0x044,
+ 0x1b8, 0x000, 0x307, 0x020, 0x385, 0x042, 0x384, 0x3ce,
+ 0x247, 0x020, 0x044, 0x044, 0x001, 0x000, 0x383, 0x0a1,
+ 0x1aa, 0x2c5, 0x020, 0x364, 0x233, 0x006, 0x001, 0x039,
+ 0x081, 0x189, 0x304, 0x232, 0x009, 0x017, 0x347, 0x272,
+ 0x275, 0x080, 0x364, 0x233, 0x004, 0x009, 0x003, 0x095,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245,
+ 0x232, 0x000, 0x0ce, 0x347, 0x272, 0x277, 0x081, 0x189,
+ 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197, 0x364, 0x233,
+ 0x002, 0x001, 0x007, 0x193, 0x384, 0x007, 0x385, 0x010,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x000, 0x008, 0x347,
+ 0x272, 0x270, 0x044, 0x013, 0x009, 0x081, 0x18a, 0x208,
+ 0x244, 0x232, 0x0a3, 0x387, 0x3f7, 0x044, 0x1b8, 0x000,
+ 0x307, 0x020, 0x0a4, 0x1a7, 0x3a5, 0x008, 0x285, 0x3b7,
+ 0x304, 0x2a8, 0x284, 0x247, 0x020, 0x0a0, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x044, 0x2f6, 0x002, 0x009, 0x00f,
+ 0x364, 0x2cf, 0x080, 0x009, 0x00a, 0x0c3, 0x26f, 0x044,
+ 0x1b6, 0x001, 0x020, 0x2e4, 0x002, 0x044, 0x051, 0x001,
+ 0x000, 0x308, 0x3c4, 0x3f7, 0x233, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x083, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x3d7, 0x200, 0x27b, 0x0c0, 0x271,
+ 0x3b7, 0x300, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x221, 0x091, 0x189, 0x044, 0x1a9, 0x000, 0x3b7, 0x201,
+ 0x387, 0x0a6, 0x044, 0x1bd, 0x000, 0x3c5, 0x021, 0x020,
+ 0x3c4, 0x3ed, 0x020, 0x3b7, 0x200, 0x387, 0x020, 0x044,
+ 0x020, 0x00a, 0x009, 0x32a, 0x347, 0x271, 0x270, 0x000,
+ 0x055, 0x3a7, 0x071, 0x307, 0x21d, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x233, 0x044, 0x2f6, 0x002, 0x009, 0x007,
+ 0x0c1, 0x2d3, 0x3c5, 0x100, 0x200, 0x081, 0x18a, 0x245,
+ 0x232, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x093, 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x0c0,
+ 0x27b, 0x0c0, 0x272, 0x3b7, 0x300, 0x387, 0x020, 0x044,
+ 0x1a9, 0x000, 0x0a1, 0x397, 0x205, 0x247, 0x27f, 0x044,
+ 0x1bd, 0x000, 0x3d5, 0x204, 0x020, 0x3c4, 0x2ff, 0x020,
+ 0x0a0, 0x387, 0x021, 0x184, 0x044, 0x020, 0x00a, 0x009,
+ 0x336, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3fe, 0x200, 0x3c4, 0x3fe, 0x26f, 0x344, 0x21f, 0x232,
+ 0x0c0, 0x020, 0x0a3, 0x1ab, 0x2c5, 0x020, 0x044, 0x20d,
+ 0x000, 0x044, 0x2f3, 0x000, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x0a9, 0x1ab, 0x09f, 0x18a, 0x208, 0x304,
+ 0x055, 0x225, 0x044, 0x394, 0x00a, 0x307, 0x055, 0x19b,
+ 0x384, 0x003, 0x227, 0x1a3, 0x225, 0x044, 0x0a4, 0x00b,
+ 0x0a7, 0x020, 0x022, 0x008, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x006, 0x387, 0x07f, 0x000, 0x013, 0x399, 0x240,
+ 0x385, 0x07f, 0x306, 0x270, 0x001, 0x06c, 0x399, 0x240,
+ 0x327, 0x270, 0x3a4, 0x380, 0x226, 0x001, 0x015, 0x327,
+ 0x220, 0x324, 0x232, 0x009, 0x019, 0x364, 0x232, 0x001,
+ 0x001, 0x02b, 0x327, 0x01d, 0x307, 0x01b, 0x190, 0x282,
+ 0x003, 0x041, 0x307, 0x01b, 0x191, 0x380, 0x010, 0x282,
+ 0x003, 0x034, 0x000, 0x019, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x00a, 0x327, 0x270, 0x384, 0x07f, 0x3a4, 0x07f,
+ 0x000, 0x008, 0x327, 0x270, 0x3a4, 0x07f, 0x001, 0x005,
+ 0x286, 0x001, 0x02f, 0x307, 0x270, 0x384, 0x07f, 0x386,
+ 0x07f, 0x009, 0x00a, 0x387, 0x080, 0x382, 0x07f, 0x240,
+ 0x270, 0x000, 0x01c, 0x141, 0x270, 0x364, 0x270, 0x07f,
+ 0x009, 0x015, 0x000, 0x013, 0x3c0, 0x040, 0x270, 0x000,
+ 0x00e, 0x307, 0x270, 0x196, 0x384, 0x00f, 0x266, 0x009,
+ 0x001, 0x008, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009,
+ 0x045, 0x309, 0x232, 0x00b, 0x013, 0x347, 0x271, 0x270,
+ 0x374, 0x27b, 0x200, 0x009, 0x005, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x020, 0x0b7, 0x009, 0x327, 0x232,
+ 0x1a2, 0x023, 0x0b7, 0x009, 0x307, 0x270, 0x196, 0x119,
+ 0x021, 0x0b7, 0x009, 0x307, 0x232, 0x181, 0x023, 0x0b7,
+ 0x009, 0x364, 0x270, 0x080, 0x029, 0x0b7, 0x009, 0x083,
+ 0x18c, 0x245, 0x232, 0x3c4, 0x3f7, 0x233, 0x3c4, 0x380,
+ 0x270, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x09c, 0x009, 0x000,
+ 0x000, 0x000, 0x002, 0x000, 0x00a, 0x000, 0x02a, 0x000,
+ 0x0aa, 0x002, 0x0aa, 0x00a, 0x0aa, 0x02a, 0x0aa, 0x06a,
+ 0x0aa, 0x07a, 0x0aa, 0x07e, 0x0aa, 0x07f, 0x0aa, 0x07f,
+ 0x0ba, 0x07f, 0x0be, 0x07f, 0x0fe, 0x07f, 0x0ff, 0x007,
+ 0x000, 0x017, 0x001, 0x027, 0x007, 0x037, 0x00f, 0x047,
+ 0x03f, 0x080, 0x0ff, 0x364, 0x270, 0x07f, 0x307, 0x270,
+ 0x384, 0x07f, 0x009, 0x00d, 0x089, 0x186, 0x306, 0x270,
+ 0x001, 0x007, 0x3c5, 0x001, 0x232, 0x000, 0x005, 0x3c4,
+ 0x3fe, 0x232, 0x347, 0x270, 0x27c, 0x3c4, 0x007, 0x27c,
+ 0x374, 0x27b, 0x200, 0x001, 0x00a, 0x364, 0x233, 0x008,
+ 0x001, 0x005, 0x044, 0x0fe, 0x00a, 0x0a3, 0x325, 0x27b,
+ 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x305, 0x27c, 0x044,
+ 0x1bd, 0x000, 0x307, 0x270, 0x192, 0x384, 0x00f, 0x3b9,
+ 0x200, 0x324, 0x233, 0x001, 0x014, 0x229, 0x387, 0x009,
+ 0x189, 0x395, 0x333, 0x280, 0x051, 0x2c7, 0x27d, 0x1c7,
+ 0x27d, 0x101, 0x051, 0x2c5, 0x27d, 0x000, 0x011, 0x0c0,
+ 0x27d, 0x384, 0x00f, 0x001, 0x00b, 0x349, 0x27d, 0x27d,
+ 0x3c5, 0x001, 0x27d, 0x111, 0x000, 0x3f7, 0x0a2, 0x325,
+ 0x27b, 0x307, 0x27d, 0x044, 0x1bd, 0x000, 0x374, 0x27b,
+ 0x200, 0x001, 0x02c, 0x347, 0x270, 0x27a, 0x3c4, 0x07f,
+ 0x27a, 0x0c0, 0x279, 0x387, 0x009, 0x189, 0x395, 0x353,
+ 0x051, 0x041, 0x101, 0x051, 0x2c7, 0x279, 0x101, 0x043,
+ 0x322, 0x27a, 0x00b, 0x004, 0x000, 0x3f4, 0x1c7, 0x279,
+ 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x0ff,
+ 0x305, 0x279, 0x044, 0x1bd, 0x000, 0x327, 0x220, 0x324,
+ 0x232, 0x009, 0x022, 0x0c0, 0x27e, 0x307, 0x270, 0x197,
+ 0x00a, 0x004, 0x000, 0x005, 0x3c5, 0x018, 0x27e, 0x0a4,
+ 0x222, 0x2c5, 0x27e, 0x0a4, 0x325, 0x27b, 0x044, 0x192,
+ 0x000, 0x384, 0x3e0, 0x3c4, 0x01f, 0x27e, 0x305, 0x27e,
+ 0x044, 0x1bd, 0x000, 0x045, 0x044, 0x1a9, 0x000, 0x083,
+ 0x18c, 0x208, 0x244, 0x232, 0x3c5, 0x001, 0x00c, 0x0a3,
+ 0x325, 0x27b, 0x088, 0x044, 0x1a9, 0x000, 0x387, 0x3f8,
+ 0x044, 0x1b8, 0x000, 0x020, 0x2f6, 0x002, 0x3c7, 0x020,
+ 0x280, 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x247, 0x247,
+ 0x151, 0x280, 0x001, 0x016, 0x0a1, 0x044, 0x182, 0x000,
+ 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x306, 0x247, 0x001,
+ 0x3f1, 0x0a2, 0x364, 0x22b, 0x003, 0x001, 0x003, 0x0a5,
+ 0x020, 0x182, 0x000, 0x1f6, 0x185, 0x1f5, 0x129, 0x1f4,
+ 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff, 0x1f6, 0x189, 0x1f5,
+ 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce, 0x123, 0x0f3, 0x0d2,
+ 0x0ff, 0x364, 0x233, 0x020, 0x009, 0x051, 0x307, 0x272,
+ 0x196, 0x384, 0x00f, 0x0c9, 0x286, 0x242, 0x286, 0x397,
+ 0x00a, 0x189, 0x395, 0x05f, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x00a, 0x051, 0x2c7, 0x284, 0x101, 0x051,
+ 0x2c7, 0x288, 0x3a4, 0x07f, 0x040, 0x307, 0x272, 0x384,
+ 0x07f, 0x222, 0x042, 0x00b, 0x005, 0x101, 0x000, 0x3ed,
+ 0x347, 0x284, 0x285, 0x3c4, 0x00f, 0x284, 0x1d3, 0x285,
+ 0x1d6, 0x288, 0x342, 0x288, 0x286, 0x003, 0x008, 0x156,
+ 0x285, 0x151, 0x286, 0x00b, 0x3fc, 0x362, 0x285, 0x003,
+ 0x003, 0x004, 0x0c3, 0x285, 0x307, 0x285, 0x0a1, 0x184,
+ 0x305, 0x284, 0x044, 0x1bd, 0x000, 0x045, 0x3db, 0x071,
+ 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c, 0x176, 0x1bf, 0x00d,
+ 0x007, 0x006, 0x003, 0x002, 0x005, 0x0b3, 0x156, 0x1fa,
+ 0x27f, 0x080, 0x09e, 0x14b, 0x1f0, 0x27f, 0x080, 0x08b,
+ 0x140, 0x1e9, 0x27f, 0x081, 0x138, 0x1e3, 0x27f, 0x00f,
+ 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080, 0x0a3, 0x145, 0x1e7,
+ 0x27f, 0x080, 0x095, 0x13d, 0x1e0, 0x27f, 0x08b, 0x136,
+ 0x1db, 0x27f, 0x0c5, 0x28b, 0x347, 0x271, 0x281, 0x3c4,
+ 0x07f, 0x281, 0x0c5, 0x282, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x0c2, 0x282, 0x044, 0x1bc, 0x00a, 0x347, 0x283,
+ 0x289, 0x307, 0x241, 0x364, 0x233, 0x002, 0x001, 0x004,
+ 0x307, 0x225, 0x384, 0x0ff, 0x242, 0x283, 0x00b, 0x056,
+ 0x0c3, 0x28b, 0x302, 0x289, 0x397, 0x00a, 0x189, 0x395,
+ 0x0ca, 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x004,
+ 0x051, 0x2c7, 0x28c, 0x2a9, 0x121, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003, 0x007,
+ 0x001, 0x005, 0x101, 0x000, 0x3ed, 0x1d5, 0x28c, 0x347,
+ 0x241, 0x281, 0x364, 0x233, 0x002, 0x001, 0x005, 0x347,
+ 0x225, 0x281, 0x3c4, 0x0ff, 0x281, 0x342, 0x289, 0x281,
+ 0x347, 0x28c, 0x282, 0x366, 0x282, 0x00f, 0x009, 0x006,
+ 0x1c0, 0x281, 0x0c1, 0x282, 0x044, 0x1bc, 0x00a, 0x001,
+ 0x00d, 0x141, 0x28b, 0x362, 0x28b, 0x03f, 0x001, 0x00d,
+ 0x151, 0x283, 0x000, 0x3f5, 0x362, 0x28b, 0x00a, 0x003,
+ 0x004, 0x0ca, 0x28b, 0x397, 0x00a, 0x189, 0x395, 0x0d8,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x013, 0x327,
+ 0x24c, 0x131, 0x003, 0x005, 0x105, 0x000, 0x3fc, 0x051,
+ 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121, 0x040, 0x307, 0x271,
+ 0x384, 0x07f, 0x282, 0x042, 0x003, 0x005, 0x101, 0x000,
+ 0x3f0, 0x1d6, 0x28a, 0x387, 0x00a, 0x189, 0x395, 0x0d3,
+ 0x300, 0x28a, 0x050, 0x247, 0x28a, 0x30a, 0x28b, 0x3b7,
+ 0x201, 0x184, 0x305, 0x28a, 0x044, 0x1bd, 0x000, 0x045,
+ 0x0c1, 0x283, 0x342, 0x282, 0x281, 0x003, 0x006, 0x141,
+ 0x283, 0x000, 0x3f9, 0x045, 0x000, 0x301, 0x103, 0x307,
+ 0x006, 0x10e, 0x21e, 0x33e, 0x13c, 0x038, 0x078, 0x1f8,
+ 0x0f0, 0x000, 0x001, 0x003, 0x002, 0x006, 0x187, 0x227,
+ 0x387, 0x00a, 0x189, 0x395, 0x1c8, 0x300, 0x227, 0x111,
+ 0x050, 0x247, 0x24c, 0x1d7, 0x24c, 0x384, 0x0ff, 0x285,
+ 0x044, 0x226, 0x00a, 0x364, 0x22a, 0x001, 0x001, 0x00e,
+ 0x3d5, 0x200, 0x232, 0x040, 0x1aa, 0x285, 0x3b7, 0x220,
+ 0x044, 0x1bd, 0x000, 0x042, 0x364, 0x22a, 0x002, 0x001,
+ 0x022, 0x040, 0x3a7, 0x073, 0x044, 0x192, 0x000, 0x384,
+ 0x31f, 0x044, 0x226, 0x00a, 0x1a4, 0x285, 0x3a7, 0x073,
+ 0x044, 0x1bd, 0x000, 0x3a7, 0x070, 0x044, 0x192, 0x000,
+ 0x19a, 0x18a, 0x043, 0x285, 0x3a7, 0x070, 0x044, 0x1bd,
+ 0x000, 0x045, 0x040, 0x387, 0x00a, 0x189, 0x395, 0x1d5,
+ 0x300, 0x278, 0x051, 0x042, 0x045, 0x00a, 0x058, 0x026,
+ 0x015, 0x044, 0x032, 0x0c0, 0x278, 0x0a0, 0x397, 0x00a,
+ 0x189, 0x395, 0x231, 0x280, 0x050, 0x247, 0x228, 0x3c4,
+ 0x00f, 0x228, 0x347, 0x228, 0x229, 0x040, 0x307, 0x278,
+ 0x001, 0x008, 0x340, 0x229, 0x228, 0x111, 0x000, 0x3fa,
+ 0x042, 0x193, 0x0c0, 0x227, 0x347, 0x225, 0x226, 0x141,
+ 0x227, 0x366, 0x227, 0x00e, 0x001, 0x00b, 0x342, 0x228,
+ 0x226, 0x007, 0x3f6, 0x140, 0x226, 0x001, 0x011, 0x121,
+ 0x2e2, 0x006, 0x009, 0x3cc, 0x141, 0x278, 0x366, 0x278,
+ 0x005, 0x009, 0x3c4, 0x0a1, 0x000, 0x006, 0x044, 0x1da,
+ 0x00a, 0x0a0, 0x045, 0x044, 0x333, 0x008, 0x020, 0x181,
+ 0x006, 0x345, 0x21d, 0x232, 0x364, 0x23f, 0x001, 0x029,
+ 0x2e4, 0x002, 0x2c7, 0x234, 0x399, 0x200, 0x245, 0x233,
+ 0x364, 0x234, 0x020, 0x009, 0x005, 0x208, 0x244, 0x233,
+ 0x083, 0x18c, 0x208, 0x244, 0x036, 0x2e4, 0x008, 0x001,
+ 0x007, 0x345, 0x220, 0x232, 0x000, 0x007, 0x081, 0x18b,
+ 0x208, 0x244, 0x232, 0x307, 0x235, 0x226, 0x3a4, 0x001,
+ 0x001, 0x010, 0x327, 0x234, 0x2e4, 0x001, 0x009, 0x041,
+ 0x041, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x043,
+ 0x0a3, 0x1ad, 0x3a5, 0x039, 0x326, 0x236, 0x001, 0x02f,
+ 0x3a7, 0x0bf, 0x1a7, 0x3a5, 0x039, 0x326, 0x236, 0x001,
+ 0x026, 0x327, 0x234, 0x307, 0x235, 0x286, 0x384, 0x002,
+ 0x001, 0x01d, 0x2e4, 0x002, 0x009, 0x051, 0x041, 0x0a1,
+ 0x1ad, 0x2c5, 0x024, 0x307, 0x024, 0x19a, 0x002, 0x005,
+ 0x2a8, 0x2c4, 0x024, 0x044, 0x3b5, 0x000, 0x044, 0x0f2,
+ 0x001, 0x044, 0x3f0, 0x000, 0x043, 0x000, 0x067, 0x364,
+ 0x026, 0x001, 0x009, 0x3c6, 0x041, 0x0a0, 0x044, 0x301,
+ 0x000, 0x364, 0x26f, 0x002, 0x021, 0x310, 0x00a, 0x044,
+ 0x290, 0x001, 0x0c1, 0x26f, 0x043, 0x307, 0x020, 0x193,
+ 0x384, 0x007, 0x110, 0x001, 0x00c, 0x364, 0x232, 0x010,
+ 0x009, 0x007, 0x044, 0x044, 0x001, 0x000, 0x3a3, 0x3c4,
+ 0x3fe, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x010, 0x020,
+ 0x3c4, 0x31f, 0x020, 0x000, 0x395, 0x364, 0x026, 0x002,
+ 0x009, 0x3c5, 0x041, 0x044, 0x003, 0x001, 0x043, 0x307,
+ 0x020, 0x197, 0x384, 0x007, 0x110, 0x001, 0x00f, 0x364,
+ 0x232, 0x020, 0x009, 0x00a, 0x044, 0x051, 0x001, 0x3c5,
+ 0x001, 0x027, 0x000, 0x3ab, 0x3c4, 0x3fc, 0x26f, 0x3c5,
+ 0x001, 0x200, 0x3c5, 0x100, 0x020, 0x087, 0x188, 0x208,
+ 0x244, 0x020, 0x000, 0x39b, 0x044, 0x11a, 0x001, 0x307,
+ 0x234, 0x306, 0x235, 0x384, 0x003, 0x021, 0x2e4, 0x002,
+ 0x347, 0x234, 0x235, 0x366, 0x26f, 0x002, 0x021, 0x2e4,
+ 0x002, 0x364, 0x234, 0x003, 0x021, 0x2e4, 0x002, 0x327,
+ 0x21c, 0x324, 0x020, 0x021, 0x385, 0x00a, 0x020, 0x247,
+ 0x002, 0x364, 0x23f, 0x001, 0x021, 0x38e, 0x00a, 0x020,
+ 0x247, 0x002, 0x345, 0x21d, 0x232, 0x020, 0x247, 0x002,
+ 0x2c7, 0x241, 0x307, 0x242, 0x286, 0x001, 0x092, 0x0c1,
+ 0x22a, 0x327, 0x242, 0x326, 0x241, 0x3a4, 0x0ff, 0x001,
+ 0x005, 0x3c4, 0x38f, 0x020, 0x347, 0x241, 0x242, 0x347,
+ 0x241, 0x226, 0x307, 0x21d, 0x304, 0x241, 0x001, 0x00a,
+ 0x0c3, 0x22a, 0x347, 0x241, 0x243, 0x044, 0x39f, 0x002,
+ 0x327, 0x241, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x300,
+ 0x230, 0x2c5, 0x230, 0x044, 0x099, 0x00b, 0x081, 0x18b,
+ 0x304, 0x226, 0x009, 0x00d, 0x364, 0x232, 0x100, 0x009,
+ 0x008, 0x081, 0x18a, 0x208, 0x044, 0x1b8, 0x000, 0x044,
+ 0x186, 0x001, 0x327, 0x226, 0x003, 0x032, 0x044, 0x2f6,
+ 0x002, 0x001, 0x019, 0x044, 0x237, 0x00a, 0x347, 0x241,
+ 0x226, 0x081, 0x306, 0x22a, 0x384, 0x003, 0x001, 0x005,
+ 0x347, 0x243, 0x226, 0x044, 0x032, 0x009, 0x002, 0x02d,
+ 0x000, 0x031, 0x044, 0x237, 0x00a, 0x009, 0x01c, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x001, 0x023,
+ 0x364, 0x233, 0x001, 0x000, 0x01e, 0x045, 0x044, 0x186,
+ 0x001, 0x3c5, 0x003, 0x22a, 0x347, 0x241, 0x243, 0x000,
+ 0x3c7, 0x3a4, 0x3ff, 0x001, 0x00e, 0x327, 0x21c, 0x324,
+ 0x020, 0x009, 0x006, 0x043, 0x3a7, 0x0ff, 0x041, 0x0c0,
+ 0x270, 0x044, 0x082, 0x00b, 0x3c4, 0x3fc, 0x22a, 0x327,
+ 0x21c, 0x324, 0x020, 0x009, 0x010, 0x307, 0x270, 0x003,
+ 0x005, 0x020, 0x181, 0x006, 0x043, 0x3a7, 0x0ff, 0x041,
+ 0x020, 0x181, 0x006, 0x045, 0x2c7, 0x243, 0x307, 0x250,
+ 0x286, 0x001, 0x3de, 0x0c2, 0x22a, 0x041, 0x044, 0x39f,
+ 0x002, 0x043, 0x2c7, 0x226, 0x3a4, 0x0ff, 0x2c7, 0x225,
+ 0x3c4, 0x0ff, 0x230, 0x1a7, 0x2c5, 0x230, 0x307, 0x21d,
+ 0x304, 0x243, 0x001, 0x013, 0x3c4, 0x0ff, 0x243, 0x347,
+ 0x243, 0x241, 0x347, 0x243, 0x242, 0x3c5, 0x003, 0x22a,
+ 0x044, 0x186, 0x001, 0x000, 0x005, 0x044, 0x191, 0x001,
+ 0x327, 0x243, 0x003, 0x394, 0x000, 0x362, 0x364, 0x232,
+ 0x004, 0x001, 0x008, 0x044, 0x232, 0x000, 0x3c4, 0x3fb,
+ 0x232, 0x364, 0x232, 0x008, 0x001, 0x008, 0x044, 0x326,
+ 0x000, 0x3c4, 0x3f7, 0x232, 0x045, 0x3b7, 0x221, 0x083,
+ 0x189, 0x044, 0x1a9, 0x000, 0x083, 0x020, 0x1b7, 0x000,
+ 0x2c7, 0x24d, 0x364, 0x233, 0x002, 0x001, 0x019, 0x3b9,
+ 0x381, 0x121, 0x081, 0x18a, 0x225, 0x304, 0x24d, 0x001,
+ 0x005, 0x044, 0x1ff, 0x00b, 0x140, 0x24d, 0x00b, 0x008,
+ 0x1a3, 0x044, 0x1ff, 0x00b, 0x000, 0x012, 0x307, 0x220,
+ 0x284, 0x001, 0x00d, 0x3c4, 0x33f, 0x232, 0x1b6, 0x3a4,
+ 0x0c0, 0x2c5, 0x232, 0x020, 0x18a, 0x00b, 0x327, 0x24d,
+ 0x364, 0x020, 0x070, 0x001, 0x00a, 0x399, 0x383, 0x101,
+ 0x326, 0x22b, 0x234, 0x001, 0x02e, 0x3c5, 0x080, 0x233,
+ 0x3d5, 0x200, 0x232, 0x364, 0x026, 0x001, 0x001, 0x00a,
+ 0x3c5, 0x004, 0x232, 0x044, 0x20d, 0x000, 0x307, 0x24d,
+ 0x307, 0x24d, 0x384, 0x007, 0x001, 0x09a, 0x113, 0x001,
+ 0x0a8, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8,
+ 0x103, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x021, 0x000,
+ 0x0af, 0x0a7, 0x1a7, 0x324, 0x020, 0x001, 0x014, 0x327,
+ 0x24d, 0x326, 0x22b, 0x397, 0x370, 0x364, 0x233, 0x002,
+ 0x001, 0x006, 0x087, 0x18b, 0x385, 0x070, 0x234, 0x001,
+ 0x02b, 0x3c5, 0x100, 0x233, 0x364, 0x026, 0x002, 0x001,
+ 0x008, 0x3c5, 0x008, 0x232, 0x044, 0x3b5, 0x000, 0x307,
+ 0x24d, 0x384, 0x070, 0x001, 0x091, 0x382, 0x030, 0x001,
+ 0x0a6, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x018, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x024,
+ 0x000, 0x0a4, 0x347, 0x24d, 0x22b, 0x364, 0x233, 0x006,
+ 0x001, 0x033, 0x307, 0x22b, 0x364, 0x233, 0x002, 0x001,
+ 0x023, 0x364, 0x233, 0x080, 0x001, 0x00a, 0x307, 0x24d,
+ 0x197, 0x384, 0x007, 0x044, 0x3a4, 0x008, 0x364, 0x233,
+ 0x100, 0x001, 0x00c, 0x307, 0x24d, 0x19b, 0x384, 0x007,
+ 0x385, 0x010, 0x044, 0x3a4, 0x008, 0x3c4, 0x27f, 0x233,
+ 0x000, 0x00b, 0x3c5, 0x003, 0x22a, 0x197, 0x044, 0x39c,
+ 0x008, 0x003, 0x298, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x021, 0x181, 0x006, 0x045, 0x3b7, 0x301,
+ 0x087, 0x044, 0x1b7, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x3f7, 0x021, 0x000, 0x36c, 0x3b7,
+ 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x105, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000,
+ 0x3c5, 0x008, 0x021, 0x344, 0x21f, 0x021, 0x3b7, 0x250,
+ 0x088, 0x044, 0x1a9, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1a9, 0x000, 0x000, 0x347, 0x3b7, 0x301, 0x387, 0x3c7,
+ 0x044, 0x1b8, 0x000, 0x3a7, 0x060, 0x083, 0x044, 0x1b7,
+ 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x024, 0x000, 0x36f, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x028, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x048, 0x024, 0x3a7, 0x060, 0x083, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1a9,
+ 0x000, 0x000, 0x351, 0x307, 0x22b, 0x284, 0x2a8, 0x2c4,
+ 0x24d, 0x245, 0x24d, 0x2a8, 0x045, 0x2f4, 0x200, 0x001,
+ 0x008, 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287,
+ 0x385, 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010,
+ 0x029, 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384,
+ 0x0ff, 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d,
+ 0x040, 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d,
+ 0x042, 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005,
+ 0x3c5, 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264,
+ 0x080, 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5,
+ 0x080, 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f,
+ 0x02d, 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff,
+ 0x1b8, 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021,
+ 0x3c4, 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a,
+ 0x020, 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x251,
+ 0x22e, 0x364, 0x026, 0x001, 0x021, 0x181, 0x006, 0x041,
+ 0x3a4, 0x01f, 0x001, 0x005, 0x044, 0x0d9, 0x001, 0x043,
+ 0x1b7, 0x001, 0x005, 0x044, 0x182, 0x000, 0x020, 0x181,
+ 0x006, 0x2e4, 0x002, 0x009, 0x01b, 0x3c4, 0x2ff, 0x232,
+ 0x120, 0x001, 0x012, 0x3d5, 0x300, 0x232, 0x044, 0x099,
+ 0x00b, 0x32a, 0x026, 0x00a, 0x008, 0x044, 0x20d, 0x000,
+ 0x044, 0x232, 0x000, 0x020, 0x181, 0x006, 0x042, 0x307,
+ 0x240, 0x20a, 0x040, 0x020, 0x181, 0x006, 0x289, 0x00a,
+ 0x004, 0x2c7, 0x256, 0x22e, 0x364, 0x026, 0x002, 0x021,
+ 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005, 0x044,
+ 0x0de, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044, 0x0e2,
+ 0x001, 0x020, 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009,
+ 0x007, 0x044, 0x340, 0x001, 0x247, 0x249, 0x054, 0x347,
+ 0x249, 0x248, 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018,
+ 0x364, 0x23f, 0x001, 0x001, 0x006, 0x245, 0x057, 0x000,
+ 0x005, 0x208, 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0,
+ 0x364, 0x23f, 0x002, 0x001, 0x009, 0x245, 0x057, 0x044,
+ 0x27d, 0x001, 0x000, 0x005, 0x208, 0x244, 0x057, 0x020,
+ 0x181, 0x006, 0x2c7, 0x246, 0x3b7, 0x300, 0x044, 0x192,
+ 0x000, 0x384, 0x33f, 0x327, 0x246, 0x3a4, 0x0c0, 0x285,
+ 0x3b7, 0x300, 0x044, 0x1bd, 0x000, 0x3b7, 0x200, 0x044,
+ 0x192, 0x000, 0x0a3, 0x1a9, 0x2a8, 0x284, 0x040, 0x307,
+ 0x246, 0x384, 0x030, 0x264, 0x010, 0x001, 0x012, 0x040,
+ 0x041, 0x3b7, 0x201, 0x044, 0x192, 0x000, 0x384, 0x3e0,
+ 0x385, 0x006, 0x044, 0x1bd, 0x000, 0x043, 0x042, 0x185,
+ 0x2a8, 0x284, 0x227, 0x042, 0x285, 0x3b7, 0x200, 0x044,
+ 0x1bd, 0x000, 0x307, 0x246, 0x384, 0x007, 0x044, 0x375,
+ 0x00b, 0x3a7, 0x073, 0x307, 0x246, 0x184, 0x00a, 0x014,
+ 0x044, 0x192, 0x000, 0x384, 0x3e2, 0x327, 0x246, 0x1b5,
+ 0x3a4, 0x01c, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000,
+ 0x000, 0x006, 0x081, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x3c5, 0x001, 0x00c, 0x187, 0x0a7, 0x1a7, 0x2a8,
+ 0x324, 0x00c, 0x225, 0x2c7, 0x00c, 0x045, 0x04e, 0x087,
+ 0x011, 0x054, 0x084, 0x020, 0x01e, 0x00f, 0x018, 0x01d,
+ 0x029, 0x00b, 0x01d, 0x02c, 0x331, 0x04e, 0x020, 0x0d3,
+ 0x000, 0x020, 0x167, 0x00b, 0x020, 0x3f6, 0x00a, 0x020,
+ 0x096, 0x00b, 0x020, 0x0d3, 0x000, 0x020, 0x0d3, 0x000,
+ 0x020, 0x00d, 0x00b, 0x020, 0x179, 0x00b, 0x0c1, 0x244,
+ 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200, 0x020, 0x0d3,
+ 0x000, 0x020, 0x2c3, 0x008, 0x04e, 0x04e, 0x045, 0x0c2,
+ 0x201, 0x045, 0x020, 0x3ed, 0x00a, 0x020, 0x256, 0x002,
+ 0x020, 0x256, 0x002, 0x020, 0x256, 0x002, 0x020, 0x23f,
+ 0x006, 0x020, 0x160, 0x00b, 0x020, 0x28d, 0x00b, 0x020,
+ 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007, 0x020,
+ 0x160, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x2d5, 0x006, 0x020, 0x326,
+ 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007, 0x020,
+ 0x35a, 0x006, 0x020, 0x177, 0x00b, 0x020, 0x38e, 0x006,
+ 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020, 0x31e,
+ 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007, 0x020,
+ 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060, 0x007,
+ 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020, 0x2d1,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252, 0x007,
+ 0x020, 0x293, 0x007, 0x020, 0x2fb, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007, 0x020,
+ 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007, 0x020,
+ 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5, 0x008,
+ 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287, 0x264,
+ 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020,
+ 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004, 0x3a4,
+ 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009, 0x093,
+ 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff, 0x053,
+ 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052,
+ 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de, 0x007,
+ 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c, 0x100,
+ 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c, 0x3d5,
+ 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066, 0x004,
+ 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b, 0x262,
+ 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef,
+ 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009, 0x007,
+ 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200, 0x004,
+ 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001, 0x028,
+ 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344, 0x21f,
+ 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183, 0x19b,
+ 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001, 0x004,
+ 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5, 0x080,
+ 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044, 0x001,
+ 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5, 0x004,
+ 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x097,
+ 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d, 0x001,
+ 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390, 0x1d7,
+ 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000, 0x021,
+ 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000, 0x025,
+ 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000, 0x041,
+ 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000, 0x031,
+ 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007, 0x044,
+ 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072, 0x2c7,
+ 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a, 0x2c7,
+ 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062, 0x2c7,
+ 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a, 0x2c7,
+ 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052, 0x2c7,
+ 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a, 0x307,
+ 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007, 0x006,
+ 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020, 0x1de,
+ 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3, 0x052,
+ 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5,
+ 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff,
+ 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf, 0x354,
+ 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222, 0x355,
+ 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a, 0x006,
+ 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357, 0x020,
+ 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4, 0x00f,
+ 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020, 0x131,
+ 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001, 0x025,
+ 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132, 0x182,
+ 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de, 0x007,
+ 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373, 0x000,
+ 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3, 0x322,
+ 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7, 0x327,
+ 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df, 0x307,
+ 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108, 0x04c,
+ 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b, 0x364,
+ 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f, 0x181,
+ 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5, 0x080,
+ 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7, 0x392,
+ 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5, 0x008,
+ 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364, 0x33c,
+ 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000, 0x018,
+ 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140, 0x04f,
+ 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3, 0x2c5,
+ 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081, 0x189,
+ 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000,
+ 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053, 0x3c4,
+ 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a, 0x364,
+ 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004, 0x020,
+ 0x08d, 0x006, 0x364, 0x026, 0x010, 0x009, 0x0e8, 0x0c9,
+ 0x2ce, 0x0c8, 0x30f, 0x362, 0x2d2, 0x002, 0x009, 0x004,
+ 0x0ce, 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x0af, 0x0c4,
+ 0x303, 0x364, 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303,
+ 0x0c1, 0x310, 0x364, 0x30f, 0x001, 0x009, 0x004, 0x152,
+ 0x310, 0x397, 0x2b2, 0x300, 0x30f, 0x04c, 0x2c7, 0x305,
+ 0x397, 0x2aa, 0x300, 0x303, 0x04c, 0x2c0, 0x305, 0x342,
+ 0x310, 0x305, 0x397, 0x2a1, 0x300, 0x303, 0x327, 0x305,
+ 0x04d, 0x044, 0x1bf, 0x003, 0x08c, 0x364, 0x30f, 0x002,
+ 0x001, 0x003, 0x108, 0x364, 0x30f, 0x004, 0x001, 0x003,
+ 0x101, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x364, 0x2cf, 0x002, 0x001, 0x014, 0x309, 0x013, 0x34b,
+ 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b,
+ 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336, 0x014, 0x397,
+ 0x2a1, 0x300, 0x303, 0x04c, 0x320, 0x310, 0x04d, 0x2e4,
+ 0x100, 0x001, 0x004, 0x000, 0x012, 0x044, 0x3f0, 0x002,
+ 0x044, 0x30e, 0x002, 0x140, 0x00f, 0x009, 0x008, 0x307,
+ 0x00e, 0x302, 0x2cd, 0x00a, 0x3e4, 0x0c4, 0x303, 0x364,
+ 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303, 0x397, 0x2a1,
+ 0x300, 0x303, 0x04c, 0x2c7, 0x305, 0x397, 0x2aa, 0x300,
+ 0x303, 0x04c, 0x2c2, 0x305, 0x342, 0x310, 0x305, 0x397,
+ 0x2b2, 0x300, 0x30f, 0x327, 0x305, 0x04d, 0x362, 0x2d2,
+ 0x002, 0x001, 0x00f, 0x364, 0x2cb, 0x002, 0x009, 0x00a,
+ 0x307, 0x30f, 0x118, 0x390, 0x2b2, 0x327, 0x305, 0x04d,
+ 0x141, 0x30f, 0x000, 0x350, 0x044, 0x275, 0x00b, 0x0c0,
+ 0x2a9, 0x0c0, 0x303, 0x362, 0x303, 0x008, 0x00f, 0x017,
+ 0x397, 0x2b2, 0x300, 0x303, 0x04c, 0x2a7, 0x362, 0x303,
+ 0x004, 0x00f, 0x006, 0x2c0, 0x2a9, 0x000, 0x004, 0x2c2,
+ 0x2a9, 0x141, 0x303, 0x000, 0x3e8, 0x30a, 0x2a9, 0x20a,
+ 0x24a, 0x2a9, 0x044, 0x022, 0x009, 0x3c4, 0x3ef, 0x026,
+ 0x045, 0x307, 0x2b2, 0x300, 0x2b3, 0x247, 0x2a1, 0x362,
+ 0x2a1, 0x000, 0x006, 0x004, 0x141, 0x2a1, 0x30a, 0x2a1,
+ 0x300, 0x2aa, 0x247, 0x2a1, 0x307, 0x2b4, 0x300, 0x2b5,
+ 0x247, 0x2a2, 0x362, 0x2a2, 0x000, 0x006, 0x004, 0x141,
+ 0x2a2, 0x30a, 0x2a2, 0x300, 0x2ab, 0x247, 0x2a2, 0x307,
+ 0x2b6, 0x300, 0x2b7, 0x247, 0x2a3, 0x362, 0x2a3, 0x000,
+ 0x006, 0x004, 0x141, 0x2a3, 0x30a, 0x2a3, 0x300, 0x2ac,
+ 0x247, 0x2a3, 0x307, 0x2b8, 0x300, 0x2b9, 0x247, 0x2a4,
+ 0x362, 0x2a4, 0x000, 0x006, 0x004, 0x141, 0x2a4, 0x30a,
+ 0x2a4, 0x300, 0x2ad, 0x247, 0x2a4, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x044, 0x3f0, 0x002, 0x045, 0x0c0,
+ 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x00c, 0x397, 0x2b2,
+ 0x300, 0x30f, 0x0a0, 0x04d, 0x141, 0x30f, 0x000, 0x3f3,
+ 0x045, 0x364, 0x2d4, 0x002, 0x001, 0x003, 0x045, 0x0c7,
+ 0x2ce, 0x347, 0x290, 0x2fb, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003, 0x309,
+ 0x013, 0x34b, 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336,
+ 0x014, 0x307, 0x2ae, 0x300, 0x2ba, 0x114, 0x247, 0x2a5,
+ 0x307, 0x2af, 0x300, 0x2bd, 0x104, 0x247, 0x2a6, 0x3c7,
+ 0x3ff, 0x301, 0x151, 0x2a6, 0x141, 0x2a5, 0x364, 0x2a6,
+ 0x100, 0x001, 0x004, 0x000, 0x01a, 0x364, 0x2a5, 0x100,
+ 0x001, 0x004, 0x000, 0x013, 0x044, 0x014, 0x003, 0x044,
+ 0x30e, 0x002, 0x307, 0x2cd, 0x0c0, 0x336, 0x302, 0x00e,
+ 0x343, 0x00f, 0x336, 0x00a, 0x3df, 0x3ce, 0x3ff, 0x2fe,
+ 0x141, 0x2a6, 0x00b, 0x004, 0x0c0, 0x2a6, 0x151, 0x2a5,
+ 0x00b, 0x004, 0x0c0, 0x2a5, 0x362, 0x2a6, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a6, 0x362, 0x2a5, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a5, 0x044, 0x014, 0x003, 0x309,
+ 0x303, 0x242, 0x290, 0x0c0, 0x304, 0x362, 0x304, 0x003,
+ 0x00f, 0x02c, 0x340, 0x303, 0x290, 0x362, 0x290, 0x000,
+ 0x00f, 0x004, 0x0c0, 0x290, 0x362, 0x290, 0x0ff, 0x006,
+ 0x004, 0x000, 0x01b, 0x044, 0x034, 0x003, 0x044, 0x1cc,
+ 0x003, 0x044, 0x30e, 0x002, 0x0c0, 0x2ff, 0x044, 0x195,
+ 0x003, 0x140, 0x2ff, 0x001, 0x005, 0x347, 0x290, 0x305,
+ 0x141, 0x304, 0x000, 0x3d3, 0x362, 0x301, 0x000, 0x00f,
+ 0x01b, 0x347, 0x305, 0x301, 0x347, 0x2fb, 0x290, 0x3c6,
+ 0x0ff, 0x01a, 0x307, 0x2ae, 0x300, 0x2be, 0x114, 0x247,
+ 0x2a5, 0x307, 0x2af, 0x300, 0x2c1, 0x104, 0x247, 0x2a6,
+ 0x000, 0x36a, 0x307, 0x305, 0x300, 0x301, 0x24a, 0x290,
+ 0x044, 0x034, 0x003, 0x045, 0x0c0, 0x2e4, 0x0c3, 0x2ce,
+ 0x044, 0x1bf, 0x003, 0x0c0, 0x2ea, 0x044, 0x078, 0x009,
+ 0x347, 0x28f, 0x318, 0x347, 0x28e, 0x319, 0x0c0, 0x31b,
+ 0x0c0, 0x31c, 0x3c7, 0x038, 0x317, 0x364, 0x2d4, 0x020,
+ 0x001, 0x005, 0x347, 0x2c6, 0x317, 0x364, 0x2d4, 0x002,
+ 0x001, 0x005, 0x347, 0x290, 0x317, 0x0c0, 0x31e, 0x0cf,
+ 0x324, 0x364, 0x2d4, 0x004, 0x001, 0x008, 0x347, 0x28f,
+ 0x31e, 0x347, 0x28f, 0x324, 0x0c0, 0x31f, 0x0cf, 0x325,
+ 0x364, 0x2d4, 0x008, 0x001, 0x008, 0x347, 0x28e, 0x31f,
+ 0x347, 0x28e, 0x325, 0x364, 0x2d4, 0x010, 0x001, 0x00d,
+ 0x347, 0x291, 0x320, 0x347, 0x291, 0x326, 0x347, 0x291,
+ 0x31a, 0x000, 0x00a, 0x0cd, 0x291, 0x0cd, 0x31a, 0x0c0,
+ 0x320, 0x0cd, 0x326, 0x044, 0x1bf, 0x00a, 0x364, 0x24d,
+ 0x030, 0x009, 0x028, 0x309, 0x2c7, 0x34b, 0x2c8, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x34e, 0x2c8, 0x337, 0x32c,
+ 0x2c7, 0x282, 0x343, 0x337, 0x336, 0x247, 0x332, 0x347,
+ 0x336, 0x333, 0x307, 0x332, 0x347, 0x333, 0x336, 0x300,
+ 0x2c7, 0x341, 0x2c8, 0x336, 0x247, 0x334, 0x347, 0x336,
+ 0x335, 0x0c0, 0x398, 0x364, 0x2df, 0x002, 0x001, 0x006,
+ 0x0c1, 0x39a, 0x000, 0x004, 0x0c4, 0x39a, 0x347, 0x2cd,
+ 0x397, 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x39a, 0x006,
+ 0x076, 0x044, 0x16d, 0x00a, 0x0cf, 0x28f, 0x362, 0x28f,
+ 0x000, 0x007, 0x065, 0x362, 0x28f, 0x00f, 0x00f, 0x009,
+ 0x08f, 0x302, 0x28f, 0x247, 0x28e, 0x000, 0x004, 0x0c0,
+ 0x28e, 0x362, 0x28f, 0x006, 0x00f, 0x004, 0x0cf, 0x28e,
+ 0x0c1, 0x394, 0x362, 0x28e, 0x00a, 0x006, 0x004, 0x0d1,
+ 0x394, 0x0c0, 0x2a9, 0x307, 0x2a9, 0x302, 0x394, 0x006,
+ 0x03b, 0x347, 0x2a9, 0x313, 0x307, 0x2aa, 0x300, 0x2a9,
+ 0x247, 0x2a1, 0x307, 0x2ab, 0x300, 0x2a9, 0x247, 0x2a2,
+ 0x307, 0x2ac, 0x302, 0x2a9, 0x247, 0x2a3, 0x307, 0x2ad,
+ 0x302, 0x2a9, 0x247, 0x2a4, 0x044, 0x12e, 0x00a, 0x347,
+ 0x31d, 0x290, 0x307, 0x290, 0x302, 0x323, 0x007, 0x00a,
+ 0x044, 0x015, 0x00a, 0x340, 0x329, 0x290, 0x000, 0x3f4,
+ 0x307, 0x314, 0x300, 0x315, 0x247, 0x316, 0x141, 0x2a9,
+ 0x000, 0x3c3, 0x151, 0x28f, 0x000, 0x39a, 0x340, 0x317,
+ 0x398, 0x141, 0x399, 0x000, 0x388, 0x347, 0x398, 0x317,
+ 0x362, 0x39a, 0x004, 0x009, 0x007, 0x307, 0x398, 0x191,
+ 0x247, 0x317, 0x044, 0x177, 0x00a, 0x347, 0x317, 0x2e9,
+ 0x347, 0x31b, 0x2a9, 0x347, 0x2a9, 0x313, 0x347, 0x316,
+ 0x2ea, 0x0c0, 0x395, 0x347, 0x31e, 0x28f, 0x307, 0x28f,
+ 0x302, 0x324, 0x007, 0x093, 0x0c3, 0x2ce, 0x347, 0x325,
+ 0x28e, 0x347, 0x326, 0x291, 0x044, 0x3ad, 0x002, 0x0c3,
+ 0x303, 0x044, 0x3d4, 0x009, 0x347, 0x2a5, 0x2e2, 0x364,
+ 0x2d4, 0x008, 0x001, 0x007, 0x364, 0x2d4, 0x010, 0x009,
+ 0x05d, 0x0cf, 0x32e, 0x362, 0x32e, 0x000, 0x007, 0x02d,
+ 0x364, 0x2d4, 0x008, 0x009, 0x00c, 0x347, 0x32e, 0x28e,
+ 0x362, 0x28e, 0x00f, 0x006, 0x004, 0x0cf, 0x28e, 0x364,
+ 0x2d4, 0x010, 0x009, 0x005, 0x347, 0x32e, 0x291, 0x044,
+ 0x3ad, 0x002, 0x0c0, 0x303, 0x044, 0x3d4, 0x009, 0x307,
+ 0x2a5, 0x302, 0x2e2, 0x00e, 0x004, 0x000, 0x006, 0x151,
+ 0x32e, 0x000, 0x3d2, 0x364, 0x2d4, 0x008, 0x001, 0x006,
+ 0x141, 0x291, 0x000, 0x022, 0x307, 0x28e, 0x302, 0x325,
+ 0x006, 0x01c, 0x307, 0x2a5, 0x302, 0x2e2, 0x00e, 0x016,
+ 0x141, 0x28e, 0x307, 0x28e, 0x302, 0x325, 0x009, 0x004,
+ 0x000, 0x00c, 0x044, 0x034, 0x003, 0x0c0, 0x303, 0x044,
+ 0x3d4, 0x009, 0x000, 0x3e2, 0x362, 0x291, 0x00d, 0x006,
+ 0x004, 0x0cd, 0x291, 0x397, 0x39b, 0x300, 0x28f, 0x327,
+ 0x28e, 0x04d, 0x397, 0x3ab, 0x300, 0x28f, 0x327, 0x291,
+ 0x04d, 0x141, 0x28f, 0x000, 0x36b, 0x044, 0x16d, 0x00a,
+ 0x347, 0x2c7, 0x2c2, 0x347, 0x2c8, 0x2c3, 0x0c4, 0x397,
+ 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x2e0, 0x006, 0x00f,
+ 0x349, 0x2c7, 0x2c7, 0x34b, 0x2c8, 0x2c8, 0x349, 0x397,
+ 0x397, 0x141, 0x399, 0x000, 0x3ef, 0x044, 0x1bf, 0x00a,
+ 0x347, 0x31e, 0x28f, 0x307, 0x28f, 0x302, 0x324, 0x007,
+ 0x055, 0x0c3, 0x2ce, 0x397, 0x39b, 0x300, 0x28f, 0x04c,
+ 0x2c7, 0x28e, 0x397, 0x3ab, 0x300, 0x28f, 0x04c, 0x2c7,
+ 0x291, 0x347, 0x317, 0x290, 0x044, 0x3ad, 0x002, 0x0c0,
+ 0x32e, 0x362, 0x32e, 0x010, 0x00f, 0x023, 0x0c1, 0x30a,
+ 0x0c0, 0x330, 0x362, 0x330, 0x004, 0x00f, 0x013, 0x364,
+ 0x026, 0x010, 0x009, 0x00a, 0x362, 0x32e, 0x002, 0x006,
+ 0x005, 0x044, 0x1e6, 0x00a, 0x141, 0x330, 0x000, 0x3ec,
+ 0x3c4, 0x3ef, 0x026, 0x141, 0x32e, 0x000, 0x3dc, 0x0c3,
+ 0x2ce, 0x044, 0x1bf, 0x003, 0x044, 0x30e, 0x002, 0x044,
+ 0x015, 0x00a, 0x307, 0x314, 0x300, 0x315, 0x247, 0x316,
+ 0x141, 0x28f, 0x000, 0x3a9, 0x044, 0x3b3, 0x002, 0x362,
+ 0x316, 0x006, 0x00f, 0x004, 0x0c0, 0x396, 0x044, 0x177,
+ 0x00a, 0x347, 0x316, 0x2ec, 0x347, 0x2c2, 0x2c7, 0x347,
+ 0x2c3, 0x2c8, 0x045, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x207, 0x384, 0x008, 0x247, 0x304, 0x09c, 0x189, 0x395,
+ 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x3c4, 0x300, 0x01a,
+ 0x140, 0x303, 0x009, 0x010, 0x140, 0x304, 0x009, 0x007,
+ 0x3c5, 0x004, 0x01a, 0x000, 0x013, 0x3c5, 0x020, 0x01a,
+ 0x000, 0x00e, 0x140, 0x304, 0x009, 0x007, 0x3c5, 0x080,
+ 0x01a, 0x000, 0x005, 0x3c5, 0x001, 0x01a, 0x044, 0x1ba,
+ 0x003, 0x3d7, 0x200, 0x013, 0x0c0, 0x014, 0x397, 0x2a5,
+ 0x044, 0x1e9, 0x003, 0x045, 0x0c0, 0x303, 0x0c0, 0x330,
+ 0x0c0, 0x331, 0x0c0, 0x32e, 0x0c0, 0x32f, 0x044, 0x3ad,
+ 0x002, 0x0c0, 0x303, 0x307, 0x303, 0x302, 0x2d1, 0x006,
+ 0x0bb, 0x364, 0x303, 0x001, 0x001, 0x016, 0x307, 0x2ae,
+ 0x302, 0x313, 0x114, 0x302, 0x315, 0x247, 0x2a5, 0x307,
+ 0x2af, 0x302, 0x313, 0x104, 0x300, 0x315, 0x247, 0x2a6,
+ 0x000, 0x014, 0x307, 0x2ae, 0x300, 0x2a9, 0x114, 0x302,
+ 0x314, 0x247, 0x2a5, 0x307, 0x2af, 0x300, 0x2a9, 0x104,
+ 0x300, 0x314, 0x247, 0x2a6, 0x362, 0x2a5, 0x000, 0x00f,
+ 0x003, 0x045, 0x362, 0x2a6, 0x0ff, 0x006, 0x003, 0x045,
+ 0x044, 0x3f0, 0x002, 0x362, 0x303, 0x002, 0x00f, 0x02c,
+ 0x083, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x044, 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x334,
+ 0x347, 0x335, 0x336, 0x302, 0x046, 0x343, 0x047, 0x336,
+ 0x00a, 0x00e, 0x307, 0x046, 0x347, 0x047, 0x336, 0x302,
+ 0x332, 0x343, 0x333, 0x336, 0x002, 0x006, 0x347, 0x323,
+ 0x290, 0x045, 0x307, 0x303, 0x384, 0x001, 0x104, 0x18a,
+ 0x385, 0x101, 0x207, 0x0a0, 0x044, 0x18a, 0x002, 0x044,
+ 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x046, 0x347,
+ 0x047, 0x336, 0x302, 0x332, 0x343, 0x333, 0x336, 0x002,
+ 0x006, 0x347, 0x323, 0x290, 0x045, 0x364, 0x303, 0x001,
+ 0x001, 0x00a, 0x340, 0x00e, 0x330, 0x341, 0x00f, 0x331,
+ 0x000, 0x008, 0x340, 0x00e, 0x32e, 0x341, 0x00f, 0x32f,
+ 0x307, 0x2fd, 0x347, 0x2fe, 0x336, 0x327, 0x32e, 0x347,
+ 0x32f, 0x337, 0x320, 0x330, 0x341, 0x331, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x003, 0x045, 0x141, 0x303,
+ 0x000, 0x343, 0x347, 0x290, 0x317, 0x347, 0x28f, 0x318,
+ 0x347, 0x28e, 0x319, 0x347, 0x291, 0x31a, 0x347, 0x2a9,
+ 0x31b, 0x347, 0x313, 0x31c, 0x347, 0x293, 0x396, 0x307,
+ 0x32e, 0x347, 0x32f, 0x336, 0x300, 0x330, 0x341, 0x331,
+ 0x336, 0x247, 0x2fd, 0x347, 0x336, 0x2fe, 0x307, 0x2fd,
+ 0x347, 0x2fe, 0x336, 0x327, 0x397, 0x0c0, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x019, 0x30a, 0x397, 0x327,
+ 0x330, 0x282, 0x00a, 0x004, 0x141, 0x315, 0x30a, 0x397,
+ 0x327, 0x32e, 0x282, 0x00a, 0x004, 0x141, 0x314, 0x3ce,
+ 0x3ff, 0x2fe, 0x000, 0x2ea, 0x045, 0x364, 0x2d4, 0x002,
+ 0x001, 0x00c, 0x347, 0x317, 0x31d, 0x347, 0x317, 0x323,
+ 0x0c1, 0x329, 0x000, 0x031, 0x362, 0x316, 0x000, 0x00e,
+ 0x007, 0x364, 0x2d4, 0x020, 0x001, 0x01f, 0x307, 0x317,
+ 0x114, 0x247, 0x31d, 0x307, 0x317, 0x104, 0x247, 0x323,
+ 0x0c4, 0x329, 0x362, 0x31d, 0x000, 0x00f, 0x004, 0x0c0,
+ 0x31d, 0x362, 0x323, 0x0b8, 0x006, 0x00f, 0x3c7, 0x0b8,
+ 0x323, 0x000, 0x00a, 0x3c7, 0x038, 0x31d, 0x3c7, 0x0b8,
+ 0x323, 0x0c4, 0x329, 0x045, 0x0c0, 0x314, 0x0c0, 0x315,
+ 0x0c0, 0x316, 0x3ce, 0x3ff, 0x2fe, 0x045, 0x347, 0x319,
+ 0x28e, 0x347, 0x318, 0x28f, 0x347, 0x317, 0x290, 0x347,
+ 0x31a, 0x291, 0x347, 0x396, 0x293, 0x347, 0x31b, 0x2bd,
+ 0x347, 0x2bd, 0x2bc, 0x347, 0x2bc, 0x2bb, 0x347, 0x2bb,
+ 0x2ba, 0x347, 0x2ba, 0x2b5, 0x347, 0x2b5, 0x2b4, 0x347,
+ 0x2b4, 0x2b3, 0x347, 0x2b3, 0x2b2, 0x080, 0x302, 0x31c,
+ 0x247, 0x2c1, 0x347, 0x2c1, 0x2c0, 0x347, 0x2c0, 0x2bf,
+ 0x347, 0x2bf, 0x2be, 0x347, 0x2be, 0x2b9, 0x347, 0x2b9,
+ 0x2b8, 0x347, 0x2b8, 0x2b7, 0x347, 0x2b7, 0x2b6, 0x044,
+ 0x3ad, 0x002, 0x044, 0x022, 0x009, 0x045, 0x309, 0x2c7,
+ 0x34b, 0x2c8, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209,
+ 0x34b, 0x336, 0x336, 0x300, 0x2c7, 0x341, 0x2c8, 0x336,
+ 0x247, 0x332, 0x347, 0x336, 0x333, 0x309, 0x2c7, 0x34b,
+ 0x2c8, 0x336, 0x300, 0x332, 0x341, 0x333, 0x336, 0x247,
+ 0x334, 0x347, 0x336, 0x335, 0x045, 0x3a7, 0x022, 0x090,
+ 0x364, 0x2d4, 0x001, 0x001, 0x00b, 0x364, 0x2cf, 0x001,
+ 0x001, 0x005, 0x044, 0x1a9, 0x000, 0x045, 0x044, 0x1b7,
+ 0x000, 0x362, 0x2f6, 0x002, 0x00f, 0x02b, 0x0c0, 0x304,
+ 0x044, 0x25b, 0x003, 0x347, 0x2a5, 0x2e2, 0x001, 0x007,
+ 0x347, 0x2a6, 0x2e1, 0x009, 0x005, 0x141, 0x2ee, 0x045,
+ 0x364, 0x2f6, 0x001, 0x001, 0x037, 0x0c1, 0x304, 0x044,
+ 0x25b, 0x003, 0x362, 0x2a5, 0x0ff, 0x001, 0x007, 0x362,
+ 0x2a6, 0x0ff, 0x009, 0x028, 0x141, 0x2ee, 0x045, 0x307,
+ 0x2bb, 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2bd, 0x300,
+ 0x2af, 0x247, 0x2a6, 0x347, 0x2a5, 0x2e2, 0x347, 0x2a6,
+ 0x2e1, 0x364, 0x2f6, 0x001, 0x001, 0x00e, 0x307, 0x2be,
+ 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2c0, 0x300, 0x2af,
+ 0x247, 0x2a6, 0x044, 0x1bf, 0x003, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x00f, 0x364, 0x2cf, 0x001, 0x001, 0x007, 0x3d7,
+ 0x200, 0x013, 0x0c0, 0x014, 0x3c5, 0x100, 0x00d, 0x0c6,
+ 0x2ce, 0x34e, 0x306, 0x306, 0x0c2, 0x308, 0x362, 0x308,
+ 0x00d, 0x00e, 0x019, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x04c, 0x2c7, 0x303, 0x34e, 0x306, 0x306, 0x364, 0x306,
+ 0x001, 0x009, 0x005, 0x044, 0x2fa, 0x00a, 0x141, 0x308,
+ 0x000, 0x3e6, 0x140, 0x30a, 0x001, 0x05c, 0x0c0, 0x303,
+ 0x0c0, 0x304, 0x362, 0x304, 0x00b, 0x00e, 0x048, 0x397,
+ 0x294, 0x300, 0x304, 0x04c, 0x2c7, 0x305, 0x362, 0x305,
+ 0x000, 0x00f, 0x007, 0x308, 0x305, 0x101, 0x247, 0x305,
+ 0x307, 0x305, 0x302, 0x303, 0x00f, 0x005, 0x347, 0x305,
+ 0x303, 0x362, 0x305, 0x00f, 0x009, 0x025, 0x141, 0x293,
+ 0x0c0, 0x30a, 0x0c4, 0x2f7, 0x0c0, 0x304, 0x362, 0x304,
+ 0x00b, 0x00e, 0x01c, 0x397, 0x294, 0x300, 0x304, 0x04c,
+ 0x2c7, 0x305, 0x397, 0x294, 0x300, 0x304, 0x329, 0x305,
+ 0x320, 0x305, 0x2aa, 0x2aa, 0x04d, 0x141, 0x304, 0x000,
+ 0x3e7, 0x141, 0x304, 0x000, 0x3b7, 0x362, 0x303, 0x00a,
+ 0x00f, 0x008, 0x151, 0x293, 0x0c0, 0x30a, 0x0c4, 0x2f7,
+ 0x044, 0x06e, 0x003, 0x044, 0x096, 0x003, 0x045, 0x307,
+ 0x2a5, 0x347, 0x2e2, 0x2a5, 0x247, 0x2e2, 0x307, 0x2a6,
+ 0x347, 0x2e1, 0x2a6, 0x247, 0x2e1, 0x044, 0x014, 0x003,
+ 0x045, 0x347, 0x303, 0x304, 0x347, 0x304, 0x30e, 0x3ce,
+ 0x3ff, 0x2fe, 0x152, 0x303, 0x0c0, 0x305, 0x362, 0x305,
+ 0x003, 0x00f, 0x0b9, 0x141, 0x303, 0x362, 0x303, 0x3f1,
+ 0x00f, 0x004, 0x000, 0x0ac, 0x362, 0x303, 0x00f, 0x006,
+ 0x007, 0x347, 0x30c, 0x30d, 0x000, 0x0a6, 0x307, 0x308,
+ 0x112, 0x390, 0x294, 0x327, 0x303, 0x04d, 0x044, 0x096,
+ 0x003, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002, 0x00f,
+ 0x00d, 0x08c, 0x189, 0x395, 0x104, 0x328, 0x308, 0x121,
+ 0x3a4, 0x0ff, 0x000, 0x007, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x30e, 0x002, 0x347,
+ 0x00e, 0x32e, 0x347, 0x00f, 0x32f, 0x364, 0x2f6, 0x001,
+ 0x001, 0x024, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x010, 0x08e, 0x189, 0x395, 0x104, 0x328, 0x308,
+ 0x121, 0x3a4, 0x0ff, 0x044, 0x18a, 0x002, 0x000, 0x005,
+ 0x3c6, 0x0ff, 0x01a, 0x044, 0x30e, 0x002, 0x340, 0x00e,
+ 0x32e, 0x341, 0x00f, 0x32f, 0x140, 0x305, 0x001, 0x007,
+ 0x362, 0x303, 0x3f1, 0x009, 0x005, 0x347, 0x32f, 0x30b,
+ 0x362, 0x305, 0x001, 0x009, 0x005, 0x347, 0x32f, 0x30c,
+ 0x362, 0x305, 0x002, 0x009, 0x005, 0x347, 0x32f, 0x30d,
+ 0x362, 0x32f, 0x000, 0x00f, 0x00d, 0x348, 0x32e, 0x32e,
+ 0x348, 0x32f, 0x32f, 0x141, 0x32e, 0x3c1, 0x000, 0x32f,
+ 0x307, 0x32e, 0x347, 0x32f, 0x336, 0x302, 0x2fd, 0x343,
+ 0x2fe, 0x336, 0x002, 0x00b, 0x347, 0x303, 0x304, 0x347,
+ 0x32e, 0x2fd, 0x347, 0x32f, 0x2fe, 0x140, 0x305, 0x009,
+ 0x007, 0x141, 0x2fd, 0x3c1, 0x000, 0x2fe, 0x141, 0x305,
+ 0x000, 0x346, 0x307, 0x30b, 0x306, 0x30d, 0x003, 0x01d,
+ 0x307, 0x30e, 0x111, 0x247, 0x304, 0x362, 0x30c, 0x000,
+ 0x00f, 0x004, 0x142, 0x304, 0x362, 0x304, 0x3f1, 0x00f,
+ 0x005, 0x3c7, 0x3f1, 0x304, 0x362, 0x304, 0x00f, 0x006,
+ 0x004, 0x0cf, 0x304, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x327, 0x304, 0x04d, 0x045, 0x044, 0x290, 0x001, 0x3c4,
+ 0x3fe, 0x200, 0x020, 0x2e0, 0x002, 0x045, 0x1f6, 0x185,
+ 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff,
+ 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce,
+ 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020, 0x009,
+ 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9, 0x286,
+ 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x3f7, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051, 0x2c7,
+ 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b, 0x005,
+ 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4, 0x00f,
+ 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288, 0x286,
+ 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b, 0x3fc,
+ 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285, 0x307,
+ 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd, 0x000,
+ 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c,
+ 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002, 0x005,
+ 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b, 0x1f0,
+ 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081, 0x138,
+ 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080,
+ 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d, 0x1e0,
+ 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b, 0x347,
+ 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044, 0x154,
+ 0x00b, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364, 0x233,
+ 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff, 0x242,
+ 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289, 0x397,
+ 0x00b, 0x189, 0x395, 0x062, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9, 0x121,
+ 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282,
+ 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000, 0x3ed,
+ 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233, 0x002,
+ 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff, 0x281,
+ 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366, 0x282,
+ 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282, 0x044,
+ 0x154, 0x00b, 0x001, 0x00d, 0x141, 0x28b, 0x362, 0x28b,
+ 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5, 0x362,
+ 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397, 0x00b,
+ 0x189, 0x395, 0x070, 0x044, 0x2ea, 0x001, 0x009, 0x004,
+ 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005, 0x105,
+ 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121,
+ 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003,
+ 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387, 0x00b,
+ 0x189, 0x395, 0x06b, 0x300, 0x28a, 0x050, 0x247, 0x28a,
+ 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a, 0x044,
+ 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282, 0x281,
+ 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045, 0x0c0,
+ 0x244, 0x044, 0x172, 0x000, 0x043, 0x047, 0x0a1, 0x1aa,
+ 0x2c5, 0x233, 0x0c0, 0x244, 0x044, 0x172, 0x000, 0x364,
+ 0x233, 0x001, 0x029, 0x0f4, 0x000, 0x047, 0x000, 0x3e9,
+ 0x364, 0x025, 0x004, 0x009, 0x0d4, 0x307, 0x2cf, 0x18a,
+ 0x003, 0x006, 0x002, 0x08c, 0x000, 0x0bd, 0x364, 0x200,
+ 0x100, 0x009, 0x0c0, 0x0c0, 0x2e6, 0x0c0, 0x2e7, 0x08d,
+ 0x18b, 0x385, 0x090, 0x040, 0x083, 0x040, 0x387, 0x080,
+ 0x044, 0x3e3, 0x001, 0x044, 0x346, 0x002, 0x3c5, 0x001,
+ 0x2cf, 0x0c0, 0x2d8, 0x0c0, 0x2f1, 0x3c4, 0x3ef, 0x027,
+ 0x387, 0x085, 0x187, 0x385, 0x080, 0x247, 0x041, 0x3c5,
+ 0x020, 0x00c, 0x0c0, 0x2f6, 0x044, 0x3b3, 0x002, 0x140,
+ 0x2d3, 0x009, 0x026, 0x364, 0x292, 0x008, 0x009, 0x021,
+ 0x044, 0x16d, 0x009, 0x0d0, 0x312, 0x0c0, 0x30a, 0x0c0,
+ 0x2ee, 0x0d0, 0x2f7, 0x044, 0x253, 0x00b, 0x342, 0x293,
+ 0x312, 0x001, 0x00e, 0x0c1, 0x30a, 0x347, 0x293, 0x312,
+ 0x0c8, 0x2f7, 0x044, 0x253, 0x00b, 0x000, 0x3f1, 0x3c4,
+ 0x1ff, 0x2cf, 0x0c0, 0x30a, 0x307, 0x290, 0x183, 0x305,
+ 0x28f, 0x183, 0x305, 0x28e, 0x247, 0x2f2, 0x0cf, 0x2ce,
+ 0x044, 0x113, 0x003, 0x044, 0x33f, 0x002, 0x055, 0x364,
+ 0x2cf, 0x020, 0x009, 0x005, 0x3c4, 0x3bf, 0x200, 0x3c4,
+ 0x3ee, 0x2cf, 0x364, 0x2df, 0x001, 0x009, 0x04a, 0x364,
+ 0x33c, 0x002, 0x009, 0x045, 0x141, 0x2cf, 0x044, 0x346,
+ 0x002, 0x3c5, 0x002, 0x2cf, 0x347, 0x2d5, 0x2f7, 0x0c4,
+ 0x303, 0x044, 0x08a, 0x009, 0x044, 0x333, 0x008, 0x151,
+ 0x2f7, 0x00b, 0x3f6, 0x347, 0x2d5, 0x2f7, 0x044, 0x253,
+ 0x00b, 0x055, 0x3c4, 0x3fc, 0x2cf, 0x364, 0x2cf, 0x040,
+ 0x009, 0x019, 0x3c4, 0x39d, 0x2cf, 0x0ce, 0x2ce, 0x364,
+ 0x2cf, 0x001, 0x009, 0x007, 0x0cf, 0x2ce, 0x3c4, 0x3fe,
+ 0x2cf, 0x364, 0x2cf, 0x010, 0x009, 0x005, 0x3c4, 0x3bf,
+ 0x200, 0x3c4, 0x1ff, 0x2cf, 0x044, 0x33f, 0x002, 0x020,
+ 0x0d3, 0x000, 0x347, 0x2cb, 0x306, 0x044, 0x1e6, 0x00a,
+ 0x044, 0x333, 0x008, 0x151, 0x2f7, 0x009, 0x3f5, 0x045,
+ 0x397, 0x201, 0x0a0, 0x044, 0x18a, 0x002, 0x020, 0x1bf,
+ 0x003, 0x397, 0x2ba, 0x04c, 0x118, 0x04d, 0x109, 0x272,
+ 0x2c1, 0x00b, 0x3fa, 0x045, 0x397, 0x2b2, 0x04c, 0x2c7,
+ 0x303, 0x101, 0x04c, 0x322, 0x303, 0x00b, 0x009, 0x04c,
+ 0x111, 0x04d, 0x101, 0x327, 0x303, 0x04d, 0x101, 0x272,
+ 0x2c1, 0x00b, 0x3ed, 0x045, 0x2f4, 0x200, 0x001, 0x008,
+ 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287, 0x385,
+ 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010, 0x029,
+ 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384, 0x0ff,
+ 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d, 0x040,
+ 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040, 0x02d,
+ 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d, 0x042,
+ 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005, 0x3c5,
+ 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264, 0x080,
+ 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5, 0x080,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f, 0x02d,
+ 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff, 0x1b8,
+ 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4,
+ 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a, 0x020,
+ 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044,
+ 0x340, 0x001, 0x247, 0x249, 0x054, 0x347, 0x249, 0x248,
+ 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f,
+ 0x001, 0x001, 0x006, 0x245, 0x057, 0x000, 0x005, 0x208,
+ 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f,
+ 0x002, 0x001, 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001,
+ 0x000, 0x005, 0x208, 0x244, 0x057, 0x020, 0x181, 0x006,
+ 0x04e, 0x0ce, 0x045, 0x03c, 0x0f5, 0x04e, 0x0fd, 0x07e,
+ 0x00a, 0x0f9
+
+}; /* end fm10000_serdes_swap_code_prd2 */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.h b/drivers/net/fm10k/switch/fm10k_spico_code.h
new file mode 100644
index 0000000..c172638
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SPICO_CODE_H_
+#define _FM10K_SPICO_CODE_H_
+#include <stdint.h>
+
+extern const uint32_t fm10000_sbus_master_code_version_build_id_prd;
+extern const uint32_t fm10000_sbus_master_code_size_prd;
+extern const uint16_t fm10000_sbus_master_code_prd[];
+
+extern const uint32_t fm10000_serdes_spico_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_spico_code_size_prd2;
+extern const uint16_t fm10000_serdes_spico_code_prd2[];
+
+extern const uint32_t fm10000_serdes_swap_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_swap_code_size_prd2;
+extern const uint16_t fm10000_serdes_swap_code_prd2[];
+
+#endif /* _FM10K_SPICO_CODE_H_ */
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-03-16 7:04 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
It also supports flowset for a group flow definition.
The config file can configure debug log, port speed,
epl port mapping dpdk port, flowset. All these configuration
will be used by switch management.
Statistics includes epl port, ffu rule, dpdk port, and error.
All these statistics data are read from HW.
Modify switch header file to support getting logical port
and glort and device info.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 3 +
drivers/net/fm10k/switch/fm10k_config.c | 863 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 +++++
drivers/net/fm10k/switch/fm10k_ffu.c | 1253 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_stats.c | 1114 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++++++
drivers/net/fm10k/switch/fm10k_switch.h | 158 +++-
8 files changed, 3848 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 0d3b688..d4d2037 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -88,6 +88,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_config.c b/drivers/net/fm10k/switch/fm10k_config.c
new file mode 100644
index 0000000..e958663
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.c
@@ -0,0 +1,863 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <unistd.h>
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+
+
+#define FM10K_CONFIG_WORD_MAX 10
+#define FM10K_CONFIG_WORD_LEN 20
+#define FM10K_CONFIG_STR_MAX 100
+
+/* used for store the key strings */
+struct fm10k_cfg_key_item {
+ const char *key_str[FM10K_CONFIG_WORD_MAX];
+ uint8_t type;
+};
+
+/* configuration file path */
+static const char *fm10k_config_dpdk_conf_file = "/etc/dpdk_fm10k.conf";
+/* configuration structure */
+static struct fm10k_dpdk_cfg fm10k_config_dpdk_cfg;
+
+/* configuration key strings */
+static struct fm10k_cfg_key_item fm10k_config_key_items[] = {
+ /* enable debug print */
+ { {"debug", "print", "enable"},
+ FM10K_CONFIG_DEBUG_ENABLE},
+ { {"debug", "print", "config"},
+ FM10K_CONFIG_DEBUG_CONFIG},
+ { {"debug", "print", "ffu", "init"},
+ FM10K_CONFIG_DEBUG_FFU_INIT},
+ { {"debug", "print", "ffu", "register"},
+ FM10K_CONFIG_DEBUG_FFU_REG},
+ { {"debug", "print", "ffu", "rule"},
+ FM10K_CONFIG_DEBUG_FFU_RULE},
+ { {"debug", "print", "stats", "port"},
+ FM10K_CONFIG_DEBUG_STATS_PORT},
+ { {"debug", "print", "stats", "queue"},
+ FM10K_CONFIG_DEBUG_STATS_QUEUE},
+ { {"debug", "print", "stats", "rule"},
+ FM10K_CONFIG_DEBUG_STATS_FFU},
+ { {"debug", "print", "stats", "detail"},
+ FM10K_CONFIG_DEBUG_STATS_MORE},
+ { {"debug", "print", "stats", "interval"},
+ FM10K_CONFIG_DEBUG_STATS_INTERVAL},
+ /* general configuration */
+ { {"dpdk", "bind", "pf", "number"},
+ FM10K_CONFIG_BIND_PF_NUMBER},
+ { {"extern", "port", "speed"},
+ FM10K_CONFIG_EXT_PORT_SPEED},
+ /* internal redirect configuration */
+ { {"dpdk", "port", "*", "map", "pf"},
+ FM10K_CONFIG_DPDK_PORT_MAP_PF},
+ { {"extern", "port", "*", "map", "pf"},
+ FM10K_CONFIG_EXT_PORT_MAP_PF},
+ /* external redirect configuration */
+ { {"flowset", "start"},
+ FM10K_CONFIG_FLOWSET_START},
+ { {"flowset", "stop"},
+ FM10K_CONFIG_FLOWSET_STOP},
+ { {"flowset", "enable"},
+ FM10K_CONFIG_FLOWSET_ENABLE},
+ { {"flow", "*", "condition", "source", "extern", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT},
+ { {"flow", "*", "condition", "source", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT},
+ { {"flow", "*", "condition", "vlan"},
+ FM10K_CONFIG_FLOW_COND_VLAN},
+ { {"flow", "*", "action", "forward", "extern", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT},
+ { {"flow", "*", "action", "forward", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT},
+ { {"flow", "*", "action", "forward", "vlan"},
+ FM10K_CONFIG_FLOW_ACT_FW_VALN},
+};
+
+static char default_flowset[10] = "default";
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_2pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PF.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF. ",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_4pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 4
+ },
+
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PFs.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF.",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+
+static int
+fm10k_config_conf_line_parser(char *buff, struct fm10k_cfg_config_item *item)
+{
+ char *p;
+ char *endptr;
+ const char *cfg_delims = { " " };
+ const char *key_delims = { "." };
+ struct fm10k_cfg_key_item *key;
+ char cfgs[3][FM10K_CONFIG_STR_MAX];
+ char cmp_keys[FM10K_CONFIG_WORD_MAX][FM10K_CONFIG_WORD_LEN];
+ long key_param = 0;
+ uint16_t i, j;
+ uint8_t val_type = FM10K_CONFIG_VALUE_NULL;
+
+ for (i = 0; i < 3; i++) {
+ if (i == 0)
+ p = strtok(buff, cfg_delims);
+ else
+ p = strtok(NULL, cfg_delims);
+ if (p == NULL)
+ return -1;
+ strncpy(cfgs[i], p, FM10K_CONFIG_STR_MAX);
+ }
+
+ p = strtok(NULL, cfg_delims);
+ if (p)
+ return -1;
+
+ memset(cmp_keys, 0, sizeof(cmp_keys));
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (i == 0)
+ p = strtok(cfgs[0], key_delims);
+ else
+ p = strtok(NULL, key_delims);
+ if (p == NULL)
+ break;
+ strncpy(cmp_keys[i], p, FM10K_CONFIG_WORD_LEN);
+ }
+
+ if (strcmp(cfgs[1], "int") == 0)
+ val_type = FM10K_CONFIG_VALUE_INT;
+ else if (strcmp(cfgs[1], "string") == 0)
+ val_type = FM10K_CONFIG_VALUE_STR;
+ else
+ return -1;
+
+ for (i = 0;
+ i < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ i++) {
+ key = &fm10k_config_key_items[i];
+ for (j = 0; j < FM10K_CONFIG_WORD_MAX; j++) {
+ if (key->key_str[j] &&
+ strlen(cmp_keys[j]) > 0) {
+ if (strcmp(key->key_str[j], "*") == 0) {
+ key_param =
+ strtol(cmp_keys[j], &endptr, 10);
+ if ((key_param == 0 &&
+ endptr == cmp_keys[j]) ||
+ (endptr != cmp_keys[j] &&
+ strlen(endptr) != 0))
+ break;
+ } else if (strcmp(key->key_str[j],
+ cmp_keys[j]) != 0) {
+ break;
+ }
+ } else if (key->key_str[j] == 0 &&
+ strlen(cmp_keys[j]) == 0) {
+ if (val_type == FM10K_CONFIG_VALUE_STR) {
+ item->val.str =
+ malloc(strlen(cfgs[2]) + 1);
+ if (item->val.str == NULL)
+ return -1;
+ strcpy(item->val.str, cfgs[2]);
+ } else {
+ item->val.int64 =
+ strtol(cfgs[2], NULL, 10);
+ }
+ item->type = key->type;
+ item->key_param = key_param;
+ item->val_type = val_type;
+ return 0;
+ }
+ }
+ }
+ return -1;
+}
+
+static bool
+fm10k_config_blank_line_check(char *line)
+{
+ uint16_t i;
+
+ for (i = 0; i < strlen(line); i++) {
+ if (line[i] == ' ' ||
+ line[i] == '\n' ||
+ line[i] == '\t' || line[i] == 0x0d) /* 0d: CR */
+ continue;
+ else
+ return false;
+ }
+ return true;
+}
+
+static int
+fm10k_config_conf_file_load(void)
+{
+ int i = 0;
+ FILE *fp;
+ char buff[255];
+ struct fm10k_cfg_config_item *item;
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "r");
+ if (fp == NULL)
+ return -1;
+
+ fm10k_config_dpdk_cfg.config_list = malloc
+ (sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+ if (fm10k_config_dpdk_cfg.config_list == NULL)
+ return -1;
+ memset(fm10k_config_dpdk_cfg.config_list, 0,
+ sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+
+ while (fgets(buff, sizeof(buff), fp)) {
+ if (buff[0] == '#' || buff[0] == 0)
+ continue;
+
+ if (fm10k_config_blank_line_check(buff))
+ continue;
+
+ item = &fm10k_config_dpdk_cfg.config_list[i++];
+ if (fm10k_config_conf_line_parser(buff, item) < 0) {
+ FM10K_SW_ERR("Unknown configuration: %s", buff);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+struct fm10k_hw*
+fm10k_config_hw_get(int port_no)
+{
+ int hw_port;
+
+ hw_port = fm10k_config_dpdk_cfg.dpdk_port_map[port_no].map_no[0];
+ return fm10k_config_dpdk_cfg.pf_hw[hw_port];
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_current_get(void)
+{
+ return fm10k_config_dpdk_cfg.current;
+}
+
+void
+fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new)
+{
+ fm10k_config_dpdk_cfg.current = new;
+}
+
+bool
+fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow)
+{
+ return (list == flow);
+}
+
+static void
+fm10k_config_flow_list_init(struct fm10k_cfg_flow *list)
+{
+ list->next = list;
+ list->prev = list;
+}
+
+static void
+fm10k_config_flow_list_add(struct fm10k_cfg_flow *new,
+ struct fm10k_cfg_flow *list)
+{
+ struct fm10k_cfg_flow *next = list->next;
+ next->prev = new;
+ new->next = next;
+ new->prev = list;
+ list->next = new;
+}
+
+void
+fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow)
+{
+ fm10k_config_flow_list_add(flow, flowset->flow_head.prev);
+}
+
+void
+fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow)
+{
+ flow->prev->next = flow->next;
+ flow->next->prev = flow->prev;
+ flow->prev = NULL;
+ flow->next = NULL;
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_get(const char *name)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ if (strcmp(flowset->name, name) == 0)
+ break;
+ flowset = flowset->next;
+ }
+
+ if (flowset == NULL) {
+ flowset = malloc(sizeof(struct fm10k_cfg_flowset));
+ if (flowset == NULL)
+ return NULL;
+ strcpy(flowset->name, name);
+ fm10k_config_flow_list_init(&flowset->flow_head);
+ flowset->next = fm10k_config_dpdk_cfg.flowset_head.next;
+ fm10k_config_dpdk_cfg.flowset_head.next = flowset;
+ }
+ return flowset;
+}
+
+
+static int
+fm10k_cfg_flow_item_set(struct fm10k_cfg_flowset *flowset,
+ uint8_t flow_no, uint8_t type, int64_t val)
+{
+ struct fm10k_cfg_flow *tmp;
+ struct fm10k_cfg_flow *flow = NULL;
+
+ if (flowset == NULL)
+ return -1;
+
+ tmp = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, tmp)) {
+ if (tmp->flow_no == flow_no) {
+ flow = tmp;
+ break;
+ }
+ tmp = tmp->next;
+ }
+ if (flow == NULL) {
+ flow = malloc(sizeof(struct fm10k_cfg_flow));
+ memset(flow, 0, sizeof(struct fm10k_cfg_flow));
+ flow->flow_no = flow_no;
+ fm10k_config_flow_list_add_tail(flowset, flow);
+ }
+ if (flow == NULL)
+ return -1;
+
+ switch (type) {
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ flow->src_port.vlan_id = val;
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flow->fw_port[0].vlan_id == 0)
+ flow->fw_port[0].vlan_id = val;
+ else
+ flow->fw_port[1].vlan_id = val;
+ break;
+ default:
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_transfer(void)
+{
+ int i;
+ int offset;
+ uint32_t tmp, data;
+ struct fm10k_cfg_config_item *item;
+ struct fm10k_cfg_flowset *flowset = NULL;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+
+ switch (item->type) {
+ case FM10K_CONFIG_BIND_PF_NUMBER:
+ fm10k_config_dpdk_cfg.pf_num = item->val.int64;
+ break;
+ case FM10K_CONFIG_EXT_PORT_SPEED:
+ fm10k_config_dpdk_cfg.ext_port_speed = item->val.int64;
+ break;
+ case FM10K_CONFIG_DPDK_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.dpdk_port_map[item->key_param];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+ case FM10K_CONFIG_EXT_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.ext_port_map
+ [item->key_param - 1];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+
+ case FM10K_CONFIG_DEBUG_ENABLE:
+ case FM10K_CONFIG_DEBUG_CONFIG:
+ case FM10K_CONFIG_DEBUG_FFU_INIT:
+ case FM10K_CONFIG_DEBUG_FFU_REG:
+ case FM10K_CONFIG_DEBUG_FFU_RULE:
+ case FM10K_CONFIG_DEBUG_STATS_PORT:
+ case FM10K_CONFIG_DEBUG_STATS_QUEUE:
+ case FM10K_CONFIG_DEBUG_STATS_FFU:
+ case FM10K_CONFIG_DEBUG_STATS_MORE:
+ offset = item->type - FM10K_CONFIG_DEBUG_START;
+ tmp = fm10k_config_dpdk_cfg.debug_cfg;
+ data = 1;
+ if (item->val.int64 != 1)
+ fm10k_config_dpdk_cfg.debug_cfg =
+ tmp & ~(data << offset);
+ else
+ fm10k_config_dpdk_cfg.debug_cfg |=
+ (data << offset);
+ break;
+ case FM10K_CONFIG_DEBUG_STATS_INTERVAL:
+ fm10k_config_dpdk_cfg.stats_interval = item->val.int64;
+ break;
+
+ case FM10K_CONFIG_FLOWSET_START:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ flowset = fm10k_config_flowset_get(item->val.str);
+ if (flowset == NULL)
+ return -1;
+ break;
+ case FM10K_CONFIG_FLOWSET_STOP:
+ flowset = NULL;
+ break;
+ case FM10K_CONFIG_FLOWSET_ENABLE:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ fm10k_config_dpdk_cfg.current =
+ fm10k_config_flowset_get(item->val.str);
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flowset == NULL ||
+ fm10k_cfg_flow_item_set(flowset,
+ item->key_param, item->type,
+ item->val.int64) != 0)
+ return -1;
+ break;
+ default:
+ return -1;
+ }
+ }
+ return 0;
+}
+
+static bool
+fm10k_config_conf_file_exist(void)
+{
+ if (access(fm10k_config_dpdk_conf_file, 0) == 0)
+ return true;
+ return false;
+}
+
+static int
+fm10k_config_conf_file_item_create(FILE *fp, char *buff,
+ struct fm10k_cfg_key_item *key,
+ struct fm10k_cfg_config_item *item)
+{
+ int i;
+
+ if (item->type != key->type)
+ return 0;
+
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (key->key_str[i] == 0)
+ break;
+ if (i == 0) {
+ sprintf(buff + strlen(buff),
+ "%s", key->key_str[i]);
+ } else {
+ if (strcmp(key->key_str[i], "*") == 0)
+ sprintf(buff + strlen(buff),
+ ".%u", item->key_param);
+ else
+ sprintf(buff + strlen(buff),
+ ".%s", key->key_str[i]);
+ }
+ }
+ if (item->val_type == FM10K_CONFIG_VALUE_INT)
+ sprintf(buff + strlen(buff),
+ " int %lld\n", (long long)item->val.int64);
+ else if (item->val_type == FM10K_CONFIG_VALUE_STR)
+ sprintf(buff + strlen(buff), " string %s\n", item->val.str);
+ else
+ return -1;
+ fwrite(buff, strlen(buff), 1, fp);
+ FM10K_SW_TRACE("[write] %s", buff);
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_create(void)
+{
+ uint16_t i, j;
+ struct fm10k_cfg_key_item *key;
+ struct fm10k_cfg_config_item *item;
+ FILE *fp;
+ char buff[255] = "";
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "w");
+ if (fp == NULL)
+ return -1;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+ buff[0] = 0;
+ if (strlen(item->describe) > 0)
+ sprintf(buff, "\n\n%s\n", item->describe);
+ for (j = 0;
+ j < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]);
+ j++) {
+ key = &fm10k_config_key_items[j];
+ if (fm10k_config_conf_file_item_create
+ (fp, buff, key, item) != 0) {
+ fclose(fp);
+ return -1;
+ }
+ }
+ }
+ fclose(fp);
+ return 0;
+}
+
+void
+fm10k_config_cfg_flowset_show(void)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d & %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+}
+
+static void
+fm10k_config_cfg_describe(struct fm10k_switch *sw,
+ struct fm10k_device_info *info)
+{
+ uint16_t i;
+ struct fm10k_cfg_flowset *flowset;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K STATIC CONFIG ---\n");
+ FM10K_SW_INFO(" Card Type: %s\n", info->desc);
+ FM10K_SW_INFO(" PF Max : %d\n", fm10k_config_dpdk_cfg.pf_max);
+ FM10K_SW_INFO(" PF Bind : %d\n", fm10k_config_dpdk_cfg.pf_num);
+ FM10K_SW_INFO(" DEBUG : %#x\n", fm10k_config_dpdk_cfg.debug_cfg);
+ FM10K_SW_INFO(" STATS GAP: %d sec\n",
+ fm10k_config_dpdk_cfg.stats_interval);
+ FM10K_SW_INFO(" EXT PORT speed: %d Gbps\n",
+ fm10k_config_dpdk_cfg.ext_port_speed);
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ for (i = 0; i < fm10k_config_dpdk_cfg.ext_port_num; i++) {
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[1]);
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_config_dpdk_cfg.dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[1]);
+ }
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d & %4s PORT %d"
+ " VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+ FM10K_SW_INFO("\n");
+}
+
+int
+fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw)
+{
+ struct fm10k_device_info *info = fm10k_get_device_info(hw);
+
+ fm10k_config_dpdk_cfg.stats_interval = 2;
+ fm10k_config_dpdk_cfg.pf_max = info->num_peps;
+ fm10k_config_dpdk_cfg.ext_port_num = info->num_ext_ports;
+
+ if (!fm10k_config_conf_file_exist()) {
+ if (info->num_epls == 2 && info->num_peps == 2)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_2pep;
+ else if (info->num_epls == 2 && info->num_peps == 4)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_4pep;
+ else
+ return -1;
+
+ fm10k_config_conf_file_create();
+ } else {
+ if (fm10k_config_conf_file_load() < 0)
+ return -1;
+ }
+
+ if (fm10k_config_conf_file_transfer() < 0)
+ return -1;
+ sw->dpdk_cfg = &fm10k_config_dpdk_cfg;
+
+ fm10k_config_cfg_describe(sw, info);
+
+ return 0;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_config.h b/drivers/net/fm10k/switch/fm10k_config.h
new file mode 100644
index 0000000..f1c6ee4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_CONFIG_H_
+#define _FM10K_CONFIG_H_
+
+
+#include <stdint.h>
+#include "fm10k_switch.h"
+
+/* General configuration */
+#define FM10K_CONFIG_TYPE_NULL 0
+#define FM10K_CONFIG_BIND_PF_NUMBER 1
+#define FM10K_CONFIG_EXT_PORT_SPEED 2
+
+/* Internal redirect configuration */
+#define FM10K_CONFIG_DPDK_PORT_MAP_PF 10
+#define FM10K_CONFIG_EXT_PORT_MAP_PF 11
+
+/* Debug configuration */
+#define FM10K_CONFIG_DEBUG_START 50
+#define FM10K_CONFIG_DEBUG_ENABLE (FM10K_CONFIG_DEBUG_START + 0)
+#define FM10K_CONFIG_DEBUG_CONFIG (FM10K_CONFIG_DEBUG_START + 1)
+#define FM10K_CONFIG_DEBUG_FFU_INIT (FM10K_CONFIG_DEBUG_START + 2)
+#define FM10K_CONFIG_DEBUG_FFU_REG (FM10K_CONFIG_DEBUG_START + 3)
+#define FM10K_CONFIG_DEBUG_FFU_RULE (FM10K_CONFIG_DEBUG_START + 4)
+#define FM10K_CONFIG_DEBUG_STATS_PORT (FM10K_CONFIG_DEBUG_START + 5)
+#define FM10K_CONFIG_DEBUG_STATS_QUEUE (FM10K_CONFIG_DEBUG_START + 6)
+#define FM10K_CONFIG_DEBUG_STATS_FFU (FM10K_CONFIG_DEBUG_START + 7)
+#define FM10K_CONFIG_DEBUG_STATS_MORE (FM10K_CONFIG_DEBUG_START + 8)
+#define FM10K_CONFIG_DEBUG_STATS_INTERVAL (FM10K_CONFIG_DEBUG_START + 9)
+
+/* external redirect configuration */
+#define FM10K_CONFIG_FLOWSET_START 80
+#define FM10K_CONFIG_FLOWSET_STOP 81
+#define FM10K_CONFIG_FLOWSET_ENABLE 82
+
+#define FM10K_CONFIG_FLOW_COND_START 100
+#define FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 0)
+#define FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 1)
+#define FM10K_CONFIG_FLOW_COND_VLAN \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+#define FM10K_CONFIG_FLOW_COND_END \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+
+#define FM10K_CONFIG_FLOW_ACT_START 200
+#define FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 0)
+#define FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 1)
+#define FM10K_CONFIG_FLOW_ACT_FW_VALN \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+#define FM10K_CONFIG_FLOW_ACT_END \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+
+#define FM10K_CONFIG_FLOW_NONE_PORT 0
+#define FM10K_CONFIG_FLOW_EXT_PORT 1
+#define FM10K_CONFIG_FLOW_DPDK_PORT 2
+
+#define FM10K_CONFIG_VALUE_NULL 0
+#define FM10K_CONFIG_VALUE_INT 1
+#define FM10K_CONFIG_VALUE_STR 2
+
+/* SWITCH config */
+#define FM10K_CONFIG_PORT_MAP_NULL 0
+#define FM10K_CONFIG_PORT_MAP_PF 1
+#define FM10K_CONFIG_PORT_MAP_PFS 2
+#define FM10K_CONFIG_PORT_MAP_PFSS 3
+
+/* DPDK port */
+#define FM10K_CONFIG_DPDK_NULL 0
+#define FM10K_CONFIG_DPDK_PF 1
+#define FM10K_CONFIG_DPDK_VF 2
+#define FM10K_CONFIG_DPDK_MAX 3
+
+struct fm10k_cfg_port_pf_map {
+ uint8_t type;
+ uint8_t map_no[2];
+};
+
+/* Configuration read from file */
+struct fm10k_cfg_config_item {
+ uint8_t type;
+ uint8_t val_type;
+ uint16_t key_param;
+ const char *describe;
+ union {
+ int64_t int64;
+ char *str;
+ } val;
+};
+
+struct fm10k_hw;
+
+struct fm10k_dpdk_port {
+ uint8_t type;
+ struct fm10k_hw *hw;
+ void *rte_dev;
+ void *flow_list;
+ uint16_t pf_no;
+ uint8_t tx_queue_num;
+ uint8_t rx_queue_num;
+};
+
+
+/* Flow configuration */
+struct fm10k_cfg_port {
+ uint8_t port_type;
+ uint8_t port_no;
+ uint16_t vlan_id;
+};
+
+struct fm10k_cfg_flow {
+ /* set by configuration */
+ struct fm10k_cfg_flow *prev;
+ struct fm10k_cfg_flow *next;
+ uint8_t flow_no; /* only configured flow has this NO. */
+ struct fm10k_cfg_port src_port;
+ struct fm10k_cfg_port fw_port[2];
+ /* set by ffu rule add */
+ uint16_t rule_id;
+};
+
+#define FM10K_CONFIG_FLOWSET_NAME_MAX 256
+struct fm10k_cfg_flowset {
+ char name[FM10K_CONFIG_FLOWSET_NAME_MAX];
+ struct fm10k_cfg_flow flow_head;
+ struct fm10k_cfg_flowset *next;
+};
+
+/* Configuration */
+struct fm10k_dpdk_cfg {
+ uint8_t pf_num; /* configure by conf */
+ uint8_t pf_bind; /* initialize by dpdk */
+ uint8_t pf_max; /* set by card type */
+ uint8_t ext_port_num; /* configure by conf */
+ uint8_t ext_port_speed; /* configure by conf */
+ uint32_t debug_cfg; /* configure by conf */
+ uint32_t stats_interval;/* configure by conf */
+
+ struct fm10k_hw *master_hw; /* initialize by dpdk */
+ struct fm10k_hw *pf_hw[FM10K_SW_PEP_PORTS_MAX]; /* initialize by dpdk */
+
+ /* initialize by dpdk */
+ struct fm10k_dpdk_port ports[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf or default*/
+ struct fm10k_cfg_port_pf_map dpdk_port_map[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf */
+ struct fm10k_cfg_port_pf_map ext_port_map[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_cfg_config_item *config_list; /* configure by conf */
+ struct fm10k_cfg_flowset flowset_head; /* transfer from conf file */
+ struct fm10k_cfg_flowset *current;
+};
+
+static inline bool
+fm10k_config_check_debug(struct fm10k_dpdk_cfg *cfg, uint16_t dbg)
+{
+ return cfg->debug_cfg & 1 << (dbg - FM10K_CONFIG_DEBUG_START) &&
+ cfg->debug_cfg & 1;
+}
+
+struct fm10k_cfg_flowset *fm10k_config_flowset_get(const char *name);
+struct fm10k_cfg_flowset *fm10k_config_flowset_current_get(void);
+void fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new);
+void fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow);
+void fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow);
+bool fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow);
+
+void fm10k_config_cfg_flowset_show(void);
+struct fm10k_hw *fm10k_config_hw_get(int port_no);
+int fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw);
+
+#endif /* _FM10K_CONFIG_H */
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.c b/drivers/net/fm10k/switch/fm10k_ffu.c
new file mode 100644
index 0000000..a9b23aa
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.c
@@ -0,0 +1,1253 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+
+/*
+ * one SLICE for 40bits SEL
+ * SLICE 28 FOR SEL SGLORT(16bits) and VLAN(16bits)
+ * FOR ACT ROUTE
+ * SLICE 29 FOR SEL ETHER_TYPE(16bits)
+ * FOR ACT MIRROR | SET_VLAN
+ * SLICE 30 FOR SEL MPLS_LABEL0(32bits)
+ * SLICE 31 FOR SEL MPLS_LABEL1(32bits)
+ */
+#define FM10K_FFU_SLICE_START 28
+#define FM10K_FFU_SLICE_SGLORT_VID 28
+#define FM10K_FFU_SLICE_ETYPE_VID2 29
+#define FM10K_FFU_SLICE_MPLS_LABEL0 30
+#define FM10K_FFU_SLICE_MPLS_LABEL1 31
+
+#define FM10K_FFU_SLICE_SGLORT_VLAN_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_FFU_SLICE_CFG_START_ACTION | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_ETHER_TYPE_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL0_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL1_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+static uint64_t fm10k_ffu_slice_cfgs[4] = {
+ FM10K_FFU_SLICE_SGLORT_VLAN_CFG,
+ FM10K_FFU_SLICE_ETHER_TYPE_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL0_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL1_CFG
+};
+
+#define FM10K_FFU_INIT_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_INIT)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_RULE_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_RULE)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_CNT_INDEX(x) (FM10K_SW_FFU_CNT_START + (x))
+#define FM10K_FFU_FLOW_START 10
+#define FM10K_FFU_MIRROR_PROFILE 1
+#define FM10K_FFU_SGLORT_TYPE_NULL 0
+#define FM10K_FFU_SGLORT_TYPE_EPL 1
+#define FM10K_FFU_SGLORT_TYPE_PF 2
+#define FM10K_FFU_SGLORT_TYPE_PFS 3
+#define FM10K_FFU_SGLORT_TYPE_DPDK 4
+
+struct fm10k_ffu_rule_data {
+ uint16_t sglort_type;
+ uint16_t cond_sglort;
+ uint16_t cond_vlan;
+ uint16_t act_dglort;
+ uint16_t act_vlan;
+ uint16_t bypass_dglort;
+ uint16_t bypass_vlan;
+};
+
+
+static uint8_t fm10k_ffu_bitmap[FM10K_FFU_RULE_MAX / 8];
+static uint8_t
+fm10k_ffu_rule_get_bit(int id)
+{
+ int num = id / 8;
+ int offset = id % 8;
+
+ return (fm10k_ffu_bitmap[num] >> offset) & 0x1;
+}
+
+static void
+fm10k_ffu_rule_set_bit(int id, uint8_t bit)
+{
+ int num = id / 8;
+ int offset = id % 8;
+ uint8_t tmp = fm10k_ffu_bitmap[num];
+ uint8_t data = 1;
+
+ if (bit == 0)
+ fm10k_ffu_bitmap[num] = tmp & ~(data << offset);
+ else
+ fm10k_ffu_bitmap[num] |= (data << offset);
+}
+
+static int
+fm10k_ffu_rule_alloc(void)
+{
+ int i;
+
+ for (i = FM10K_FFU_FLOW_START; i < FM10K_FFU_RULE_MAX; i++) {
+ if (fm10k_ffu_rule_get_bit(i) != 0)
+ continue;
+ fm10k_ffu_rule_set_bit(i, 1);
+ return i;
+ }
+ return -1;
+}
+
+static void
+fm10k_ffu_rule_free(int id)
+{
+ fm10k_ffu_rule_set_bit(id, 0);
+}
+
+static void
+fm10k_ffu_always_mismatch(struct fm10k_switch *sw,
+ unsigned int slice, unsigned int idx)
+{
+ uint64_t temp64;
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~0ULL) |
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY_TOP, ~0ULL);
+ fm10k_write_switch_reg128(sw, FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64, temp64);
+}
+
+static void
+fm10k_ffu_route_dglort(struct fm10k_switch *sw, unsigned int slice,
+ unsigned int idx, uint16_t sglort, uint16_t dglort)
+{
+ uint64_t temp64, temp64_2;
+
+ /*
+ * Set the key to exact match on the 16 SGLORT bits and
+ * always match everywhere else.
+ */
+ temp64 = FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, sglort);
+ temp64_2 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~sglort & 0xffff);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64_2, temp64);
+
+ /*
+ * Set the corresponding SRAM entry to ROUTE_GLORT to the
+ * corresponding DGLORT.
+ */
+ temp64 = 0x40;
+ temp64 = temp64 << 32 |
+ FM10K_SW_MAKE_REG_FIELD(FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ /*
+ * 11.5.4.2 FFU_SLICE_SRAM[0..31][0..1023]
+ */
+ temp64_2 =
+ FM10K_SW_FFU_CNT_BANK << (35 - 23) |
+ (FM10K_SW_FFU_CNT_START + idx);
+ temp64 |= temp64_2 << 23;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(slice, idx), temp64);
+}
+
+
+static void
+fm10k_ffu_set_dest_glort_mask(struct fm10k_switch *sw,
+ unsigned int idx, uint16_t dglort, uint64_t dport_mask)
+{
+ uint64_t temp64;
+ unsigned int multiple_dports;
+ unsigned int num_dports;
+ unsigned int amplification_factor;
+ unsigned int hashed_entries;
+ unsigned int i, j;
+
+ multiple_dports = (dport_mask & (dport_mask - 1));
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to port ", __func__, dglort);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ if (multiple_dports) {
+ /*
+ * Create a pair of entries and use the hash value to select
+ * among them.
+ */
+ num_dports = 0;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i)) {
+ num_dports++;
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d", i);
+ }
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, "\n");
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ amplification_factor = 16 / num_dports;
+ hashed_entries = num_dports * amplification_factor;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_HASHED);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_COUNT,
+ hashed_entries);
+ fm10k_write_switch_reg64(sw, FM10K_SW_GLORT_RAM(idx), temp64);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ for (j = 0; j < amplification_factor; j++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ (1ULL << i)));
+ sw->glort_dest_table_idx++;
+ }
+ } else {
+ /*
+ * Set the corresponding entry in the DGLORT map RAM to use
+ * strict indexing straight into the DEST_TABLE, then write
+ * the corresponding destination port in the DEST_TABLE.
+ */
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ dport_mask));
+ sw->glort_dest_table_idx++;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d\n", i);
+ }
+}
+
+static struct fm10k_multi_glort {
+ uint8_t lport1;
+ uint8_t lport2;
+ uint16_t vlan1;
+ uint16_t vlan2;
+} fm10k_multi_glorts[FM10K_SW_FFU_RULE_MAX];
+
+static uint32_t
+fm10k_ffu_multi_glort_get(uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (lport1 == fm10k_multi_glorts[i].lport1 &&
+ lport2 == fm10k_multi_glorts[i].lport2 &&
+ vlan1 == fm10k_multi_glorts[i].vlan1 &&
+ vlan2 == fm10k_multi_glorts[i].vlan2) {
+ if (p_new != NULL)
+ *p_new = false;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (fm10k_multi_glorts[i].lport1 == 0 &&
+ fm10k_multi_glorts[i].lport2 == 0 &&
+ fm10k_multi_glorts[i].vlan1 == 0 &&
+ fm10k_multi_glorts[i].vlan2 == 0) {
+ fm10k_multi_glorts[i].lport1 = lport1;
+ fm10k_multi_glorts[i].lport2 = lport2;
+ fm10k_multi_glorts[i].vlan1 = vlan1;
+ fm10k_multi_glorts[i].vlan2 = vlan2;
+ if (p_new != NULL)
+ *p_new = true;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ return 0;
+}
+
+static uint32_t
+fm10k_ffu_set_dest_glort_multi_cast(struct fm10k_switch *sw,
+ uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2)
+{
+ uint32_t dglort;
+ bool is_new = false;
+ uint32_t idx;
+ uint64_t temp64;
+
+ dglort =
+ fm10k_ffu_multi_glort_get(lport1, lport2,
+ vlan1, vlan2, &is_new);
+
+ if (!is_new)
+ return dglort;
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to (%d:%d) (%d:%d)\n",
+ __func__, dglort, lport1, vlan1, lport2, vlan2);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ idx = sw->glort_cam_ram_idx++;
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ /* GLORT_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * DestMask 47:0 RW 0x0
+ * IP_MulticastIndex59:48 RW 0x0
+ * Reserved 63:60 RSV 0x0
+ */
+ temp64 = sw->mcast_dest_table_idx;
+ temp64 = temp64 << 48 | 1 << lport1 | 1 << lport2;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx), temp64);
+ sw->glort_dest_table_idx++;
+
+ /* MCAST_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * PortMask 47:0 RW 0x0
+ * LenTableIdx 61:48 RW 0x0
+ * Reserved 63:62 RSV 00b
+ */
+ temp64 = sw->mcast_len_table_idx;
+ temp64 = 1 << lport1 | 1 << lport2 | temp64 << 48;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE
+ (sw->mcast_dest_table_idx++), temp64);
+
+ /* MCAST_LEN_TABLE
+ * Field Name Bit(s) Type Default
+ * L3_McastIdx 14:0 RW 0x0
+ * L3_Repcnt 26:15 RW 0x0
+ * Reserved 31:27 RSV 0x0
+ */
+ temp64 =
+ sw->mcast_vlan_table_idx | 1 << 15;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE
+ (sw->mcast_len_table_idx++), temp64);
+
+ /* MCAST_VLAN_TABLE
+ * Field Name Bit(s) Type Default
+ * VID 11:0 RW 0x0
+ * DGLORT 27:12 RW 0x0
+ * ReplaceVID 28 RW 0b
+ * ReplaceDGLORT 29 RW 0b
+ * Reserved 31:30 RSV 00b
+ */
+ temp64 = vlan1 |
+ fm10k_switch_pf_glort_get
+ (lport1) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+ temp64 = vlan2 |
+ fm10k_switch_pf_glort_get
+ (lport2) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+
+ return dglort;
+}
+
+static uint64_t
+fm10k_data64_field64_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+
+ if (start == end) {
+ tmp64 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp64 = tmp64 >> (64 - end + start);
+ }
+ return tmp64;
+}
+
+static uint32_t
+fm10k_data64_field32_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+ uint32_t tmp32;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp32 = tmp64 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+static uint32_t
+fm10k_data32_field_get(uint32_t data, int start, int end)
+{
+ uint32_t tmp32 = data;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp32 = tmp32 << (64 - end);
+ tmp32 = tmp32 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+
+static void
+fm10k_glort_register_dump(struct fm10k_switch *sw)
+{
+ uint32_t i;
+ uint64_t data64;
+ uint32_t data32;
+ uint32_t tmp32;
+
+ if (!fm10k_config_check_debug
+ (sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("----- GLORT -----\n");
+
+ for (i = 0; i < sw->glort_cam_ram_idx; i++) {
+ data32 = fm10k_read_switch_reg(sw, FM10K_SW_GLORT_CAM(i));
+ FM10K_SW_INFO("[%02u]GLORT_CAM %#x Key %#x\n", i, data32,
+ fm10k_data32_field_get(data32, 0, 15));
+ data64 = fm10k_read_switch_reg64(sw, FM10K_SW_GLORT_RAM(i));
+ FM10K_SW_INFO(" GLORT_RAM %#llx Strict %u DestIndex %u\n",
+ (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 0, 1),
+ fm10k_data64_field32_get(data64, 2, 13));
+ tmp32 = fm10k_data64_field32_get(data64, 2, 13);
+ data64 =
+ fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" GLORT_DEST_TABLE[%u] %#llx "
+ "IP_MulticastIndex %u\n",
+ tmp32, (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 48, 59));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 59);
+ if (tmp32 == 0)
+ continue;
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_DEST_TABLE[%u] "
+ "%#llx PortMask %#llx"
+ " LenTableIdx %u\n", tmp32,
+ (unsigned long long)data64,
+ (unsigned long long)
+ fm10k_data64_field64_get(data64, 0, 47),
+ fm10k_data64_field32_get(data64, 48, 61));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 61);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_LEN_TABLE[%u] %#x "
+ "L3_McastIdx %u L3_Repcnt %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 14),
+ fm10k_data32_field_get(data32, 15, 26));
+ tmp32 = fm10k_data32_field_get(data32, 0, 14);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32 + 1));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32 + 1, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ }
+}
+
+static void
+fm10k_ffu_register_dump(struct fm10k_switch *sw)
+{
+ int i, j;
+ uint32_t data[8];
+ uint32_t ffu_valid;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("--------------- FFU REGISTERS DUMP -----------------\n");
+
+ fm10k_read_switch_array(sw, FM10K_SW_FFU_MASTER_VALID, data, 2);
+ FM10K_SW_INFO("FFU_MASTER_VALID: %#x %#x\n", data[0], data[1]);
+ ffu_valid = data[0];
+
+ for (i = 0; i < 32; i++) {
+ if ((ffu_valid & (1 << i)) == 0)
+ continue;
+
+ FM10K_SW_INFO("------ SLICE%d ------\n", i);
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_VALID(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_VALID[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CASCADE_ACTION[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ for (j = 0; j < 1; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CFG(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CFG[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ for (j = 0; j < 32; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_TCAM(i, j), data, 4);
+ if ((data[0] != 0 || data[1] != 0 ||
+ data[2] != 0 || data[3] != 0))
+ FM10K_SW_INFO("FFU_SLICE_TCAM[%d][%d]: "
+ "%#x %#x %#x %#x\n",
+ i, j, data[0], data[1],
+ data[2], data[3]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_SRAM[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ }
+}
+
+
+static void
+fm10k_ffu_mirror_set_action(struct fm10k_switch *sw,
+ uint16_t ffu_slice, uint16_t table_idx)
+{
+ uint64_t data64;
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice, table_idx), 0xffffffffff, 0x1);
+ /* SET FFU RX_MIRROR */
+ data64 = 0x60; /* higher than route, not necessary */
+ data64 = data64 << 32 | 0x2 << 21 | 1 << (8 + 4) | 1 << 4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), data64);
+}
+
+static void
+fm10k_ffu_mirror_set_forward(struct fm10k_switch *sw,
+ int profile,
+ uint16_t vlan,
+ uint16_t dest_lport,
+ uint32_t dest_sglort)
+{
+ uint64_t data64;
+
+ /* RX_MIRROR_CFG */
+ fm10k_write_switch_reg(sw, 0xd50000 + 0xd, profile);
+
+ /* FH_MIRROR_PROFILE_TABLE */
+ fm10k_write_switch_reg(sw,
+ 0xd50000 + 0x1 * profile + 0x40, dest_lport);
+
+ /* MOD_MIRROR_PROFILE_TABLE don't work */
+ data64 = vlan;
+ data64 = (dest_sglort & 0xffff) | data64 << 17;
+ fm10k_write_switch_reg64(sw,
+ 0xe80000 + 0x2 * profile + 0x24000, data64);
+}
+
+
+int
+fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_ext_port,
+ uint16_t dest_ext_port,
+ uint16_t vlan)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+ uint16_t mirror_profile = FM10K_FFU_MIRROR_PROFILE;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_ffu_mirror_set_action(sw, ffu_slice, table_idx);
+
+ fm10k_ffu_mirror_set_forward(sw, mirror_profile,
+ vlan, sw->epl_map[dest_ext_port - 1].logical_port,
+ sw->epl_map[dest_ext_port - 1].glort);
+ return 0;
+}
+
+int
+fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_ext_port)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), 0);
+ return 0;
+}
+
+static void
+fm10k_ffu_logical_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t lport, uint16_t vlan_id)
+{
+ uint64_t data;
+
+ /* 11.21.3.9 INGRESS_VID_TABLE[0..4095] */
+ data = fm10k_read_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000);
+ data |= 1 << lport;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000, data);
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "%s lport:%d, vlan_id:%d, reg:%#llx\n",
+ __func__, lport, vlan_id, (unsigned long long)data);
+}
+
+static void
+fm10k_ffu_glort_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t glort, uint16_t vlan_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ if (sw->pep_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->pep_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->epl_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+}
+
+
+static uint16_t
+fm10k_ffu_dpdk_port_glort_get(struct fm10k_switch *sw, int port)
+{
+ int pf;
+ uint16_t glort = 0;
+
+ if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFS ||
+ sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ glort = fm10k_switch_pfs_glort_get
+ (sw->dpdk_cfg->dpdk_port_map[port].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[port].map_no[1]);
+ } else if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf = sw->dpdk_cfg->dpdk_port_map[port].map_no[0];
+ glort = fm10k_switch_pf_glort_get(pf);
+ }
+ return glort;
+}
+
+static void
+fm10k_ffu_rule_enable_single_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint16_t dglort,
+ uint16_t dvlan)
+{
+ uint64_t temp64;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (dvlan) {
+ /* Force updating VLAN tag if present.
+ * Add if absent. 11.5.3.4
+ */
+ temp64 = 3;
+ sram[sram_idx] =
+ temp64 << 16 | dvlan |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 2) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET);
+ fm10k_ffu_glort_port_vlan_set(sw, dglort, dvlan);
+ sram_idx++;
+ }
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+static void
+fm10k_ffu_rule_enable_multi_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint8_t lport1,
+ uint8_t lport2,
+ uint16_t vlan1,
+ uint16_t vlan2)
+{
+ uint64_t temp64;
+ uint32_t dglort;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ dglort =
+ fm10k_ffu_set_dest_glort_multi_cast(sw,
+ lport1, lport2, vlan1, vlan2);
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ fm10k_ffu_logical_port_vlan_set(sw, lport1, vlan1);
+ fm10k_ffu_logical_port_vlan_set(sw, lport2, vlan2);
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+
+void
+fm10k_ffu_flow_enable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ uint16_t sglort = 0;
+ uint16_t svlan = 0;
+ uint16_t dglort = 0;
+ uint16_t dvlan = 0;
+ uint16_t lport1 = 0, lport2 = 0;
+ uint16_t dvlan2 = 0;
+
+ switch (flow->src_port.port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ sglort = fm10k_switch_epl_glort_get(flow->src_port.port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ sglort =
+ fm10k_ffu_dpdk_port_glort_get(sw,
+ flow->src_port.port_no);
+ break;
+ }
+ switch (flow->fw_port[0].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[0].port_no - 1);
+ lport1 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[0].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[0].port_no);
+ lport1 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[0].port_no);
+ break;
+ }
+ svlan = flow->src_port.vlan_id;
+ dvlan = flow->fw_port[0].vlan_id;
+
+ flow->rule_id = fm10k_ffu_rule_alloc();
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Set rule cond sglort:%#x, vlan:%d ==> "
+ "act dglort:%#x, vlan:%d",
+ sglort, svlan, dglort, dvlan);
+
+ if (flow->fw_port[1].port_type) {
+ switch (flow->fw_port[1].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[1].port_no - 1);
+ lport2 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[1].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[1].port_no);
+ lport2 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[1].port_no);
+ break;
+ }
+ dvlan2 = flow->fw_port[1].vlan_id;
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ " (bypass glort:%#x, vlan:%d)\n",
+ dglort, dvlan2);
+
+ fm10k_ffu_rule_enable_multi_cast(sw,
+ flow->rule_id, sglort, svlan,
+ lport1, lport2, dvlan, dvlan2);
+ } else {
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg, "\n");
+ fm10k_ffu_rule_enable_single_cast(sw,
+ flow->rule_id, sglort, svlan, dglort, dvlan);
+ }
+}
+
+
+void
+fm10k_ffu_flow_disable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ int i;
+
+ if (flow->rule_id == 0)
+ return;
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Remove flow %d rule %d\n",
+ flow->flow_no, flow->rule_id);
+
+ for (i = FM10K_FFU_SLICE_START; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_ffu_always_mismatch(sw, i, flow->rule_id);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, flow->rule_id), 0);
+ }
+ fm10k_ffu_rule_free(flow->rule_id);
+}
+
+
+static int
+fm10k_ffu_configured_flowset_enable(struct fm10k_switch *sw)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flow *flow;
+
+ flowset = fm10k_config_flowset_current_get();
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+
+ return 0;
+}
+
+
+void
+fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flowset *cur_fs;
+ struct fm10k_cfg_flow *flow;
+
+ cur_fs = fm10k_config_flowset_current_get();
+ if (strcmp(cur_fs->name, new_name) == 0)
+ return;
+
+ flowset = fm10k_config_flowset_get(new_name);
+ if (flowset == NULL) {
+ FM10K_SW_ERR("Can not find flowset %s!!\n", new_name);
+ return;
+ }
+
+ /* disable current flowset */
+ flow = cur_fs->flow_head.next;
+ while (!fm10k_config_flow_list_end(&cur_fs->flow_head, flow)) {
+ fm10k_ffu_flow_disable(sw, flow);
+ flow = flow->next;
+ }
+
+ /* enable new flowset */
+ fm10k_config_flowset_current_set(flowset);
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+}
+
+
+int
+fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg)
+{
+ int ret = 0;
+ uint64_t data64;
+ uint16_t i, j;
+ uint32_t sglort = 0, dglort = 0;
+ uint16_t table_idx = 0;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_START;
+
+ sw->mcast_dest_table_idx = 1;
+ sw->mcast_len_table_idx = 1;
+ sw->mcast_vlan_table_idx = 1;
+
+ for (i = 0;
+ i < sizeof(fm10k_ffu_slice_cfgs) / sizeof(uint64_t);
+ i++) {
+ for (j = 0; j < FM10K_SW_FFU_NUM_SCENARIOS; j++) {
+ data64 = fm10k_ffu_slice_cfgs[i];
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CFG
+ (FM10K_FFU_SLICE_START + i, j),
+ data64);
+ }
+ FM10K_FFU_INIT_PRINT(cfg, "SET slice %d cfg = %#llx\n",
+ FM10K_FFU_SLICE_START + i,
+ (unsigned long long)data64);
+ }
+
+ for (table_idx = 0;
+ table_idx < FM10K_SW_FFU_SLICE_TCAM_ENTRIES / 2;
+ table_idx++)
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ fm10k_ffu_always_mismatch(sw, i, table_idx);
+
+ /*
+ * Create a TCAM entry to match each SGLORT that might be used, and
+ * set the corresponding SRAM action entries to ROUTE_GLORT to the
+ * corresponding DGLORT (see above table).
+ * SGLORT ROUTE is always the first slice
+ */
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pf_glort_get
+ (cfg->ext_port_map[i].map_no[0]);
+ } else if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ }
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, sglort, dglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ table_idx = FM10K_FFU_EXT_PORT_RULE_EGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, dglort, sglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ }
+
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i),
+ 0xf0f000f);
+ /* Set slice 0 to valid for all scenarios */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_VALID(i),
+ FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS);
+ }
+
+ /* Mark slice 0 as valid and all chunks as invalid */
+ data64 = 0;
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ data64 |= FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(i);
+ fm10k_write_switch_reg64(sw, FM10K_SW_FFU_MASTER_VALID, data64);
+
+ /*
+ * Set up the DGLORT map according to the desired
+ * SGLORT -> { DGLORT, logical_port } map.
+ */
+ for (i = 0; i < sw->info->num_peps; i++) {
+ if (sw->pep_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->pep_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->epl_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->epl_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ uint64_t dport_mask = 0;
+
+ if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ dport_mask =
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[0]].logical_port) |
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[1]].logical_port);
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++, dglort, dport_mask);
+ }
+ }
+
+ /* Ensure the rest of the DGLORT TCAM won't match */
+ for (table_idx = sw->glort_cam_ram_idx;
+ table_idx < FM10K_SW_GLORT_CAM_ENTRIES;
+ table_idx++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_GLORT_CAM(table_idx),
+ FM10K_SW_GLORT_CAM_MATCH_NONE);
+
+ ret = fm10k_ffu_configured_flowset_enable(sw);
+
+ fm10k_ffu_register_dump(sw);
+ fm10k_glort_register_dump(sw);
+ return ret;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.h b/drivers/net/fm10k/switch/fm10k_ffu.h
new file mode 100644
index 0000000..bcd9329
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_FFU_H_
+#define _FM10K_FFU_H_
+
+
+#include <stdint.h>
+
+#define FM10K_FFU_EXT_PORT_RULE_INGRESS(i) ((i) * 2)
+#define FM10K_FFU_EXT_PORT_RULE_EGRESS(i) ((i) * 2 + 1)
+#define FM10K_FFU_RULE_MAX FM10K_SW_FFU_RULE_MAX
+
+struct fm10k_switch;
+struct fm10k_dpdk_cfg;
+struct fm10k_cfg_flow;
+
+int fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg);
+
+int fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_dpdk_port, u16 dest_dpdk_port, uint16_t vlan);
+int fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_dpdk_port);
+
+void fm10k_ffu_flow_enable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flow_disable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name);
+
+#endif /* _FM10K_FFU_H */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
new file mode 100644
index 0000000..563f9ac
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -0,0 +1,1114 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_stats.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+
+
+#define FM10K_MAX_VLAN_COUNTER 63
+#define FM10K_NB_RX_STATS_BANKS 6
+#define FM10K_BINS_PER_RX_STATS_BANK 16
+#define FM10K_WORDS_PER_RX_STATS_COUNTER 4
+
+/* Max expected entries in read stats scatter gather list */
+#define MAX_STATS_SGLIST 128
+#define FM10K_RX_STATS_BASE (0xE00000)
+#define FM10K_MOD_BASE (0xE80000)
+#define FM10K_CM_APPLY_BASE (0xD40000)
+#define FM10K_EPL_BASE (0x0E0000)
+
+#define FM10K_RX_STATS_BANK(index1, index0, word) \
+ ((0x001000) * ((index1) - 0) + \
+ (0x000004) * ((index0) - 0) + \
+ (word) + (0x000000) + \
+ (FM10K_RX_STATS_BASE))
+#define FM10K_MOD_STATS_BANK_FRAME(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x025000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_MOD_STATS_BANK_BYTE(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x026000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_CM_APPLY_DROP_COUNT(index, word) \
+ ((0x000002) * ((index) - 0) + \
+ (word) + (0x000880) + \
+ (FM10K_CM_APPLY_BASE))
+#define FM10K_MAC_OVERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000021) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_JABBER_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000022) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000023) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_RUNT_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000024) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_OVERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000025) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000026) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_CODE_ERROR_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000027) + \
+ (FM10K_EPL_BASE))
+
+/* Rx Bank Definitions */
+#define FM10K_RX_STAT_BANK_TYPE 0
+#define FM10K_RX_STAT_BANK_SIZE 1
+#define FM10K_RX_STAT_BANK_PRI 2
+#define FM10K_RX_STAT_BANK_FWD_1 3
+#define FM10K_RX_STAT_BANK_FWD_2 4
+#define FM10K_RX_STAT_BANK_VLAN 5
+
+/* FM10K_RX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_RX_STAT_NONIP_L2UCAST 0
+#define FM10K_RX_STAT_NONIP_L2MCAST 1
+#define FM10K_RX_STAT_NONIP_L2BCAST 2
+#define FM10K_RX_STAT_IPV4_L2UCAST 3
+#define FM10K_RX_STAT_IPV4_L2MCAST 4
+#define FM10K_RX_STAT_IPV4_L2BCAST 5
+#define FM10K_RX_STAT_IPV6_L2UCAST 6
+#define FM10K_RX_STAT_IPV6_L2MCAST 7
+#define FM10K_RX_STAT_IPV6_L2BCAST 8
+#define FM10K_RX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_RX_STAT_CLASS_BASED_PAUSE 10
+#define FM10K_RX_STAT_FRAMING_ERR 11
+#define FM10K_RX_STAT_FCS_ERR 12
+
+/* FM10K_RX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_RX_STAT_LEN_LT_64 0
+#define FM10K_RX_STAT_LEN_EQ_64 1
+#define FM10K_RX_STAT_LEN_65_127 2
+#define FM10K_RX_STAT_LEN_128_255 3
+#define FM10K_RX_STAT_LEN_256_511 4
+#define FM10K_RX_STAT_LEN_512_1023 5
+#define FM10K_RX_STAT_LEN_1024_1522 6
+#define FM10K_RX_STAT_LEN_1523_2047 7
+#define FM10K_RX_STAT_LEN_2048_4095 8
+#define FM10K_RX_STAT_LEN_4096_8191 9
+#define FM10K_RX_STAT_LEN_8192_10239 10
+#define FM10K_RX_STAT_LEN_GE_10240 11
+
+/* FM10K_RX_STAT_BANK_PRI Bin Definitions */
+#define FM10K_RX_STAT_PRI_0 0
+#define FM10K_RX_STAT_PRI_1 1
+#define FM10K_RX_STAT_PRI_2 2
+#define FM10K_RX_STAT_PRI_3 3
+#define FM10K_RX_STAT_PRI_4 4
+#define FM10K_RX_STAT_PRI_5 5
+#define FM10K_RX_STAT_PRI_6 6
+#define FM10K_RX_STAT_PRI_7 7
+#define FM10K_RX_STAT_PRI_8 8
+#define FM10K_RX_STAT_PRI_9 9
+#define FM10K_RX_STAT_PRI_10 10
+#define FM10K_RX_STAT_PRI_11 11
+#define FM10K_RX_STAT_PRI_12 12
+#define FM10K_RX_STAT_PRI_13 13
+#define FM10K_RX_STAT_PRI_14 14
+#define FM10K_RX_STAT_PRI_15 15
+
+/* FM10K_RX_STAT_BANK_FWD_1 Bin Definitions */
+#define FM10K_RX_STAT_FID_FORWARDED 0
+#define FM10K_RX_STAT_FLOOD_FORWARDED 1
+#define FM10K_RX_STAT_SPECIALLY_HANDLED 2
+#define FM10K_RX_STAT_PARSER_ERROR_DROP 3
+#define FM10K_RX_STAT_ECC_ERROR_DROP 4
+#define FM10K_RX_STAT_TRAPPED 5
+#define FM10K_RX_STAT_PAUSE_DROPS 6
+#define FM10K_RX_STAT_STP_DROPS 7
+#define FM10K_RX_STAT_SECURITY_VIOLATIONS 8
+#define FM10K_RX_STAT_VLAN_TAG_DROPS 9
+#define FM10K_RX_STAT_VLAN_INGRESS_DROPS 10
+#define FM10K_RX_STAT_VLAN_EGRESS_DROPS 11
+#define FM10K_RX_STAT_GLORT_MISS_DROPS 12
+#define FM10K_RX_STAT_FFU_DROPS 13
+#define FM10K_RX_STAT_TRIGGER_DROPS 14
+#define FM10K_RX_STAT_OVERFLOW4 15
+
+/* FM10K_RX_STAT_BANK_FWD_2 Bin Definitions */
+#define FM10K_RX_STAT_POLICER_DROPS 0
+#define FM10K_RX_STAT_TTL_DROPS 1
+#define FM10K_RX_STAT_CM_PRIV_DROPS 2
+#define FM10K_RX_STAT_CM_SMP0_DROPS 3
+#define FM10K_RX_STAT_CM_SMP1_DROPS 4
+#define FM10K_RX_STAT_CM_RX_HOG_0_DROPS 5
+#define FM10K_RX_STAT_CM_RX_HOG_1_DROPS 6
+#define FM10K_RX_STAT_CM_TX_HOG_0_DROPS 7
+#define FM10K_RX_STAT_CM_TX_HOG_1_DROPS 8
+/* Bin 9 reserved */
+#define FM10K_RX_STAT_TRIGGER_REDIRECTS 10
+#define FM10K_RX_STAT_FLOOD_CONTROL_DROPS 11
+#define FM10K_RX_STAT_GLORT_FORWARDED 12
+#define FM10K_RX_STAT_LOOPBACK_SUPPRESS 13
+#define FM10K_RX_STAT_OTHERS 14
+#define FM10K_RX_STAT_OVERFLOW5 15
+
+/* FM10K_RX_STAT_BANK_VLAN Bin definitions */
+#define FM10K_RX_STAT_VLAN_UCAST 0
+#define FM10K_RX_STAT_VLAN_MCAST 1
+#define FM10K_RX_STAT_VLAN_BCAST 2
+
+/* Tx Bank Definitions */
+#define FM10K_TX_STAT_BANK_TYPE 0
+#define FM10K_TX_STAT_BANK_SIZE 1
+
+/* FM10K_TX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_TX_STAT_L2UCAST 0
+#define FM10K_TX_STAT_L2MCAST 1
+#define FM10K_TX_STAT_L2BCAST 2
+#define FM10K_TX_STAT_ERR_SENT 3
+#define FM10K_TX_STAT_TIMEOUT_DROP 4
+#define FM10K_TX_STAT_ERR_DROP 5
+#define FM10K_TX_STAT_ECC_DROP 6
+#define FM10K_TX_STAT_LOOPBACK_DROP 7
+#define FM10K_TX_STAT_TTL1_DROP 8
+#define FM10K_TX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_TX_STAT_CLASS_BASED_PAUSE 10
+
+/* FM10K_TX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_TX_STAT_LEN_LT_64 0
+#define FM10K_TX_STAT_LEN_EQ_64 1
+#define FM10K_TX_STAT_LEN_65_127 2
+#define FM10K_TX_STAT_LEN_128_255 3
+#define FM10K_TX_STAT_LEN_256_511 4
+#define FM10K_TX_STAT_LEN_512_1023 5
+#define FM10K_TX_STAT_LEN_1024_1522 6
+#define FM10K_TX_STAT_LEN_1523_2047 7
+#define FM10K_TX_STAT_LEN_2048_4095 8
+#define FM10K_TX_STAT_LEN_4096_8191 9
+#define FM10K_TX_STAT_LEN_8192_10239 10
+#define FM10K_TX_STAT_LEN_GE_10240 11
+
+
+/* This structure is used to build a table of all rx / tx counters */
+struct fm10k_port_count_mapping {
+ /* Bank in which the counter is located */
+ uint32_t bank;
+
+ /* Bin in which the counter is located */
+ uint32_t bin;
+
+ /*
+ * Offset (in bytes) of a the frame counter in
+ * the fm10k_counters structure
+ */
+ uint32_t frame_offset;
+
+ /*
+ * Offset (in bytes) of a the byte counter in
+ * the fm10k_counters structure
+ */
+ uint32_t byte_offset;
+};
+
+
+/*
+ * Table of all RX port counters in the RX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping rx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FRAMING_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FCS_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors),
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors_octets)},
+
+ /* Size Bank */
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_64_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_octets)},
+
+ /* priority Bank */
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_0,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[0]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[0])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_1,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[1]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[1])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_2,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[2]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[2])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_3,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[3]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[3])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_4,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[4]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[4])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_5,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[5]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[5])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_6,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[6]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[6])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_7,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[7]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[7])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_8,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[8]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[8])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_9,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[9]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[9])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_10,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[10]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[10])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_11,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[11]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[11])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_12,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[12]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[12])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_13,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[13]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[13])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_14,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[14]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[14])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_15,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[15]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[15])},
+
+ /* Forwarding Bank */
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FID_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FLOOD_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SPECIALLY_HANDLED,
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_pkts),
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PARSER_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_ECC_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parity_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parity_error_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRAPPED,
+ offsetof(struct fm10k_port_counters, cnt_trapped_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trapped_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PAUSE_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_STP_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SECURITY_VIOLATIONS,
+ offsetof(struct fm10k_port_counters, cnt_security_violation_pkts),
+ offsetof(struct fm10k_port_counters, cnt_security_violation_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_TAG_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_INGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_EGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_GLORT_MISS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FFU_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRIGGER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_POLICER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TTL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_PRIV_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TRIGGER_REDIRECTS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_FLOOD_CONTROL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_GLORT_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_LOOPBACK_SUPPRESS,
+ offsetof(struct fm10k_port_counters, cnt_loopback_drops_pkts),
+ offsetof(struct fm10k_port_counters, cnt_loopback_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_OTHERS,
+ offsetof(struct fm10k_port_counters, cnt_other_pkts),
+ offsetof(struct fm10k_port_counters, cnt_other_octets)},
+
+}; /* end rxPortCntMapTable */
+
+
+/*
+ * Table of all TX port counters in the TX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping tx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * -------------------------- ------------------------------
+ * frame_offset
+ * ---------------------------------------------
+ * byte_offset
+ * ---------------------------------------------------
+ */
+ /* Type Bank */
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_SENT,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TIMEOUT_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ECC_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_LOOPBACK_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TTL1_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_octets)},
+
+ /* Size Bank */
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_64_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_octets)},
+
+}; /* end tx_port_cnt_map_table */
+
+
+/*
+ * Get a 32bit EPL counter by adding it to the scatter gather
+ * list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+static inline void
+fm10k_get_epl_port_stat32(struct fm10k_scatter_gather_entry *sg_list,
+ struct fm10k_port_counters *counters,
+ int *sg_list_cnt, int epl, int lane)
+{
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_oversized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_JABBER_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_jabber_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_UNDERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_undersized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_RUNT_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_fragment_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERRUN_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_over_run_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_CODE_ERROR_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_code_errors;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+}
+
+
+static void
+fm10k_read_scatter_gather(struct fm10k_switch *sw,
+ int n_entries,
+ struct fm10k_scatter_gather_entry *sg_list)
+{
+ int i;
+
+ for (i = 0; i < n_entries; i++) {
+ uint32_t addr = sg_list[i].addr;
+ uint32_t count = sg_list[i].count;
+ uint32_t *data = sg_list[i].data;
+
+ fm10k_read_switch_array(sw, addr, data, count);
+ }
+}
+
+
+static int
+fm10k_get_port_counters(struct fm10k_switch *sw,
+ struct fm10k_ext_port *ext_port,
+ struct fm10k_port_counters *counters)
+{
+ int phys_port;
+ int epl;
+ int lane;
+ struct timeval ts;
+ bool valid_ip_stats = true;
+ uint32_t i;
+ struct fm10k_scatter_gather_entry sg_list[MAX_STATS_SGLIST];
+ int sg_list_cnt = 0;
+ /* Temporary bin array to retrieve 128bit
+ * port counters (frame + bytes).
+ */
+ uint32_t cnt_rx_port_stats_bank
+ [FM10K_NB_RX_STATS_BANKS]
+ [FM10K_BINS_PER_RX_STATS_BANK]
+ [FM10K_WORDS_PER_RX_STATS_COUNTER];
+ /*
+ * Fill the counters structure with 0 to assure
+ * all fields not explicitly set below, which will
+ * be the fields not supported by the FM10000,
+ * will be 0 on return.
+ */
+ memset(counters, 0, sizeof(*counters));
+
+ phys_port = ext_port->portno;
+ epl = ext_port->eplno;
+ lane = ext_port->first_lane;
+
+ counters->cnt_version = FM10K_STATS_VERSION;
+
+ /*
+ * Reading counters for each RX bank
+ *
+ * Because the RX_STATS_BANK register contains
+ * both, frame count and byte count in a 128bit
+ * register, we first store the data in a temporary
+ * array.
+ * 1. Fill Scatter Gather to retrieve each bin
+ * counter from the rx bank and store in a temp
+ * array.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 128bit read of an rx counter to the scatter gather
+ * list.
+ * NOTE: Read values will be stored in the temporary
+ * array switchExt->cnt_rx_port_stats_bank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [out] - Array to store the
+ * 128bit values sgList
+ * [out] - scatter gather array to hold the reads sgListCnt
+ * [out] - number of entry in the sgList
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_RX_STATS_BANK(bank, (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data = cnt_rx_port_stats_bank[bank][bin];
+ sg_list[sg_list_cnt].count = 4;
+ sg_list_cnt++;
+}
+
+ /*
+ * 2. Fill in the scatter gather for tx bank
+ * counters. They will directly be stored
+ * in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 64bit read of a tx counter to the scatter gather list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+ uint32_t bank = tx_port_cnt_map_table[i].bank;
+ uint32_t bin = tx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = tx_port_cnt_map_table[i].frame_offset;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_MOD_STATS_BANK_FRAME(bank,
+ (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)(((uint8_t *)counters) + (frame_offset));
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ }
+
+ /*
+ * 3. Fill in the scatter gather for TX CM DROP
+ * and for EPL counters. They will directly be
+ * stored in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ sg_list[sg_list_cnt].addr =
+ FM10K_CM_APPLY_DROP_COUNT(phys_port, 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_cm_drop_pkts;
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ /* EPL Counters */
+ fm10k_get_epl_port_stat32(sg_list, counters, &sg_list_cnt, epl, lane);
+
+ /*
+ * 4. Execute scatter gather read.
+ */
+ if (sg_list_cnt >= MAX_STATS_SGLIST) {
+ /*
+ * Pretty static. Mainly to warn if something new added,
+ * but the array size is not adjust accordingly
+ */
+ FM10K_SW_ERR("Scatter list array %d for port %d overflow.\n",
+ sg_list_cnt, phys_port);
+ return -1;
+ }
+
+ /*
+ * Taking lock to protect temporary structures used to
+ * store 128b counters
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ /* now get the stats in one shot, optimized for fibm */
+ fm10k_read_scatter_gather(sw, sg_list_cnt, sg_list);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ counters->timestamp = ts.tv_sec * 1000000 + ts.tv_usec;
+
+ /*
+ * 5. Retrieve the frame/byte counts from 128bit
+ * registers stored in temporary array and set
+ * the proper fm_portCounter structure members.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Update the counter variable related to a 128bit HW counter
+ * NOTE: Read values will be retrieved from the temporary
+ * array switchExt->cnt_rx_PortStatsBank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [in] - Array to read the
+ * 128bit values from
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = rx_port_cnt_map_table[i].frame_offset;
+ uint32_t byte_offset = rx_port_cnt_map_table[i].byte_offset;
+
+ *((uint64_t *)(((uint8_t *)counters) + frame_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][1]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][0])));
+
+ *((uint64_t *)(((uint8_t *)counters) + byte_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][3]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][2])));
+ }
+
+ /*
+ * 6. Set some counters that are not available
+ * in HW but can be computed from two or more
+ * HW counters.
+ */
+ /*
+ * If IP parsing is enabled then the IP stats will be read and will be
+ * valid. Else all traffic, whether IP or not, is counted as _nonip.
+ */
+ if (valid_ip_stats) {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip +
+ counters->cnt_rx_ucst_pkts_ipv4 +
+ counters->cnt_rx_ucst_pkts_ipv6;
+
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip +
+ counters->cnt_rx_mcst_pkts_ipv4 +
+ counters->cnt_rx_mcst_pkts_ipv6;
+
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip +
+ counters->cnt_rx_bcst_pkts_ipv4 +
+ counters->cnt_rx_bcst_pkts_ipv6;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_octets_ipv4 =
+ counters->cnt_rx_ucst_octets_ipv4 +
+ counters->cnt_rx_mcst_octets_ipv4 +
+ counters->cnt_rx_bcst_octets_ipv4;
+
+ counters->cnt_rx_octets_ipv6 =
+ counters->cnt_rx_ucst_octets_ipv6 +
+ counters->cnt_rx_mcst_octets_ipv6 +
+ counters->cnt_rx_bcst_octets_ipv6;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip +
+ counters->cnt_rx_octets_ipv4 +
+ counters->cnt_rx_octets_ipv6;
+ } else {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip;
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip;
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip;
+ }
+
+ counters->cnt_trigger_drop_redir_pkts =
+ counters->cnt_trigger_redir_pkts +
+ counters->cnt_trigger_drop_pkts;
+
+ /* Emulate Tx _octets counter using the sum of all size bins. */
+ counters->cnt_tx_octets += counters->cnt_tx_minto63_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_64_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_65to127_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_128to255_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_256to511_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_512to1023_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1024to1522_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1523to2047_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_2048to4095_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_4096to8191_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_8192to10239_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_10240tomax_octets;
+
+ return 0;
+} /* end fm10k_get_port_counters */
+
+
+struct fm10k_stats_data {
+ uint64_t rx_pkts;
+ uint64_t tx_pkts;
+ uint64_t rx_drop;
+};
+
+
+static uint64_t
+fm10k_stats_ffu_count_get(struct fm10k_switch *sw, int table_id)
+{
+ uint64_t data;
+
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x4000 * FM10K_SW_FFU_CNT_BANK +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x8000);
+ } else {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x800 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x10000);
+ }
+ return data;
+}
+
+static void
+fm10k_stats_epl_ffu_print(struct fm10k_switch *sw, int epl)
+{
+ int table_id;
+ uint64_t data;
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_INGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] ingress %-12llu\n",
+ table_id, (unsigned long long)(data));
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_EGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] egress %-12llu\n",
+ table_id, (unsigned long long)(data));
+}
+
+#define FM10K_STATS_RULE_NUM_MAX 100
+static uint16_t fm10k_stats_rule_list[FM10K_STATS_RULE_NUM_MAX];
+void
+fm10k_stats_rule_count_reg(uint16_t rule_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0) {
+ fm10k_stats_rule_list[i] = rule_id;
+ break;
+ }
+ }
+}
+
+void
+fm10k_stats_ffu_count_print(struct fm10k_switch *sw)
+{
+ int i, rule_id;
+ uint64_t data;
+ static uint64_t ffu_count[FM10K_STATS_RULE_NUM_MAX];
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0)
+ continue;
+
+ rule_id = fm10k_stats_rule_list[i];
+ data = fm10k_stats_ffu_count_get(sw, rule_id);
+ FM10K_SW_INFO("FFU[%d] count %-12llu\n",
+ rule_id,
+ (unsigned long long)
+ (data - ffu_count[rule_id]));
+ ffu_count[rule_id] = data;
+ }
+}
+
+void
+fm10k_stats_epl_port_print(struct fm10k_switch *sw)
+{
+ int i, lport;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_stats_data data;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ lport = sw->epl_map[i].logical_port;
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ data.rx_pkts =
+ counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts =
+ counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ FM10K_SW_INFO("EPL port %-2d lport %-5d tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, lport,
+ (unsigned long long)
+ (data.tx_pkts - last_data[i].tx_pkts),
+ (unsigned long long)
+ (data.rx_pkts - last_data[i].rx_pkts),
+ (unsigned long long)
+ (data.rx_drop - last_data[i].rx_drop));
+ last_data[i] = data;
+
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU))
+ fm10k_stats_epl_ffu_print(sw, i);
+ }
+}
+
+
+static void
+fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
+{
+ unsigned int i;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ uint64_t *pdata;
+
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ rx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d rx bank %d idx %d pkt %llu\n",
+ lport,
+ rx_port_cnt_map_table[i].bank,
+ rx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ tx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d tx bank %d idx %d pkt %llu\n",
+ lport,
+ tx_port_cnt_map_table[i].bank,
+ tx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+}
+
+void
+fm10k_stats_port_bank_print(struct fm10k_switch *sw)
+{
+ int i;
+
+ for (i = 0;
+ i < FM10K_SW_EPLS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->epl_map[i].logical_port);
+ }
+ for (i = 0;
+ i < FM10K_SW_PEPS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->pep_map[i].logical_port);
+ }
+}
+
+
+void *
+fm10k_switch_process_stats(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ usec_delay(5000000);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_ENABLE))
+ return NULL;
+
+ if (!sw->dpdk_cfg->stats_interval)
+ return NULL;
+
+ while (sw->detaching == 0) {
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_PORT)) {
+ FM10K_SW_INFO("--- port statistic ---\n");
+ fm10k_stats_epl_port_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU)) {
+ FM10K_SW_INFO("--- ffu statistic ---\n");
+ fm10k_stats_ffu_count_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_MORE)) {
+ FM10K_SW_INFO("--- detail statistic ---\n");
+ fm10k_stats_port_bank_print(sw);
+ }
+ usec_delay(1000000 * sw->dpdk_cfg->stats_interval);
+ }
+ return NULL;
+}
+
diff --git a/drivers/net/fm10k/switch/fm10k_stats.h b/drivers/net/fm10k/switch/fm10k_stats.h
new file mode 100644
index 0000000..0bc84e4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.h
@@ -0,0 +1,257 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#ifndef _fm10k_stats_h_
+#define _fm10k_stats_h_
+
+
+#include <stdint.h>
+
+
+#define FM10K_STATS_VERSION (1 << 4)
+
+
+struct fm10k_scatter_gather_entry {
+ uint32_t addr;
+ uint32_t count;
+ uint32_t *data;
+
+};
+
+
+struct fm10k_port_counters {
+ uint64_t cnt_version;
+ uint64_t cnt_rx_ucst_pkts;
+ uint64_t cnt_rx_ucst_pkts_nonip;
+ uint64_t cnt_rx_ucst_pkts_ipv4;
+ uint64_t cnt_rx_ucst_pkts_ipv6;
+ uint64_t cnt_rx_bcst_pkts;
+ uint64_t cnt_rx_bcst_pkts_nonip;
+ uint64_t cnt_rx_bcst_pkts_ipv4;
+ uint64_t cnt_rx_bcst_pkts_ipv6;
+ uint64_t cnt_rx_mcst_pkts;
+ uint64_t cnt_rx_mcst_pkts_nonip;
+ uint64_t cnt_rx_mcst_pkts_ipv4;
+ uint64_t cnt_rx_mcst_pkts_ipv6;
+ uint64_t cnt_rx_pause_pkts;
+ uint64_t cnt_rx_cbpause_pkts;
+ uint64_t cnt_rx_fcs_errors;
+ uint64_t cnt_rx_symbol_errors;
+ uint64_t cnt_rx_framesize_errors;
+ uint64_t cnt_rx_framing_error_pkts;
+ uint64_t cnt_rx_minto63_pkts;
+ uint64_t cnt_rx_64_pkts;
+ uint64_t cnt_rx_65to127_pkts;
+ uint64_t cnt_rx_128to255_pkts;
+ uint64_t cnt_rx_256to511_pkts;
+ uint64_t cnt_rx_512to1023_pkts;
+ uint64_t cnt_rx_1024to1522_pkts;
+ uint64_t cnt_rx_1523to2047_pkts;
+ uint64_t cnt_rx_2048to4095_pkts;
+ uint64_t cnt_rx_4096to8191_pkts;
+ uint64_t cnt_rx_8192to10239_pkts;
+ uint64_t cnt_rx_10240tomax_pkts;
+ uint64_t cnt_rx_minto63_octets;
+ uint64_t cnt_rx_64_octets;
+ uint64_t cnt_rx_65to127_octets;
+ uint64_t cnt_rx_128to255_octets;
+ uint64_t cnt_rx_256to511_octets;
+ uint64_t cnt_rx_512to1023_octets;
+ uint64_t cnt_rx_1024to1522_octets;
+ uint64_t cnt_rx_1523to2047_octets;
+ uint64_t cnt_rx_2048to4095_octets;
+ uint64_t cnt_rx_4096to8191_octets;
+ uint64_t cnt_rx_8192to10239_octets;
+ uint64_t cnt_rx_10240tomax_octets;
+ uint64_t cnt_rx_octets_nonip;
+ uint64_t cnt_rx_octets_ipv4;
+ uint64_t cnt_rx_octets_ipv6;
+ uint64_t cnt_rx_ucst_octets_nonip;
+ uint64_t cnt_rx_ucst_octets_ipv4;
+ uint64_t cnt_rx_ucst_octets_ipv6;
+ uint64_t cnt_rx_bcst_octets_nonip;
+ uint64_t cnt_rx_bcst_octets_ipv4;
+ uint64_t cnt_rx_bcst_octets_ipv6;
+ uint64_t cnt_rx_mcst_octets_nonip;
+ uint64_t cnt_rx_mcst_octets_ipv4;
+ uint64_t cnt_rx_mcst_octets_ipv6;
+ uint64_t cnt_rx_pause_octets;
+ uint64_t cnt_rx_cbpause_octets;
+ uint64_t cnt_rx_fcs_errors_octets;
+ uint64_t cnt_rx_framing_error_octets;
+ uint64_t cnt_rx_good_octets;
+ uint64_t cnt_rx_bad_octets;
+ uint64_t cnt_rx_priority_pkts[16];
+ uint64_t cnt_rx_invalid_priority_pkts;
+ uint64_t cnt_rx_priority_octets[16];
+ uint64_t cnt_rx_invalid_priority_octets;
+ uint64_t cnt_fid_forwarded_pkts;
+ uint64_t cnt_flood_forwarded_pkts;
+ uint64_t cnt_glort_switched_pkts;
+ uint64_t cnt_glort_routed_pkts;
+ uint64_t cnt_specially_handled_pkts;
+ uint64_t cnt_parse_err_drop_pkts;
+ uint64_t cnt_parity_error_pkts;
+ uint64_t cnt_trapped_pkts;
+ uint64_t cnt_pause_drop_pkts;
+ uint64_t cnt_stp_drop_pkts;
+ uint64_t cnt_stp_ingress_drops_pkts;
+ uint64_t cnt_stp_egress_drops_pkts;
+ uint64_t cnt_reserved_trap_pkts;
+ uint64_t cnt_security_violation_pkts;
+ uint64_t cnt_vlan_tag_drop_pkts;
+ uint64_t cnt_vlan_ingressbv_pkts;
+ uint64_t cnt_vlan_egressbv_pkts;
+ uint64_t cnt_loopback_drops_pkts;
+ uint64_t cnt_glort_miss_drop_pkts;
+ uint64_t cnt_ffu_drop_pkts;
+ uint64_t cnt_invalid_drop_pkts;
+ uint64_t cnt_policer_drop_pkts;
+ uint64_t cnt_ttl_drop_pkts;
+ uint64_t cnt_global_wmdrop_pkts;
+ uint64_t cnt_rx_mpdrop_pkts;
+ uint64_t cnt_rx_hogdrop_pkts;
+ uint64_t cnt_tx_hogdrop_pkts;
+ uint64_t cnt_other_pkts;
+ uint64_t cnt_flood_control_drop_pkts;
+ uint64_t cnt_cmpriv_drop_pkts;
+ uint64_t cnt_smp0_drop_pkts;
+ uint64_t cnt_smp1_drop_pkts;
+ uint64_t cnt_rx_hog0_drop_pkts;
+ uint64_t cnt_rx_hog1_drop_pkts;
+ uint64_t cnt_tx_hog0_drop_pkts;
+ uint64_t cnt_tx_hog1_drop_pkts;
+ uint64_t cnt_rate_limit0_drop_pkts;
+ uint64_t cnt_rate_limit1_drop_pkts;
+ uint64_t cnt_bad_smp_drop_pkts;
+ uint64_t cnt_trigger_drop_redir_pkts;
+ uint64_t cnt_trigger_drop_pkts;
+ uint64_t cnt_trigger_redir_pkts;
+ uint64_t cnt_glort_forwarded_pkts;
+ uint64_t cnt_trigger_mirrored_pkts;
+ uint64_t cnt_broadcast_drop_pkts;
+ uint64_t cnt_dlf_drop_pkts;
+ uint64_t cnt_rx_cm_drop_pkts;
+ uint64_t cnt_fid_forwarded_octets;
+ uint64_t cnt_flood_forwarded_octets;
+ uint64_t cnt_specially_handled_octets;
+ uint64_t cnt_parse_err_drop_octets;
+ uint64_t cnt_parity_error_octets;
+ uint64_t cnt_trapped_octets;
+ uint64_t cnt_pause_drop_octets;
+ uint64_t cnt_stp_drop_octets;
+ uint64_t cnt_security_violation_octets;
+ uint64_t cnt_vlan_tag_drop_octets;
+ uint64_t cnt_vlan_ingressbv_octets;
+ uint64_t cnt_vlan_egressbv_octets;
+ uint64_t cnt_loopback_drop_octets;
+ uint64_t cnt_glort_miss_drop_octets;
+ uint64_t cnt_ffu_drop_octets;
+ uint64_t cnt_policer_drop_octets;
+ uint64_t cnt_ttl_drop_octets;
+ uint64_t cnt_other_octets;
+ uint64_t cnt_flood_control_drop_octets;
+ uint64_t cnt_cmpriv_drop_octets;
+ uint64_t cnt_smp0_drop_octets;
+ uint64_t cnt_smp1_drop_octets;
+ uint64_t cnt_rx_hog0_drop_octets;
+ uint64_t cnt_rx_hog1_drop_octets;
+ uint64_t cnt_tx_hog0_drop_octets;
+ uint64_t cnt_tx_hog1_drop_octets;
+ uint64_t cnt_trigger_drop_octets;
+ uint64_t cnt_trigger_redir_octets;
+ uint64_t cnt_glort_forwarded_octets;
+ uint64_t cnt_tx_ucst_pkts;
+ uint64_t cnt_tx_bcst_pkts;
+ uint64_t cnt_tx_mcst_pkts;
+ uint64_t cnt_tx_ucst_pkts_nonip;
+ uint64_t cnt_tx_bcst_pkts_nonip;
+ uint64_t cnt_tx_mcst_pkts_nonip;
+ uint64_t cnt_tx_ucst_pkts_ip;
+ uint64_t cnt_tx_bcst_pkts_ip;
+ uint64_t cnt_tx_mcst_pkts_ip;
+ uint64_t cnt_tx_pause_pkts;
+ uint64_t cnt_tx_cbpause_pkts;
+ uint64_t cnt_tx_fcs_err_drop_pkts;
+ uint64_t cnt_tx_framing_error_pkts;
+ uint64_t cnt_tx_error_sent_pkts;
+ uint64_t cnt_tx_error_drop_pkts;
+ uint64_t cnt_tx_timeout_pkts;
+ uint64_t cnt_tx_outofmem_err_pkts;
+ uint64_t cnt_tx_unrepair_ecc_pkts;
+ uint64_t cnt_tx_loopback_pkts;
+ uint64_t cnt_tx_ttl_drop_pkts;
+ uint64_t cnt_tx_minto63_pkts;
+ uint64_t cnt_tx_64_pkts;
+ uint64_t cnt_tx_65to127_pkts;
+ uint64_t cnt_tx_128to255_pkts;
+ uint64_t cnt_tx_256to511_pkts;
+ uint64_t cnt_tx_512to1023_pkts;
+ uint64_t cnt_tx_1024to1522_pkts;
+ uint64_t cnt_tx_1523to2047_pkts;
+ uint64_t cnt_tx_2048to4095_pkts;
+ uint64_t cnt_tx_4096to8191_pkts;
+ uint64_t cnt_tx_8192to10239_pkts;
+ uint64_t cnt_tx_10240tomax_pkts;
+ uint64_t cnt_tx_minto63_octets;
+ uint64_t cnt_tx_64_octets;
+ uint64_t cnt_tx_65to127_octets;
+ uint64_t cnt_tx_128to255_octets;
+ uint64_t cnt_tx_256to511_octets;
+ uint64_t cnt_tx_512to1023_octets;
+ uint64_t cnt_tx_1024to1522_octets;
+ uint64_t cnt_tx_1523to2047_octets;
+ uint64_t cnt_tx_2048to4095_octets;
+ uint64_t cnt_tx_4096to8191_octets;
+ uint64_t cnt_tx_8192to10239_octets;
+ uint64_t cnt_tx_10240tomax_octets;
+ uint64_t cnt_tx_ucst_octets_nonip;
+ uint64_t cnt_tx_bcst_octets_nonip;
+ uint64_t cnt_tx_mcst_octets_nonip;
+ uint64_t cnt_tx_ucst_octets_ip;
+ uint64_t cnt_tx_bcst_octets_ip;
+ uint64_t cnt_tx_mcst_octets_ip;
+ uint64_t cnt_tx_ucst_octets;
+ uint64_t cnt_tx_mcst_octets;
+ uint64_t cnt_tx_bcst_octets;
+ uint64_t cnt_tx_fcs_err_drop_octets;
+ uint64_t cnt_tx_octets;
+ uint64_t cnt_tx_error_octets;
+ uint64_t cnt_tx_framing_error_octets;
+ uint64_t cnt_tx_pause_octets;
+ uint64_t cnt_tx_cbpause_octets;
+ uint64_t cnt_tx_fcs_errored_octets;
+ uint64_t cnt_tx_error_sent_octets;
+ uint64_t cnt_tx_timeout_octets;
+ uint64_t cnt_tx_outofmem_err_octets;
+ uint64_t cnt_tx_unrepair_ecc_octets;
+ uint64_t cnt_tx_loopback_octets;
+ uint64_t cnt_tx_ttl_drop_octets;
+ uint64_t cnt_tx_priority_octets[16];
+ uint64_t cnt_under_run_pkts;
+ uint64_t cnt_over_run_pkts;
+ uint64_t cnt_rx_fragment_pkts;
+ uint64_t cnt_rx_undersized_pkts;
+ uint64_t cnt_rx_jabber_pkts;
+ uint64_t cnt_corrupted_pkts;
+ uint64_t cnt_code_errors;
+ uint64_t cnt_rx_oversized_pkts;
+ uint64_t cnt_tx_fcs_errored_pkts;
+ uint64_t cnt_stats_drop_count_tx;
+ uint64_t cnt_stats_drop_count_rx;
+ uint64_t cnt_tx_mirror_pkts;
+ uint64_t cnt_tx_mirror_octets;
+ uint64_t cnt_tx_cmdrop_pkts;
+ uint64_t timestamp;
+};
+
+void fm10k_stats_rule_count_reg(uint16_t rule_id);
+void *fm10k_switch_process_stats(void *ctx_);
+
+void fm10k_stats_epl_port_print(struct fm10k_switch *sw);
+void fm10k_stats_dpdk_port_print(struct fm10k_switch *sw);
+void fm10k_stats_ffu_count_print(struct fm10k_switch *sw);
+void fm10k_stats_port_bank_print(struct fm10k_switch *sw);
+
+#endif /* _fm10k_stats_h */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
index 2423dbf..b4b363c 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.h
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -76,6 +76,37 @@
#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+/*
+ * GLORT MAP
+ */
+#define FM10K_SW_EPLA_GLORT 0x10
+#define FM10K_SW_EPLB_GLORT 0x20
+#define FM10K_SW_PEP01_GLORT 0x30
+#define FM10K_SW_PEP23_GLORT 0x40
+#define FM10K_SW_PEP45_GLORT 0x50
+#define FM10K_SW_PEP67_GLORT 0x60
+
+/*
+ * logical port number
+ */
+#define FM10K_SW_EPLA_LOGICAL_PORT 1
+#define FM10K_SW_EPLB_LOGICAL_PORT 2
+
+#define FM10K_SW_PEP01_LOGICAL_PORT 3
+#define FM10K_SW_PEP23_LOGICAL_PORT 4
+#define FM10K_SW_PEP45_LOGICAL_PORT 5
+#define FM10K_SW_PEP67_LOGICAL_PORT 6
+
+/*
+ * physical port number
+ */
+#define FM10K_SW_EPLA_PHYSICAL_PORT 0
+#define FM10K_SW_EPLB_PHYSICAL_PORT 4
+
+#define FM10K_SW_PEP01_PHYSICAL_PORT 36
+#define FM10K_SW_PEP23_PHYSICAL_PORT 40
+#define FM10K_SW_PEP45_PHYSICAL_PORT 44
+#define FM10K_SW_PEP67_PHYSICAL_PORT 48
#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
@@ -111,6 +142,7 @@
#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
/*
* SWITCH
*/
@@ -125,6 +157,15 @@ struct fm10k_device_info {
uint8_t num_peps;
};
+static struct fm10k_device_info fm10k_device_table[] = {
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4,
+ "Silicom PE3100G2DQiR-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4,
+ "Silicom PE3100G2DQiRL-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4,
+ "Silicom PE3100G2DQiRM-QX4/QS4/QL4", 2, 100, 2, 4 },
+};
+
struct fm10k_i2c;
struct fm10k_sbus;
@@ -287,6 +328,114 @@ struct fm10k_switch {
fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
}
+
+static struct fm10k_sw_port_map fm10k_pep_port_map[FM10K_SW_PEPS_SUPPORTED] = {
+ {
+ FM10K_SW_PEP01_GLORT,
+ FM10K_SW_PEP01_LOGICAL_PORT,
+ FM10K_SW_PEP01_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP23_GLORT,
+ FM10K_SW_PEP23_LOGICAL_PORT,
+ FM10K_SW_PEP23_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP45_GLORT,
+ FM10K_SW_PEP45_LOGICAL_PORT,
+ FM10K_SW_PEP45_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP67_GLORT,
+ FM10K_SW_PEP67_LOGICAL_PORT,
+ FM10K_SW_PEP67_PHYSICAL_PORT
+ },
+};
+
+static struct fm10k_sw_port_map fm10k_epl_port_map[FM10K_SW_EPLS_SUPPORTED] = {
+ {
+ FM10K_SW_EPLA_GLORT,
+ FM10K_SW_EPLA_LOGICAL_PORT,
+ FM10K_SW_EPLA_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_EPLB_GLORT,
+ FM10K_SW_EPLB_LOGICAL_PORT,
+ FM10K_SW_EPLB_PHYSICAL_PORT
+ },
+};
+
+static inline uint32_t
+fm10k_switch_pf_logical_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_epl_logical_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_vf_glort_get(uint8_t vf_no)
+{
+ return FM10K_SW_VF_GLORT_START + vf_no;
+}
+
+static inline uint32_t
+fm10k_switch_pf_glort_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_epl_glort_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2)
+{
+ uint8_t idx;
+ if (pf1 > pf2)
+ idx = (pf2 & 0xf) | (pf1 << 4 & 0xf0);
+ else
+ idx = (pf1 & 0xf) | (pf2 << 4 & 0xf0);
+ return FM10K_SW_PFS_GLORT_START + idx;
+}
+
+
+static inline struct fm10k_device_info*
+fm10k_get_device_info(struct fm10k_hw *hw)
+{
+ unsigned int i;
+ struct fm10k_device_info *info;
+ uint16_t pci_vendor = hw->vendor_id;
+ uint16_t pci_device = hw->device_id;
+ uint16_t pci_subvendor = hw->subsystem_vendor_id;
+ uint16_t pci_subdevice = hw->subsystem_device_id;
+
+ if (pci_vendor != FM10K_SW_VENDOR_ID_INTEL ||
+ pci_device != FM10K_SW_DEV_ID_FM10K)
+ return (NULL);
+
+ for (i = 0;
+ i < sizeof(fm10k_device_table) /
+ sizeof(fm10k_device_table[0]);
+ i++) {
+ info = &fm10k_device_table[i];
+ if (pci_subvendor == info->subvendor &&
+ pci_subdevice == info->subdevice) {
+ return info;
+ }
+ }
+
+ return NULL;
+}
+
+
#define fm10k_udelay usec_delay
typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
@@ -312,15 +461,6 @@ struct fm10k_flow_list *
void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
-uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
-uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
-uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
-uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
-uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
-uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
-uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
- uint16_t vlan1, uint16_t vlan2, bool *p_new);
-
int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 4/5] net/fm10k: add flow interface and switch management
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
` (2 preceding siblings ...)
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add flow interface to support offload flow into HW.
It supports parse vlan and parse mpls, all these
data will be transferred to ffu data.
Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
It create 3 threads. One for interrupt handler, one for
LED controller, one for statistics.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 2 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_stats.c | 128 ++
drivers/net/fm10k/switch/fm10k_switch.c | 2376 +++++++++++++++++++++++++++++++
5 files changed, 3404 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index d4d2037..9a245c9 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -91,6 +91,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_flow.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_switch.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_flow.c b/drivers/net/fm10k/switch/fm10k_flow.c
new file mode 100644
index 0000000..353f021
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.c
@@ -0,0 +1,872 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <sys/queue.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_eth_ctrl.h>
+#include <rte_tailq.h>
+#include <rte_flow_driver.h>
+
+#include "fm10k_flow.h"
+#include "fm10k_switch.h"
+#include "fm10k_ffu.h"
+#include "fm10k_config.h"
+
+
+static int fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static struct rte_flow *fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static int fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error);
+static int fm10k_flow_flush(struct rte_eth_dev *dev,
+ struct rte_flow_error *error);
+static int fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error);
+
+const struct rte_flow_ops fm10k_flow_ops = {
+ .validate = fm10k_flow_validate,
+ .create = fm10k_flow_create,
+ .destroy = fm10k_flow_destroy,
+ .flush = fm10k_flow_flush,
+};
+
+union fm10k_filter_t cons_filter;
+enum rte_filter_type fm10k_cons_filter_type = RTE_ETH_FILTER_NONE;
+
+/**
+ * MPLS filter configuration.
+ */
+enum fm10k_mpls_type {
+ FM10K_MPLS_TYPE_UNI,
+ FM10K_MPLS_TYPE_MULTI,
+};
+
+enum fm10k_mpls_action {
+ FM10K_MPLS_ACTION_DROP,
+ FM10K_MPLS_ACTION_QUEUE,
+};
+
+struct fm10k_mpls_filter_conf {
+ enum fm10k_mpls_type mpls_type; /**< mandatory for MPLS */
+ uint32_t mpls_header; /**< MPLS header */
+ uint32_t mpls_header_mask; /**< MPLS header mask */
+ enum fm10k_mpls_action mpls_action;
+ uint16_t queue;
+ uint8_t ffu_id;
+ uint8_t ffu_prio;
+};
+
+/**
+ * VLAN filter configuration.
+ */
+struct fm10k_vlan_filter_conf {
+ int ffu_id;
+ uint8_t ffu_prio;
+ uint8_t is_ingress;
+ uint8_t port;
+ uint8_t in_ext_port;
+ uint8_t out_ext_port;
+ uint16_t in_vlan;
+ uint16_t out_vlan;
+};
+
+
+union fm10k_filter_t {
+ struct fm10k_mpls_filter_conf mpls_filter;
+ struct fm10k_vlan_filter_conf vlan_filter;
+};
+
+typedef int (*parse_filter_t)(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+struct fm10k_valid_pattern {
+ enum rte_flow_item_type *items;
+ parse_filter_t parse_filter;
+};
+
+static enum rte_flow_item_type pattern_mpls_1[] = {
+ RTE_FLOW_ITEM_TYPE_ETH,
+ RTE_FLOW_ITEM_TYPE_MPLS,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_1[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_2[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static int fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+static struct fm10k_valid_pattern fm10k_supported_patterns[] = {
+ /* MPLS */
+ { pattern_mpls_1, fm10k_flow_parse_mpls_filter },
+ /* VLAN */
+ { pattern_vlan_1, fm10k_flow_parse_vlan_filter },
+ /* VLAN */
+ { pattern_vlan_2, fm10k_flow_parse_vlan_filter },
+};
+
+static const struct rte_flow_action *
+fm10k_next_item_of_action(const struct rte_flow_action *actions,
+ uint32_t *index)
+{
+ static const struct rte_flow_action *act;
+
+ act = actions + *index;
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID) {
+ (*index)++;
+ act = actions + *index;
+ }
+ return act;
+}
+
+/* Find the first VOID or non-VOID item pointer */
+static const struct rte_flow_item *
+fm10k_find_first_item(const struct rte_flow_item *item, bool is_void)
+{
+ bool is_find;
+
+ while (item->type != RTE_FLOW_ITEM_TYPE_END) {
+ if (is_void)
+ is_find = item->type == RTE_FLOW_ITEM_TYPE_VOID;
+ else
+ is_find = item->type != RTE_FLOW_ITEM_TYPE_VOID;
+ if (is_find)
+ break;
+ item++;
+ }
+ return item;
+}
+
+/* Skip all VOID items of the pattern */
+static void
+fm10k_pattern_skip_void_item(struct rte_flow_item *items,
+ const struct rte_flow_item *pattern)
+{
+ uint32_t cpy_count = 0;
+ const struct rte_flow_item *pb = pattern, *pe = pattern;
+
+ for (;;) {
+ /* Find a non-void item first */
+ pb = fm10k_find_first_item(pb, false);
+ if (pb->type == RTE_FLOW_ITEM_TYPE_END) {
+ pe = pb;
+ break;
+ }
+
+ /* Find a void item */
+ pe = fm10k_find_first_item(pb + 1, true);
+
+ cpy_count = pe - pb;
+ rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+
+ items += cpy_count;
+
+ if (pe->type == RTE_FLOW_ITEM_TYPE_END) {
+ pb = pe;
+ break;
+ }
+
+ pb = pe + 1;
+ }
+ /* Copy the END item. */
+ rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+}
+
+/* Check if the pattern matches a supported item type array */
+static bool
+fm10k_match_pattern(enum rte_flow_item_type *item_array,
+ struct rte_flow_item *pattern)
+{
+ struct rte_flow_item *item = pattern;
+
+ while ((*item_array == item->type) &&
+ (*item_array != RTE_FLOW_ITEM_TYPE_END)) {
+ item_array++;
+ item++;
+ }
+
+ return (*item_array == RTE_FLOW_ITEM_TYPE_END &&
+ item->type == RTE_FLOW_ITEM_TYPE_END);
+}
+
+/* Find if there's parse filter function matched */
+static parse_filter_t
+fm10k_find_parse_filter_func(struct rte_flow_item *pattern, uint32_t *idx)
+{
+ parse_filter_t parse_filter = NULL;
+ uint8_t i = *idx;
+
+ for (; i < RTE_DIM(fm10k_supported_patterns); i++) {
+ if (fm10k_match_pattern(fm10k_supported_patterns[i].items,
+ pattern)) {
+ parse_filter = fm10k_supported_patterns[i].parse_filter;
+ break;
+ }
+ }
+
+ *idx = ++i;
+
+ return parse_filter;
+}
+
+/* Parse attributes */
+static int
+fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ /* Not supported */
+ if (attr->group) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+ attr, "Not support group.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+/*
+ * MPLS
+ */
+/* 1. Last in item should be NULL as range is not supported.
+ * 2. Supported filter types: MPLS label.
+ * 3. Mask of fields which need to be matched should be
+ * filled with 1.
+ * 4. Mask of fields which needn't to be matched should be
+ * filled with 0.
+ */
+static int
+fm10k_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_mpls *mpls_spec;
+ const struct rte_flow_item_mpls *mpls_mask;
+ const struct rte_flow_item_eth *eth_spec;
+ enum rte_flow_item_type item_type;
+ const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
+ uint32_t label_be = 0;
+ uint32_t be_mask = 0;
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_ETH:
+ if (!item->spec || item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid ETH item");
+ return -rte_errno;
+ }
+ eth_spec =
+ (const struct rte_flow_item_eth *)item->spec;
+
+ if (rte_be_to_cpu_16(eth_spec->type) == 0x8847) {
+ filter->mpls_type = FM10K_MPLS_TYPE_UNI;
+ } else if (rte_be_to_cpu_16(eth_spec->type) == 0x8848) {
+ filter->mpls_type = FM10K_MPLS_TYPE_MULTI;
+ } else {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Invalid ETH item");
+ return -rte_errno;
+ }
+ break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ mpls_spec =
+ (const struct rte_flow_item_mpls *)item->spec;
+ mpls_mask =
+ (const struct rte_flow_item_mpls *)item->mask;
+
+ if (!mpls_spec || !mpls_mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS item");
+ return -rte_errno;
+ }
+
+ if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS label mask");
+ return -rte_errno;
+ }
+ rte_memcpy(((uint8_t *)&label_be + 1),
+ mpls_spec->label_tc_s, 3);
+ rte_memcpy(((uint8_t *)&be_mask + 1),
+ mpls_mask->label_tc_s, 3);
+ filter->mpls_header =
+ rte_be_to_cpu_32(label_be) >> 4;
+ filter->mpls_header_mask =
+ rte_be_to_cpu_32(be_mask) >> 4;
+
+ fm10k_cons_filter_type = RTE_ETH_FILTER_TUNNEL;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* MPLS action only supports QUEUE or DROP. */
+static int
+fm10k_flow_parse_mpls_action(const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ const struct rte_flow_action_queue *act_q;
+ uint32_t index = 0;
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_QUEUE &&
+ act->type != RTE_FLOW_ACTION_TYPE_DROP) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ if (act->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->mpls_action = FM10K_MPLS_ACTION_QUEUE;
+ filter->queue = act_q->index;
+ } else {
+ filter->mpls_action = FM10K_MPLS_ACTION_DROP;
+ }
+
+ /* Check if the next non-void item is END */
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ struct fm10k_mpls_filter_conf *mpls_filter =
+ &filter->mpls_filter;
+ int ret;
+
+ ret = fm10k_flow_parse_mpls_pattern(dev, pattern,
+ error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_mpls_action(actions, error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+
+/*
+ * VLAN
+ */
+static int
+fm10k_flow_parse_vlan_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_vlan *vlan_spec;
+ const struct rte_flow_item_phy_port *pp_spec;
+ enum rte_flow_item_type item_type;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan pattern ffu id %d", filter->ffu_id);
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_VLAN:
+ vlan_spec =
+ (const struct rte_flow_item_vlan *)item->spec;
+
+ if (!vlan_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid VLAN item");
+ return -rte_errno;
+ }
+ break;
+
+ case RTE_FLOW_ITEM_TYPE_PHY_PORT:
+ pp_spec =
+ (const struct rte_flow_item_phy_port *)item->spec;
+ if (!pp_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid PHY PORT item");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_flow_parse_vlan_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ uint32_t index = 0;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan action name %s ffu id %d",
+ dev->device->name, filter->ffu_id);
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK &&
+ act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+ if (act->type == RTE_FLOW_ACTION_TYPE_END)
+ return 0;
+
+ index++;
+ /* Check if the next non-void item is END */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ int ret;
+ struct fm10k_vlan_filter_conf *vlan_filter =
+ &filter->vlan_filter;
+
+ ret = fm10k_flow_parse_vlan_pattern(dev, pattern,
+ error, vlan_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_vlan_action(dev, actions, error, vlan_filter);
+ if (ret)
+ return ret;
+
+ if (attr->ingress)
+ vlan_filter->is_ingress = 1;
+ else if (attr->egress)
+ vlan_filter->is_ingress = 0;
+ vlan_filter->ffu_prio = attr->priority;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+/*
+ *
+ */
+static int
+fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct rte_flow_item *items; /* internal pattern w/o VOID items */
+ parse_filter_t parse_filter;
+ uint32_t item_num = 0; /* non-void item number of pattern*/
+ uint32_t i = 0;
+ bool flag = false;
+ int ret = -1;
+
+ if (!pattern) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "NULL pattern.");
+ return -rte_errno;
+ }
+
+ if (!actions) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_NUM,
+ NULL, "NULL action.");
+ return -rte_errno;
+ }
+
+ if (!attr) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR,
+ NULL, "NULL attribute.");
+ return -rte_errno;
+ }
+
+ /* Get the non-void item number of pattern */
+ while ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_END) {
+ if ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_VOID)
+ item_num++;
+ i++;
+ }
+ item_num++;
+
+ items = rte_zmalloc("fm10k_pattern",
+ item_num * sizeof(struct rte_flow_item), 0);
+ if (!items) {
+ rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "No memory for PMD internal items.");
+ return -ENOMEM;
+ }
+
+ fm10k_pattern_skip_void_item(items, pattern);
+
+ i = 0;
+ do {
+ parse_filter = fm10k_find_parse_filter_func(items, &i);
+ if (!parse_filter && !flag) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ pattern, "Unsupported pattern");
+ rte_free(items);
+ return -rte_errno;
+ }
+ if (parse_filter)
+ ret = parse_filter(dev, attr, items, actions,
+ error, &cons_filter);
+ flag = true;
+ } while ((ret < 0) && (i < RTE_DIM(fm10k_supported_patterns)));
+
+ rte_free(items);
+
+ return ret;
+}
+
+static struct fm10k_cfg_flow *
+fm10k_flow_cfg_transfer(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ int i;
+ u8 port_id;
+ int set_port_num = 0, set_vlan_num = 0;
+ u16 fw_port_id = 0, bp_port_id = 0;
+ u16 filter_vlan_id = 0, fw_vlan_id = 0, bp_vlan_id = 0;
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_cfg_flow *cf;
+
+ cf = rte_zmalloc("fm10k_rule", sizeof(struct fm10k_cfg_flow), 0);
+ if (!cf) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return NULL;
+ }
+ memset(cf, 0, sizeof(struct fm10k_cfg_flow));
+
+ port_id = fm10k_switch_dpdk_port_no_get(hw);
+ for (i = 0; i < 4; i++) {
+ if (pattern[i].type == RTE_FLOW_ITEM_TYPE_VLAN) {
+ filter_vlan_id =
+ rte_be_to_cpu_16
+ (((const struct rte_flow_item_vlan *)
+ pattern[i].spec)->tci);
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_PHY_PORT) {
+ if (set_port_num)
+ bp_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ else
+ fw_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ set_port_num++;
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_END) {
+ break;
+ }
+ }
+
+ for (i = 0; i < 3; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_MARK) {
+ if (set_vlan_num)
+ bp_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ else
+ fw_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ set_vlan_num++;
+ } else if (actions[i].type == RTE_FLOW_ACTION_TYPE_END) {
+ break;
+ }
+ }
+
+ if (attr->ingress && !attr->egress) {
+ /* this port is DPDK port and it is destination port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = fw_port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && attr->egress) {
+ /* this port is DPDK port and it is source port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && !attr->egress) {
+ /* two ports are external port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else {
+ /* two ports are DPDK port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ }
+
+ if (set_port_num == 2 && set_vlan_num == 2) {
+ cf->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[1].port_no = bp_port_id;
+ cf->fw_port[1].vlan_id = bp_vlan_id;
+ }
+
+ return cf;
+}
+
+
+static struct rte_flow *
+fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ struct fm10k_switch *sw = fm10k_switch_get();
+ struct fm10k_cfg_flow *cf;
+ int ret;
+
+ flow = rte_zmalloc("fm10k_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return flow;
+ }
+
+ ret = fm10k_flow_validate(dev, attr, pattern, actions, error);
+ if (ret < 0)
+ return NULL;
+
+ cf = fm10k_flow_cfg_transfer(dev, attr, pattern, actions, error);
+ if (!cf)
+ goto free_flow;
+
+ flow->rule = cf;
+ fm10k_ffu_flow_enable(sw, cf);
+ fm10k_config_flow_list_add_tail(fm10k_config_flowset_current_get(), cf);
+
+ TAILQ_INSERT_TAIL((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw), flow, node);
+ return flow;
+
+free_flow:
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to create flow.");
+ rte_free(flow);
+ return NULL;
+}
+
+static int
+fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret = 0;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (flow->rule) {
+ fm10k_config_flow_list_delete
+ ((struct fm10k_cfg_flow *)flow->rule);
+ fm10k_ffu_flow_disable(sw,
+ (struct fm10k_cfg_flow *)flow->rule);
+ }
+
+ if (!ret) {
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow->rule);
+ rte_free(flow);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to destroy flow.");
+ }
+ return ret;
+}
+
+
+static int
+fm10k_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ void *temp;
+ int ret = 0;
+ struct fm10k_cfg_flow *cf;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ /* Delete flows in flow list. */
+ TAILQ_FOREACH_SAFE(flow,
+ (struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ node, temp) {
+ cf = flow->rule;
+ if (cf) {
+ fm10k_config_flow_list_delete(cf);
+ fm10k_ffu_flow_disable(sw, cf);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "No such rule in flow.");
+ }
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+void
+fm10k_flow_list_init(void *flow_list)
+{
+ TAILQ_INIT((struct fm10k_flow_list *)flow_list);
+}
+
+/* Flow operations */
+const struct rte_flow_ops *
+fm10k_flow_ops_get(void)
+{
+ return &fm10k_flow_ops;
+}
+
+
diff --git a/drivers/net/fm10k/switch/fm10k_flow.h b/drivers/net/fm10k/switch/fm10k_flow.h
new file mode 100644
index 0000000..c538544
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_FLOW_H_
+#define _FM10K_SW_FLOW_H_
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+/*
+ * Struct to store flow created.
+ */
+struct rte_flow {
+ TAILQ_ENTRY(rte_flow) node;
+ enum rte_filter_type filter_type;
+ void *rule;
+};
+
+TAILQ_HEAD(fm10k_flow_list, rte_flow);
+
+#endif /* _FM10K_SW_FLOW_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
index 563f9ac..b90f190 100644
--- a/drivers/net/fm10k/switch/fm10k_stats.c
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -1012,6 +1012,133 @@ struct fm10k_stats_data {
}
}
+void
+fm10k_stats_dpdk_port_print(struct fm10k_switch *sw)
+{
+ int i, j, lport, pf_no;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_PEPS_MAX];
+ static struct fm10k_stats_data last_queue_data[FM10K_SW_PEPS_MAX][4];
+ struct fm10k_stats_data data, mydata;
+ char pf_ports[10];
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].hw == NULL)
+ continue;
+ if (sw->dpdk_cfg->ports[i].type != FM10K_CONFIG_DPDK_PF)
+ continue;
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ memset(&mydata, 0, sizeof(mydata));
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ for (j = 0; j < 2; j++) {
+ pf_no =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[j];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters
+ (sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ }
+ } else if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ } else {
+ continue;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ sprintf(pf_ports, "%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0]);
+ else
+ sprintf(pf_ports, "%d/%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1]);
+
+ FM10K_SW_INFO("DPDK port %-2d pf %-5s tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, pf_ports,
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_QUEUE))
+ continue;
+ memset(&mydata, 0, sizeof(mydata));
+ for (j = 0;
+ j < sw->dpdk_cfg->ports[i].tx_queue_num;
+ j++) {
+ struct fm10k_hw *hw = sw->dpdk_cfg->ports[i].hw;
+ uint16_t queue_id = j, pf_no;
+
+ fm10k_switch_dpdk_hw_queue_map(hw, queue_id,
+ sw->dpdk_cfg->ports[i].tx_queue_num,
+ &hw, &queue_id);
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ data.tx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPTC(queue_id));
+ data.rx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPRC(queue_id));
+ data.rx_drop =
+ FM10K_READ_REG(hw, FM10K_QPRDC(queue_id));
+ mydata.tx_pkts += data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop;
+ FM10K_SW_INFO(" queue %d(%d:%d) tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n", j,
+ pf_no, queue_id,
+ (unsigned long long)(data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts),
+ (unsigned long long)(data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts),
+ (unsigned long long)(data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop));
+ last_queue_data[pf_no][queue_id] = data;
+ }
+ FM10K_SW_INFO(" total tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+ }
+}
static void
fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
@@ -1096,6 +1223,7 @@ struct fm10k_stats_data {
FM10K_CONFIG_DEBUG_STATS_PORT)) {
FM10K_SW_INFO("--- port statistic ---\n");
fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
}
if (fm10k_config_check_debug(sw->dpdk_cfg,
FM10K_CONFIG_DEBUG_STATS_FFU)) {
diff --git a/drivers/net/fm10k/switch/fm10k_switch.c b/drivers/net/fm10k/switch/fm10k_switch.c
new file mode 100644
index 0000000..009d62a
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.c
@@ -0,0 +1,2376 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_ethdev.h>
+#include <rte_ethdev_pci.h>
+
+#include <rte_malloc.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_serdes.h"
+#include "fm10k_i2c.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+#include "fm10k_config.h"
+
+
+static struct fm10k_switch fm10k_sw;
+
+#define FM10K_AM_TIMEOUT 16384
+#define FM10K_COMP_PPM_SCALE 1000000
+
+#define FM10K_LED_POLL_INTERVAL_MS 500
+#define FM10K_LED_BLINKS_PER_SECOND 2
+
+
+/*
+ * use epl as external port map, only support QUAD_ON
+ */
+struct fm10k_sched_prog {
+ uint8_t idle;
+ uint8_t phys; /* physical port */
+ uint8_t log; /* logical port */
+ uint8_t quad; /* now, only support QUAD_ON */
+/* convert entry to idle if EPL in quad port mode */
+#define FM10K_SW_QUAD_OFF 0
+/* always use quad port mode */
+#define FM10K_SW_QUAD_ON 1
+/* use quad port mode if link speed is 40/100G */
+#define FM10K_SW_QUAD_40_100 2
+};
+
+static struct fm10k_sched_prog
+ fm10k_sched_prog[FM10K_SW_PEPS_SUPPORTED + FM10K_SW_EPLS_SUPPORTED + 1];
+
+
+/*
+ * Note that for physical port numbers, the initial values used for
+ * the EPL entries assume EPL[A] is EPL[0] and EPL[B] is EPL[1].
+ * fm10k_switch_determine_epls() will update these physical port
+ * numbers based on the actual A and B indices.
+ */
+static void
+fm10k_switch_set_sched_prog(void)
+{
+ int i;
+ int start = 0;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_sched_prog[i].idle = 0;
+ fm10k_sched_prog[i].phys = fm10k_epl_port_map[i].physical_port;
+ fm10k_sched_prog[i].log = fm10k_epl_port_map[i].logical_port;
+ fm10k_sched_prog[i].quad = FM10K_SW_QUAD_ON;
+ }
+ start += FM10K_SW_EPLS_SUPPORTED;
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ fm10k_sched_prog[start + i].idle = 0;
+ fm10k_sched_prog[start + i].phys =
+ fm10k_pep_port_map[i].physical_port;
+ fm10k_sched_prog[start + i].log =
+ fm10k_pep_port_map[i].logical_port;
+ fm10k_sched_prog[start + i].quad = FM10K_SW_QUAD_40_100;
+ }
+ start += FM10K_SW_PEPS_SUPPORTED;
+ fm10k_sched_prog[start].idle = 1;
+}
+
+
+static void
+fm10k_switch_determine_epls(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t phys;
+
+ sw->epla_no = 0;
+ sw->eplb_no = 6;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ sw->epla_no = 1;
+ sw->eplb_no = 6;
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE325G2DSIR):
+ sw->epla_no = 1;
+ sw->eplb_no = 7;
+ break;
+ }
+
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ if (fm10k_sched_prog[i].idle)
+ continue;
+
+ phys = fm10k_sched_prog[i].phys;
+ if (phys <= 3) {
+ /* Substitute actual epla phys port number */
+ fm10k_sched_prog[i].phys = sw->epla_no * 4 + phys;
+ } else if (phys >= 4 && phys <= 7) {
+ /* Substitute actual eplb phys port number */
+ fm10k_sched_prog[i].phys = sw->eplb_no * 4 + (phys - 4);
+ }
+ }
+}
+
+static void
+fm10k_hw_eicr_disable_source(struct fm10k_switch *sw, unsigned int source)
+{
+ unsigned int shift;
+
+ switch (source) {
+ case FM10K_SW_EICR_PCA_FAULT:
+ shift = FM10K_SW_EICR_PCA_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_THI_FAULT:
+ shift = FM10K_SW_EICR_THI_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_FUM_FAULT:
+ shift = FM10K_SW_EICR_FUM_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAILBOX:
+ shift = FM10K_SW_EICR_MAILBOX_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_READY:
+ shift = FM10K_SW_EICR_SWITCH_READY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_NREADY:
+ shift = FM10K_SW_EICR_SWITCH_NREADY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_INT:
+ shift = FM10K_SW_EICR_SWITCH_INT_SHIFT;
+ break;
+ case FM10K_SW_EICR_SRAM_ERROR:
+ shift = FM10K_SW_EICR_SRAM_ERROR_SHIFT;
+ break;
+ case FM10K_SW_EICR_VFLR:
+ shift = FM10K_SW_EICR_VFLR_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAX_HOLD_TIME:
+ shift = FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT;
+ break;
+ default:
+ return;
+ }
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_DISABLE << (shift * 2));
+ fm10k_write_flush(sw);
+}
+
+
+unsigned int
+fm10k_switch_eplidx_to_eplno(struct fm10k_switch *sw, unsigned int eplidx)
+{
+ return eplidx ? sw->eplb_no : sw->epla_no;
+}
+
+
+static uint16_t
+fm10k_switch_ppm_to_tx_clk_compensation_timeout(uint32_t pcs, uint32_t num_ppm)
+{
+ unsigned int scale;
+ unsigned int ppm;
+ unsigned int timeout;
+ unsigned int am_ppm;
+
+ if (num_ppm == 0 || pcs == FM10K_SW_EPL_PCS_SEL_DISABLE) {
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER)
+ return (FM10K_AM_TIMEOUT / 2);
+ else
+ return (0);
+ }
+
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER) {
+ am_ppm = 1000000 / FM10K_AM_TIMEOUT;
+ scale = FM10K_COMP_PPM_SCALE / 2;
+ } else {
+ am_ppm = 0;
+ scale = FM10K_COMP_PPM_SCALE;
+ }
+
+ ppm = num_ppm + am_ppm;
+ timeout = scale / ppm;
+
+ if (timeout >= 0xffff)
+ return 0xffff;
+ else
+ return timeout;
+}
+
+
+static int
+fm10k_switch_configure_epls(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ext_ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_device_info *cfg = sw->info;
+ u32 mac_cfg[FM10K_SW_MAC_CFG_ARRAY_SIZE];
+ u32 data, pcs, qpl;
+ u32 pcstmp;
+ unsigned int i, j;
+ unsigned int dic_enable;
+ unsigned int anti_bubble_wm;
+ unsigned int rate_fifo_wm;
+ unsigned int rate_fifo_slow_inc, rate_fifo_fast_inc;
+ int error;
+ u16 timeout;
+
+ cfg = fm10k_get_device_info(hw);
+ if (cfg == NULL)
+ return -1;
+
+ /*
+ * Assumptions:
+ * - All external interfaces are the same speed
+ * - 1G/10G ports are packed into the minimum number of EPLs
+ * - quad-mode EPLs use lane 0 as master
+ * - The lowest numbered PEPs are used
+ * - PEPs are always used in x8 mode.
+ */
+ switch (cfg->ext_port_speed) {
+ case 10:
+ pcs = FM10K_SW_EPL_PCS_SEL_10GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 1;
+ anti_bubble_wm = 5;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 12;
+ rate_fifo_fast_inc = 13;
+ break;
+ case 25:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 0;
+ anti_bubble_wm = 6;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 32;
+ rate_fifo_fast_inc = 33;
+ break;
+ case 40:
+ pcs = FM10K_SW_EPL_PCS_SEL_40GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 5;
+ rate_fifo_slow_inc = 51;
+ rate_fifo_fast_inc = 52;
+ break;
+ case 100:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 129;
+ rate_fifo_fast_inc = 130;
+ break;
+ default:
+ error = -1;
+ goto done;
+ }
+
+ /*
+ * EPL_CFG_A
+ *
+ * Mark all used lanes as active and all unused lanes as
+ * inactive. Adjust timeout and skew tolerance values for EPLs that
+ * are used.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ data = fm10k_read_switch_reg(sw, FM10K_SW_EPL_CFG_A(i));
+ data &= ~FM10K_SW_EPL_CFG_A_ACTIVE_QUAD;
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < ext_ports->num_ports; j++) {
+ port = &ext_ports->ports[j];
+ if (port->eplno == i)
+ data |= port->is_quad ?
+ FM10K_SW_EPL_CFG_A_ACTIVE_QUAD :
+ FM10K_SW_EPL_CFG_A_ACTIVE
+ (port->first_lane);
+ }
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_TIMEOUT, 19, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_SKEW_TOLERANCE, 38, data);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_A(i), data);
+ }
+
+ /*
+ * EPL_CFG_B
+ *
+ * Disable all unused lanes and configure all used ones
+ * appropriately. For EPLs used in quad port mode, the master lane
+ * is the only one that gets configured.
+ */
+ data = 0;
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++) {
+ if (j < ext_ports->ports_per_epl)
+ pcstmp = pcs;
+ else
+ pcstmp = FM10K_SW_EPL_PCS_SEL_DISABLE;
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j, pcstmp, j, j);
+ }
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE, qpl);
+ } else {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++)
+ data |=
+ FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j,
+ FM10K_SW_EPL_PCS_SEL_DISABLE, j, j);
+ data |=
+ FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE,
+ FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_B(i), data);
+ }
+
+ /*
+ * MAC_CFG, LINK_RULES and PCS_ML_BASER_CFG
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_read_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ /* dic enable */
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_IDLE_ENABLE_DIC,
+ dic_enable, mac_cfg);
+
+ /* ifg */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_IDLE_MIN_IFG_BYTES,
+ 12, mac_cfg);
+
+ /* tx pad size: (64 bytes + 8 preamble) / 4 */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_MIN_COLUMNS,
+ 18, mac_cfg);
+
+ /* tx clock compensation */
+ timeout =
+ fm10k_switch_ppm_to_tx_clk_compensation_timeout(pcs,
+ 100);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE,
+ (timeout > 0), mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT,
+ timeout, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_PREAMBLE_MODE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MIN_FRAME_LENGTH,
+ 64, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MAX_FRAME_LENGTH,
+ FM10K_SW_PACKET_SIZE_MAX, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_RX_IGNORE_IFG_ERRORS,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FCS_MODE,
+ FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_IEEE_1588_ENABLE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_DRAIN_MODE,
+ FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIMEOUT,
+ 100, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIME_SCALE,
+ 3, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIMEOUT,
+ 180, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIME_SCALE,
+ 1, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIMEOUT,
+ 20, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIME_SCALE,
+ 1, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LPI_AUTOMATIC,
+ 1, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LP_IDLE_REQUEST,
+ 0, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FAULT_MODE,
+ FM10K_SW_TX_MAC_FAULT_MODE_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_ANTI_BUBBLE_WATERMARK,
+ anti_bubble_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_WATERMARK,
+ rate_fifo_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_FAST_INC,
+ rate_fifo_fast_inc, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_SLOW_INC,
+ rate_fifo_slow_inc, mac_cfg);
+
+ fm10k_write_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j));
+
+ /* link-up debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_UP,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_UP,
+ 30, data);
+
+ /* link-down debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_DOWN,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_DOWN,
+ 5, data);
+
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_HEARTBEAT_TIME_SCALE,
+ cfg->ext_port_speed == 10 ? 4 : 0, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j), data);
+ }
+
+ if (cfg->ext_port_speed != 10)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PCS_ML_BASER_CFG(i), 0x00003fff);
+ }
+
+
+ /*
+ * LANE_CFG, LANE_SERDES_CFG, LANE_ENERGY_DETECT_CFG,
+ * LANE_SIGNAL_DETECT_CFG, and EPL_FIFO_ERROR_STATUS
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_CFG(i, j), 0);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_CFG(i, j), 0x1106);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j));
+
+ data |=
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY;
+
+ data &=
+ ~FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j), data);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j));
+
+ data &=
+ ~(FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY);
+
+ data |=
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j), data);
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+ }
+
+ error = fm10k_epl_serdes_reset_and_load_all(sw);
+ if (error)
+ goto done;
+
+ /*
+ * EPL_FIFO_ERROR_STATUS LINK_IP
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IP(i, j), FM10K_SW_MASK32(31, 0));
+ }
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+done:
+ return (error);
+}
+
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+
+int
+fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ /* source port is external port number */
+ if (src_port < 0 || src_port == dest_port)
+ return -1;
+
+ return fm10k_ffu_mirror_set(sw, src_port, dest_port, vlan);
+}
+
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (src_port < 0)
+ return -1;
+
+ return fm10k_ffu_mirror_reset(sw, src_port);
+}
+
+
+typedef struct {
+ u8 lport;
+ u8 has_ftag;
+} fm10k_sw_lport;
+
+static int
+fm10k_switch_init(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ u32 data;
+ unsigned int num_lports = sw->info->num_peps + FM10K_SW_EPLS_SUPPORTED;
+ fm10k_sw_lport all_lports[num_lports];
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ unsigned int is_quad;
+ unsigned int table_idx;
+ int error;
+ u32 watermark;
+ u64 data64, data64_2;
+
+ /*
+ * Build list of all logical ports that might appear in the
+ * scheduler program. Note that for any particular card
+ * configuration, not all of these logical ports may be used.
+ */
+ table_idx = 0;
+ for (i = 0; i < sw->info->num_peps; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->pep_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 1;
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->epl_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 0;
+ }
+
+ if (table_idx != num_lports) {
+ FM10K_SW_ERR("fm10k switch lport table construction error");
+ return -1;
+ }
+
+ /*
+ * Reset the switch to get to the default state
+ */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+
+ usec_delay(100);
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ usec_delay(1000);
+
+ /* Clear memories */
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL);
+ fm10k_write_flush(sw);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_EPL);
+ fm10k_write_flush(sw);
+ usec_delay(800);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL, 0);
+ fm10k_write_flush(sw);
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~(FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET);
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ /* ensure switch reset is deasserted for at least 100ns */
+ usec_delay(1);
+
+ sw->epl_sbus = fm10k_sbus_attach(sw, "EPL", FM10K_SW_SBUS_EPL_CFG);
+ if (sw->epl_sbus == NULL) {
+ error = -1;
+ goto done;
+ }
+
+ /* Clear non-BIST accessible pause state memories */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 0), 0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 1), 0);
+ }
+
+ /* Initialize RXQ_MCAST list */
+ for (i = 0; i < FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(i),
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE, i) |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES));
+ }
+ /* Initialize TXQ list */
+ for (i = 0; i < FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_HEAD_PERQ_HEAD, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL0_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL0_PERQ_TAIL, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL1_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL1_PERQ_TAIL, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_TXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES));
+ }
+ /* Initialize free segment list */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_SSCHED_RX_PERPORT(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SSCHED_RX_PERPORT_NEXT, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_FREELIST_INIT_ENTRIES -
+ FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_LOGICAL_PORTS_MAX));
+ }
+ /* Disable switch scan chain */
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCAN_DATA_IN,
+ FM10K_SW_SCAN_DATA_IN_UPDATE_NODES |
+ FM10K_SW_SCAN_DATA_IN_PASSTHRU);
+
+ error = fm10k_switch_configure_epls(hw, sw);
+ if (error)
+ goto done;
+
+ /*
+ * for now configure store-and-forward between PEPs and external
+ * ports regardless of relative speeds
+ */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SAF_MATRIX(all_lports[i].lport),
+ FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS);
+ }
+
+ /* Disable MTU violation trap */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SYS_CFG_1);
+ data &= ~FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS;
+ fm10k_write_switch_reg(sw, FM10K_SW_SYS_CFG_1, data);
+
+ /* Disable ingress VLAN filtering and learning */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_3(all_lports[i].lport), 0);
+ }
+ /*
+ * Make all ports members of every VLAN, and configure every VLAN to
+ * use MST instance 0.
+ */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ fm10k_write_switch_reg128(sw, FM10K_SW_INGRESS_VID_TABLE(i),
+ FM10K_SW_INGRESS_VID_TABLE_REFLECT, data64);
+ }
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_EGRESS_VID_TABLE_ENTRIES; i++)
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_EGRESS_VID_TABLE(i), 0, data64);
+
+ /* Init MOD_VLAN_TAG_VID1_MAP */
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ data64 = i;
+ data64 = data64 << 48;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * i + 0x20000, data64);
+ }
+
+ /* Configure MST instance 0 to forward for all ports */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_MST_TABLE_FORWARDING
+ (all_lports[i].lport);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_EGRESS_MST_TABLE(0), data64);
+
+ data64 = 0;
+ data64_2 = 0;
+ for (i = 0; i < num_lports; i++) {
+ if (all_lports[i].lport <
+ FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) {
+ data64 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ } else {
+ data64_2 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ }
+ }
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(0, 0), data64);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(1, 0), data64_2);
+
+ for (i = 0; i < num_lports; i++) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(0) |
+ FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport), data64);
+
+ /*
+ * Configure tags for f-tagged lports
+ */
+ if (all_lports[i].has_ftag) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport),
+ data64);
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_MOD_PER_PORT_CFG_2_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport),
+ data64);
+ }
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3;
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport), data64);
+ }
+
+ /*
+ * Assign default SGLORTs for the EPL ports in the PARSER config.
+ * This isn't necessary for the PEPs as for those, as all frames
+ * ingressing from PEPs are tagged with an SGLORT that is derived
+ * from a per-tx-queue setting.
+ *
+ * The PORT_CFG_ISL register offset is determined by logical port
+ * number and the register contents determined by the corresponding
+ * SGLORT.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_ISL
+ (fm10k_sw.epl_map[i].logical_port),
+ FM10K_SW_MAKE_REG_FIELD
+ (PORT_CFG_ISL_SGLORT,
+ fm10k_sw.epl_map[i].glort));
+ }
+
+ /*
+ * FFU counter, 11.10.3.5 POLICER_CFG[0..3]
+ */
+ fm10k_write_switch_reg64(sw,
+ 0xE40000 + 0x2 * FM10K_SW_FFU_CNT_BANK + 0x11000,
+ FM10K_SW_POLICER_LAST);
+
+ /*
+ * 11.10.3.1 POLICER_CFG_4K[0..1][0..4095]
+ * 11.10.3.2 POLICER_CFG_512[0..1][0..511]
+ */
+ for (i = 0; i < FM10K_SW_FFU_CNT_MAX; i++) {
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x2000 * FM10K_SW_FFU_CNT_BANK +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x0);
+ } else {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x400 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x4000);
+ }
+ }
+
+ /*
+ * FFU
+ *
+ * The FFU is programmed to match on SGLORT and to set the DGLORT to
+ * a unique value corresponding to the given SGLORT. One TCAM entry
+ * is required per host port per PEP and one per external interface.
+ * Only slice 0 is used.
+ */
+ if (fm10k_ffu_init(sw, sw->dpdk_cfg) < 0)
+ return -1;
+
+ /*
+ * Program the segment scheduler (see tables at top)
+ *
+ * The TX and RX schedules are the same. Page 0 is used.
+ */
+ if (cfg->ext_port_speed == 40 || cfg->ext_port_speed == 100)
+ is_quad = 1;
+ else
+ is_quad = 0;
+ for (i = 0;
+ i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]);
+ i++) {
+ /*
+ * In addition to explicit idle cycles, non-quad port
+ * entries are converted to idle cycles if the interfaces
+ * are configured in quad-port mode.
+ */
+ if (fm10k_sched_prog[i].idle ||
+ (!fm10k_sched_prog[i].quad && is_quad))
+ data = FM10K_SW_SCHED_SCHEDULE_IDLE;
+ else
+ data = FM10K_SW_SCHED_SCHEDULE_ENTRY
+ (fm10k_sched_prog[i].phys,
+ fm10k_sched_prog[i].log,
+ (fm10k_sched_prog[i].quad == FM10K_SW_QUAD_40_100) ?
+ is_quad : fm10k_sched_prog[i].quad);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RX_SCHEDULE(0, i), data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TX_SCHEDULE(0, i), data);
+ }
+
+ fm10k_write_switch_reg(sw, FM10K_SW_SCHED_SCHEDULE_CTRL,
+ FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_RX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1) |
+ FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_TX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1));
+
+ /* Per 5.7.10.4 */
+ watermark = FM10K_SW_MEM_POOL_SEGS_MAX -
+ ((sw->info->num_peps +
+ FM10K_SW_EPLS_SUPPORTED * FM10K_SW_EPL_LANES) *
+ FM10K_SW_HOWMANY(FM10K_SW_PACKET_SIZE_MAX,
+ FM10K_SW_MEM_POOL_SEG_SIZE, FM10K_SW_MEM_POOL_SEG_SIZE)) -
+ FM10K_SW_MEM_POOL_SEGS_RSVD;
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_WM,
+ FM10K_SW_MAKE_REG_FIELD(CM_GLOBAL_WM_WATERMARK, watermark));
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_CFG,
+ FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN |
+ FM10K_SW_MAKE_REG_FIELD
+ (CM_GLOBAL_CFG_NUM_SWEEPER_PORTS,
+ FM10K_SW_LOGICAL_PORTS_MAX));
+
+ /* Configure stats counters */
+ data = FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS;
+ data64 =
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 |
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8;
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_RX_STATS_CFG(all_lports[i].lport),
+ data |
+ ((all_lports[i].has_ftag) ?
+ FM10K_SW_MAKE_REG_FIELD
+ (RX_STATS_CFG_PER_FRAME_ADJUSTMENT,
+ FM10K_SW_FTAG_SIZE) :
+ 0));
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_STATS_CFG(all_lports[i].lport),
+ data64);
+ }
+
+ /* Transition switch to ready */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+
+ done:
+ return (error);
+}
+
+
+static void
+fm10k_switch_leds_init(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t addr;
+ uint32_t data;
+ int max;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ /* MODE1 - put into sleep mode to ensure it is brought out
+ * of sleep without violating the oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x00);
+
+ /* Wait for oscillator to stabilize after coming out of sleep */
+ fm10k_udelay(500);
+
+ /* MODE2 - group control is blinking, open drain outputs,
+ * OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x01, 0x20);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x02, 0xff);
+
+ /* PWM1 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x03, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x13, max - 1);
+
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ addr = 0x6a;
+ for (i = 0; i < 2; i++) {
+ /* MODE1 - put into sleep mode to ensure it is
+ * brought out of sleep without violating the
+ * oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x00);
+
+ /* MODE2 - group control is blinking, open drain
+ * outputs, OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x20);
+
+ /* Wait for oscillator to stabilize
+ * after coming out of sleep
+ */
+ fm10k_udelay(500);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x02, 0xff);
+
+ /* PWM3 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x05, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, addr, 0x13, max - 1);
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x88);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * PCA9538 initialization
+ * only differences from defaults are noted
+ */
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x65;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0xf0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x66;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ /* set LEC_CFG */
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+
+ /* port from rdifd, LED */
+ fm10k_gpio_output_set(sw, 4, 0);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to operate LEDs for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+
+static unsigned int
+fm10k_switch_pca9635_led_bits(uint8_t led_flags)
+{
+ unsigned int bits;
+
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE)
+ bits = 0x3; /* group blink */
+ else
+ bits = 0x1; /* full on */
+ } else {
+ bits = 0; /* off */
+ }
+ return (bits);
+}
+
+static void
+fm10k_switch_process_leds(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_ports *ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ unsigned int num_ports = ports->num_ports;
+ uint32_t data;
+ uint8_t update_port[num_ports];
+ uint8_t led_flags = 0, read;
+ uint8_t addr;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ if (sw->master_hw->sw_addr == NULL) {
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ return;
+ }
+
+ for (i = 0; i < num_ports; i++) {
+ port = &ports->ports[i];
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_EPL_LED_STATUS(port->eplno));
+ led_flags =
+ ((data & FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP
+ (port->first_lane)) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_UP : 0) |
+ ((data &
+ (FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING
+ (port->first_lane) |
+ FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING
+ (port->first_lane))) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE : 0);
+ update_port[i] = (led_flags != port->last_led_flags);
+ port->last_led_flags = led_flags;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ if (!(update_port[0] || update_port[1])) {
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (ports->ports[0].last_led_flags) |
+ (fm10k_switch_pca9635_led_bits
+ (ports->ports[1].last_led_flags) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs operating
+ * as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (update_port[i] == 0)
+ continue;
+
+ addr = (i == 0) ? 0x6a : 0x69;
+
+ port = &ports->ports[i];
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (port->last_led_flags);
+
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ led_flags = 0;
+ addr = 0x62;
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ case 25:
+ led_flags |= 0x6 << (4 * i); /* 100G */
+ break;
+ case 40:
+ case 10:
+ led_flags |= 0x4 << (4 * i); /* 40G */
+ break;
+ default:
+ led_flags = 0;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ }
+
+ if (update_port[0] || update_port[1]) {
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ if (update_port[0])
+ led_flags |= read & 0xf0;
+ else
+ led_flags |= read & 0xf;
+ fm10k_i2c_write16(sw->i2c, addr, 0x1, led_flags);
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = (i == 0) ? 0x62 : 0x66;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ led_flags = 0x08; /* 100G */
+ break;
+ case 40:
+ led_flags = 0x02; /* 40G */
+ break;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x1, led_flags);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+static void *
+fm10k_switch_leds_update(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ while (sw->detaching == 0) {
+ fm10k_switch_process_leds(ctx);
+ usec_delay(FM10K_LED_POLL_INTERVAL_MS * 1000);
+ }
+ return NULL;
+}
+
+static void
+fm10k_switch_leds_off(struct fm10k_switch *sw, struct fm10k_ext_ports *ports)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ uint8_t led_flags;
+ uint8_t addr;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits(0) |
+ (fm10k_switch_pca9635_led_bits(0) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ addr = 0x6a;
+ led_flags = fm10k_switch_pca9635_led_bits(0);
+ for (i = 0; i < 2; i++) {
+ port = &ports->ports[i];
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+
+static void *
+fm10k_switch_process_intr(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_ext_ports *ports;
+ uint64_t gid;
+ uint64_t reenable_mask;
+ uint16_t epl_mask;
+
+ while (sw->detaching == 0) {
+ sem_wait(&sw->intr_tq);
+ /*
+ * Mask off all global interrupt detect interrupts
+ * toward PCIe to ensure that the PEP sees the
+ * interrupt condition clear so that all subsequent
+ * events will be recognized. The same result could
+ * be achieved by masking off all block-internal
+ * interrupt sources in all blocks prior to handling
+ * any global interrupt detect interrupts, but the
+ * approach taken here makes the interrupt processing
+ * easier to decompose and increases the likelihood that
+ * more work will be coalesced into fewer interrupts.
+ */
+ FM10K_SW_TRACE("masking off PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ /*
+ * sw->ext_ports will be set to NULL during detach
+ * to indicate that we are to no longer process EPL
+ * interrupts.
+ */
+ ports = sw->ext_ports;
+ /*
+ * sw->sm_mailbox_enabled will be set to 0 during
+ * detach to indicate that we are to no longer process
+ * PEP interrupts.
+ */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ FM10K_SW_INTERRUPT_MASK_PCIE_ALL);
+ gid = fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ reenable_mask = 0;
+ if (ports) {
+ epl_mask = fm10k_ext_ports_epl_intrs(ports, gid);
+ reenable_mask |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (INTERRUPT_MASK_PCIE_EPL, epl_mask);
+ }
+
+ if (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C)
+ fm10k_i2c_intr(sw->i2c);
+
+ reenable_mask |= FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+
+ FM10K_SW_TRACE("re-enabling PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ ~reenable_mask);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ }
+ return NULL;
+}
+
+void
+fm10k_switch_intr(struct fm10k_hw *hw)
+{
+ if (hw)
+ sem_post(&fm10k_sw.intr_tq);
+}
+
+
+static void
+fm10k_switch_enable_interrupts(struct fm10k_switch *sw)
+{
+ uint64_t data64;
+ unsigned int i;
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno));
+
+ /* enable interrupts from all EPLs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_EPL,
+ ~sw->ext_ports->epl_mask, data64);
+
+ /* enable interrupts from all non-master PEPs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_PCIE,
+ ~sw->pep_mask, data64);
+
+ /* enable I2C interrupt */
+ data64 &= ~FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno), data64);
+
+ /* enable outbound mailbox interrupts from all non-master PEPs */
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (sw->pep_mask & (1 << i))
+ fm10k_write_switch_reg(sw, FM10K_SW_PCIE_GLOBAL(i, IM),
+ ~FM10K_SW_IM_MAILBOX);
+ }
+ FM10K_WRITE_REG(sw->master_hw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_FIELD(SWITCH_INT, ENABLE));
+}
+
+
+static void
+fm10k_switch_detach(struct fm10k_hw *hw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_switch *sw = &fm10k_sw;
+
+ if (hw == NULL || sw == NULL)
+ return;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->detaching = 1;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (sw->ext_ports) {
+ ports = sw->ext_ports;
+ /*
+ * Ensure that any further switch interrupts will not
+ * process sw->ext_ports before detaching them.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->ext_ports = NULL;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_switch_leds_off(sw, ports);
+ fm10k_ext_ports_detach(ports);
+ }
+
+ if (sw->epl_sbus)
+ fm10k_sbus_detach(sw->epl_sbus);
+
+ /*
+ * Detach i2c after ext_ports so ext_ports can use i2c to disable
+ * phys.
+ */
+ if (sw->i2c)
+ fm10k_i2c_detach(sw->i2c);
+
+ fm10k_hw_eicr_disable_source(sw, FM10K_SW_EICR_SWITCH_INT);
+
+ pthread_join(sw->intr_task, NULL);
+ pthread_join(sw->led_task, NULL);
+ pthread_join(sw->stats_task, NULL);
+}
+
+
+static unsigned int
+fm10k_switch_get_fabric_clock(struct fm10k_switch *sw)
+{
+ uint32_t pll_fabric;
+ unsigned int freq;
+
+ pll_fabric = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_LOCK);
+
+ switch (FM10K_SW_REG_FIELD(pll_fabric, PLL_FABRIC_FREQSEL)) {
+ case FM10K_SW_PLL_FABRIC_FREQSEL_CTRL: {
+ uint32_t ctrl;
+ unsigned int refdiv, fbdiv4, fbdiv255;
+
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_CTRL);
+ refdiv = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_REFDIV);
+ fbdiv4 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV4);
+ fbdiv255 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV255);
+
+ freq = (15625 * 4 * fbdiv255 * (1 + fbdiv4)) / (refdiv * 100);
+ break;
+ }
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F600:
+ freq = 600;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F500:
+ freq = 500;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F400:
+ freq = 400;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F300:
+ freq = 300;
+ break;
+ default:
+ freq = 0;
+ break;
+ }
+
+ return freq;
+}
+
+static unsigned int
+fm10k_switch_get_sku(struct fm10k_switch *sw)
+{
+ uint32_t fuse_data;
+
+ fuse_data = fm10k_read_switch_reg(sw, FM10K_SW_FUSE_DATA_0);
+ return FM10K_SW_REG_FIELD(fuse_data, FUSE_SKU);
+}
+
+static const char *
+fm10k_switch_get_sku_name(unsigned int sku)
+{
+ const char *name;
+
+ switch (sku) {
+ case FM10K_SW_FUSE_SKU_FM10840:
+ name = "FM10840";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10420:
+ name = "FM10420";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10064:
+ name = "FM10064";
+ break;
+ default:
+ name = "FM10xxx-unknown";
+ break;
+ }
+
+ return name;
+}
+
+
+static void
+fm10k_switch_describe(struct fm10k_switch *sw)
+{
+ unsigned int i, pair;
+ unsigned int reset_state, active_state;
+ uint32_t device_cfg, resets;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("switch: device is %s, fabric clock %u MHz",
+ fm10k_switch_get_sku_name(fm10k_switch_get_sku(sw)),
+ fm10k_switch_get_fabric_clock(sw));
+
+ device_cfg = fm10k_read_switch_reg(sw, FM10K_SW_DEVICE_CFG);
+ FM10K_SW_INFO("switch: 100G Ethernet is %s",
+ (device_cfg & FM10K_SW_DEVICE_CFG_PCIE_100G_DIS) ?
+ "disabled" : "enabled");
+ switch (FM10K_SW_REG_FIELD(device_cfg, DEVICE_CFG_FEATURE)) {
+ case FM10K_SW_DEVICE_CFG_PCIE_FULL:
+ FM10K_SW_INFO("switch: PEPs 0-7 support 2x4 and 1x8 "
+ "modes, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_HALF:
+ FM10K_SW_INFO("switch: PEPs 0-3 support 2x4 and 1x8 "
+ "modes, PEPs 4 and 6 are 1x4, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_BASIC:
+ FM10K_SW_INFO("switch: PEP 0 supports, PEP 8 is x1, "
+ "only one can be used");
+ break;
+ }
+
+ resets = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (!(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_EN(i)))
+ continue;
+
+ pair = i / 2;
+ reset_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_RESET(i));
+ active_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_ACTIVE(i));
+ if (pair < FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS &&
+ !(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(pair))) {
+ if (i % 2 == 0)
+ FM10K_SW_INFO("switch: PEP[%u,%u] is enabled in 1x8 "
+ "mode (Reset=%u, Active=%u)", i, i + 1,
+ reset_state, active_state);
+ else
+ FM10K_SW_INFO("switch: PEP[%u] unexpectedly enabled when "
+ "PEP[%u,%u] is in 1x8 mode (Reset=%u, "
+ "Active=%u)", i, i - 1, i, reset_state,
+ active_state);
+ } else {
+ FM10K_SW_INFO("switch: PEP[%u] is enabled in 1x4 "
+ "mode (Reset=%u, Active=%u)", i, reset_state,
+ active_state);
+ }
+ }
+}
+
+
+static struct fm10k_switch *
+fm10k_switch_attach(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ int i;
+ int error = 0;
+
+ /*
+ * apparently no way to determine one's own PEP number.
+ */
+ switch (sw->info->num_peps) {
+ case 1:
+ sw->pepno = 0;
+ break;
+
+ case 2:
+ /*
+ * assumption is using even numbered PEPs
+ * starting with 0, and the highest numbered PEP
+ * is the master
+ */
+ sw->pepno = 2;
+ for (i = 0; i < sw->info->num_peps - 1; i++)
+ sw->pep_mask |= (1 << (i * 2));
+ break;
+
+ case 4:
+ sw->pepno = 6;
+ sw->pep_mask = 0x7f;
+ break;
+
+ default:
+ sw->pepno = 0;
+ }
+
+ /* When not zero, initialize serdes in 'near loopback' mode */
+ sw->serdes_loopback = 0;
+
+ pthread_mutex_init(&sw->lock, NULL);
+
+ fm10k_switch_determine_epls(sw);
+
+ sw->ext_ports = fm10k_ext_ports_attach(sw);
+ if (sw->ext_ports == NULL)
+ goto fail;
+
+ /* label all external ports with their logical port numbers */
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ sw->ext_ports->ports[i].lport = sw->epl_map[i].logical_port;
+
+ /* interrupt */
+ sem_init(&sw->intr_tq, 0, 0);
+ pthread_create(&sw->intr_task, NULL, fm10k_switch_process_intr, sw);
+
+ fm10k_switch_enable_interrupts(sw);
+ fm10k_switch_describe(sw);
+
+ sw->i2c = fm10k_i2c_attach(sw);
+ if (sw->i2c == NULL)
+ goto fail;
+
+ error = fm10k_switch_init(hw, sw);
+ if (error)
+ goto fail;
+
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ fm10k_ext_port_up(&sw->ext_ports->ports[i]);
+
+ usec_delay(10000);
+ fm10k_switch_leds_init(sw);
+
+ pthread_create(&sw->led_task, NULL, &fm10k_switch_leds_update, sw);
+ pthread_create(&sw->stats_task, NULL, &fm10k_switch_process_stats, sw);
+ sw->inited = 1;
+ return sw;
+fail:
+ if (sw != NULL)
+ fm10k_switch_detach(hw);
+ return NULL;
+}
+
+struct fm10k_switch*
+fm10k_switch_get(void)
+{
+ return &fm10k_sw;
+}
+
+static int
+fm10k_switch_dpdk_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].pf_no == pf_no)
+ return i;
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_mapped_ext_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ if (map->map_no[0] == pf_no)
+ return i;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ if (map->map_no[0] == pf_no || map->map_no[1] == pf_no)
+ return i;
+ }
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_start(struct fm10k_switch *sw,
+ struct fm10k_hw *master_hw, eth_fm10k_dev_init_half_func *func)
+{
+ int i, j;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_cfg_port_pf_map *dpdk_map;
+ int dpdk_port_no, ext_port_no = 0;
+ int pf_no;
+
+ sw->info = fm10k_get_device_info(master_hw);
+ sw->master_hw = master_hw;
+ sw->pep_map = fm10k_pep_port_map;
+ sw->epl_map = fm10k_epl_port_map;
+
+ sw->info->ext_port_speed = sw->dpdk_cfg->ext_port_speed;
+ fm10k_switch_set_sched_prog();
+ fm10k_switch_attach(master_hw, &fm10k_sw);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type != FM10K_CONFIG_DPDK_PF ||
+ map->type != FM10K_CONFIG_PORT_MAP_PFS)
+ continue;
+
+ for (j = 0; j < 2; j++) {
+ pf_no = map->map_no[j];
+ dpdk_port_no =
+ fm10k_switch_dpdk_port_get(sw, pf_no);
+ dpdk_map =
+ &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ dpdk_map->type = FM10K_CONFIG_PORT_MAP_PFSS;
+ dpdk_map->map_no[0] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ dpdk_map->map_no[1] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1];
+ }
+ }
+ }
+
+ /* do initialize all ports, after switch is ready */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ uint32_t sglort;
+ struct fm10k_dpdk_port *port = &sw->dpdk_cfg->ports[i];
+ struct fm10k_hw *hw = port->hw;
+
+ if (hw == NULL)
+ break;
+
+ map = &sw->dpdk_cfg->ext_port_map[ext_port_no];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_pf_glort_get(pf_no);
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ ext_port_no =
+ fm10k_switch_mapped_ext_port_get
+ (sw, pf_no);
+ sglort =
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]);
+ } else {
+ FM10K_SW_ERR("Unknown mapped port type %d!",
+ port->type);
+ return -1;
+ }
+ hw->mac.dglort_map = sglort | 0xffff0000;
+ }
+ func(hw);
+ }
+ return 0;
+}
+
+
+static int
+fm10k_switch_dpdk_port_reg(struct fm10k_switch *sw,
+ struct fm10k_hw *hw, void *rte_dev, uint8_t is_pf, bool master)
+{
+ int i;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg == NULL && is_pf) {
+ if (fm10k_config_init(sw, hw) < 0)
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ if (master && port->hw &&
+ port->type == FM10K_CONFIG_DPDK_PF) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (hw->mac.addr[5] - port->hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ if (port->hw == NULL) {
+ port->hw = hw;
+ port->rte_dev = rte_dev;
+ fm10k_flow_list_init(&port->flow_list);
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (is_pf) {
+ sw->dpdk_cfg->pf_bind++;
+ port->type = FM10K_CONFIG_DPDK_PF;
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ } else {
+ port->type = FM10K_CONFIG_DPDK_VF;
+ }
+ if (master)
+ sw->dpdk_cfg->master_hw = hw;
+ if (sw->dpdk_cfg->master_hw) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (sw->dpdk_cfg->master_hw->mac.addr[5] -
+ hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ return 0;
+ }
+ }
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == NULL)
+ break;
+
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == hw)
+ return i;
+ }
+ return -1;
+}
+
+void *
+fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].rte_dev;
+}
+
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].flow_list;
+}
+
+static int
+fm10k_switch_dpdk_cfg_check(struct fm10k_switch *sw)
+{
+ int i;
+ bool need_default = true;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg->master_hw == NULL) {
+ FM10K_SW_ERR("Master PF is not bound!!!");
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF &&
+ map->type != FM10K_CONFIG_PORT_MAP_NULL) {
+ need_default = false;
+ break;
+ }
+ }
+
+ if (need_default) {
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ }
+ }
+
+ return 0;
+}
+
+
+static void
+fm10k_switch_dpdk_cfg_describe(struct fm10k_switch *sw)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K DYNAMIC CONFIG ---\n");
+ FM10K_SW_INFO(" PF Bind : %d\n", sw->dpdk_cfg->pf_bind);
+
+ FM10K_SW_INFO("--- PF ---\n");
+ for (i = 0; i < FM10K_SW_PEP_PORTS_MAX; i++) {
+ uint8_t *mac;
+ struct rte_eth_dev *dev;
+ struct rte_pci_device *pdev;
+
+ if (sw->dpdk_cfg->pf_hw[i] == NULL)
+ continue;
+ dev = (struct rte_eth_dev *)
+ fm10k_switch_dpdk_port_rte_dev_get
+ (sw->dpdk_cfg->pf_hw[i]);
+ pdev = RTE_ETH_DEV_TO_PCI(dev);
+ mac = sw->dpdk_cfg->pf_hw[i]->mac.addr;
+ FM10K_SW_INFO(" PF%d : Logical %d Glort %#x "
+ "PCI Addr %02x:%02x.%x "
+ "MAC Addr %02x:%02x:%02x:%02x:%02x:%02x\n",
+ i, fm10k_pep_port_map[i].logical_port,
+ fm10k_switch_pf_glort_get(i), pdev->addr.bus,
+ pdev->addr.devid,
+ pdev->addr.function,
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+ }
+
+ FM10K_SW_INFO("--- DPDK PORT ---\n");
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].type == FM10K_CONFIG_DPDK_NULL)
+ break;
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ break;
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d\n",
+ i, map->map_no[0]);
+ } else {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d&PF%d Glort %#x\n",
+ i,
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+ }
+
+ FM10K_SW_INFO("--- EXT PORT ---\n");
+ for (i = 0; i < sw->dpdk_cfg->ext_port_num; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ FM10K_SW_INFO(" EXT PORT%d : Logical %d Glort %#x", i,
+ fm10k_epl_port_map[i].logical_port,
+ fm10k_switch_epl_glort_get(i));
+
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ FM10K_SW_INFO("\n");
+ else if (map->type == FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" Map to PF%d\n", map->map_no[0]);
+ else
+ FM10K_SW_INFO(" Map to PF%d&PF%d Glort %#x\n",
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+}
+
+
+int
+fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func)
+{
+ int ret;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (fm10k_switch_dpdk_port_reg(sw, hw, rte_dev, is_pf, master) != 0) {
+ FM10K_SW_ERR("Register ports failed!!!");
+ return -1;
+ }
+
+ /*
+ * After all pfs are started
+ * start switch here
+ */
+ if (fm10k_sw.dpdk_cfg->pf_max != 0 &&
+ fm10k_sw.dpdk_cfg->pf_bind == fm10k_sw.dpdk_cfg->pf_num) {
+ if (fm10k_switch_dpdk_cfg_check(&fm10k_sw) != 0)
+ return -1;
+
+ ret = fm10k_switch_start(&fm10k_sw,
+ fm10k_sw.dpdk_cfg->master_hw, func);
+ fm10k_switch_dpdk_cfg_describe(&fm10k_sw);
+ return ret;
+ }
+ return 0;
+}
+
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw)
+{
+ if (hw)
+ return;
+}
+
+
+/*
+ * for multi-host, only dpdk port 0 and 1 are real rx/tx packets
+ * so we need init/setup/start dpdk port queue for them.
+ * for dpdk port 2/3, we need init/setup except start.
+ * we map the pf queue to 0/1 dpdk queue first, then map
+ * the other pf queue to 2/3 dpdk queue.
+ */
+int
+fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue)
+{
+ int idx, pf_no;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ FM10K_SW_ERR("Not support yet!!!");
+ return -1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[0];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue / 2;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = (max_queue + 1) / 2 + queue / 2;
+ return 0;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_NULL) {
+ FM10K_SW_ERR("Unmapped dpdk port %d!!!", dpdk_port_no);
+ return -1;
+ }
+ FM10K_SW_ERR("Unknown mapped type!!!");
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw, struct fm10k_hw *hw_list[])
+{
+ int pf_no, pf_no_ext;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+ map = &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ hw_list[0] = hw;
+ hw_list[1] = NULL;
+ return 1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ hw_list[1] = NULL;
+ return 1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 2;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFSS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 0;
+ }
+
+ hw_list[0] = NULL;
+ hw_list[1] = NULL;
+ return 0;
+}
+
+void
+fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].tx_queue_num = num;
+}
+
+void
+fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].rx_queue_num = num;
+}
+
+int
+fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ return sw->dpdk_cfg->ports[port_no].pf_no;
+}
+
+
+void
+fm10k_switch_flowset_switchto(const char *name)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_ffu_flowset_switch(sw, name);
+}
+
+void
+fm10k_switch_show_port(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
+}
+
+void
+fm10k_switch_show_ffu(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_ffu_count_print(sw);
+}
+
+void
+fm10k_switch_show_bank(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_port_bank_print(sw);
+}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
` (3 preceding siblings ...)
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
@ 2020-02-28 8:38 ` Xiaojun Liu
2020-03-16 7:34 ` Wang, Xiao W
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
4 siblings, 2 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-02-28 8:38 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Split dev init to 2 parts.
First only register the port in switch
management; second init hook will be
called after all the pf are registered
and switch initialization. It will finish
dev init. Also add switch interrupt support.
Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
to support mirror operation. Add fm10k_dev_filter_ctrl
to support flow operation.
Add dpdk port and pf mapping, so
the dpdk port can map to a specific pf
and 1 dpdk port can map to 2 pf to get
total 100G throughput.
To enable the switch management, you need add
CONFIG_RTE_FM10K_MANAGEMENT=y in
config/common_linux when building.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/base/fm10k_type.h | 1 +
drivers/net/fm10k/fm10k_ethdev.c | 562 +++++++++++++++++++++++++++++---
drivers/net/fm10k/switch/fm10k_switch.c | 20 +-
drivers/net/fm10k/switch/fm10k_switch.h | 1 +
4 files changed, 541 insertions(+), 43 deletions(-)
diff --git a/drivers/net/fm10k/base/fm10k_type.h b/drivers/net/fm10k/base/fm10k_type.h
index 84781ba..810e5ee 100644
--- a/drivers/net/fm10k/base/fm10k_type.h
+++ b/drivers/net/fm10k/base/fm10k_type.h
@@ -717,6 +717,7 @@ struct fm10k_hw {
u16 subsystem_vendor_id;
u8 revision_id;
u32 flags;
+ bool switch_ready;
#define FM10K_HW_FLAG_CLOCK_OWNER BIT(0)
};
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 581c690..f11fa53 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -13,6 +13,10 @@
#include "fm10k.h"
#include "base/fm10k_api.h"
+#ifdef RTE_FM10K_MANAGEMENT
+#include "switch/fm10k_regs.h"
+#include "switch/fm10k_switch.h"
+#endif
/* Default delay to acquire mailbox lock */
#define FM10K_MBXLOCK_DELAY_US 20
@@ -461,9 +465,6 @@ struct fm10k_xstats_name_off {
PMD_INIT_FUNC_TRACE();
- if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG)
- dev->data->dev_conf.rxmode.offloads |= DEV_RX_OFFLOAD_RSS_HASH;
-
/* multipe queue mode checking */
ret = fm10k_check_mq_mode(dev);
if (ret != 0) {
@@ -512,6 +513,15 @@ struct fm10k_xstats_name_off {
struct rte_eth_conf *dev_conf = &dev->data->dev_conf;
uint32_t mrqc, *key, i, reta, j;
uint64_t hf;
+#ifdef RTE_FM10K_MANAGEMENT
+ uint16_t nb_rx_queues = dev->data->nb_rx_queues;
+ int mapped_num;
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num == 2)
+ nb_rx_queues /= 2;
+#endif
#define RSS_KEY_SIZE 40
static uint8_t rss_intel_key[RSS_KEY_SIZE] = {
@@ -642,26 +652,44 @@ struct fm10k_xstats_name_off {
fm10k_dev_tx_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ uint32_t data;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_tx_queue *txq;
uint64_t base_addr;
uint32_t size;
+#ifndef RTE_FM10K_MANAGEMENT
/* Disable TXINT to avoid possible interrupt */
for (i = 0; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_TXINT(i),
3 << FM10K_TXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_tx_queue_num_set(unmap_hw,
+ dev->data->nb_tx_queues);
+#endif
/* Setup TX queue */
for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef RTE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id);
+#endif
txq = dev->data->tx_queues[i];
base_addr = txq->hw_ring_phys_addr;
size = txq->nb_desc * sizeof(struct fm10k_tx_desc);
/* disable queue to avoid issues while updating state */
- ret = tx_queue_disable(hw, i);
+ ret = tx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Enable use of FTAG bit in TX descriptor, PFVTCTL
@@ -669,7 +697,7 @@ struct fm10k_xstats_name_off {
*/
if (fm10k_check_ftag(dev->device->devargs)) {
if (hw->mac.type == fm10k_mac_pf) {
- FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_PFVTCTL(hw_queue_id),
FM10K_PFVTCTL_FTAG_DESC_ENABLE);
PMD_INIT_LOG(DEBUG, "FTAG mode is enabled");
} else {
@@ -679,15 +707,25 @@ struct fm10k_xstats_name_off {
}
/* set location and size for descriptor ring */
- FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_TDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_TDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_TDLEN(hw_queue_id), size);
/* assign default SGLORT for each TX queue by PF */
+#ifndef RTE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf)
- FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(i), hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw,
+ FM10K_TX_SGLORT(hw_queue_id),
+ hw->mac.dglort_map);
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ data = FM10K_SW_MAKE_REG_FIELD
+ (TX_SGLORT_SGLORT, hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(hw_queue_id), data);
+ }
+#endif
}
/* set up vector or scalar TX function as appropriate */
@@ -700,18 +738,24 @@ struct fm10k_xstats_name_off {
fm10k_dev_rx_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifndef RTE_FM10K_MANAGEMENT
struct fm10k_macvlan_filter_info *macvlan;
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+ uint32_t logic_port = hw->mac.dglort_map;
+ uint16_t queue_stride = 0;
+#else
+ struct fm10k_hw *unmap_hw = hw;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_rx_queue *rxq;
uint64_t base_addr;
uint32_t size;
uint32_t rxdctl = FM10K_RXDCTL_WRITE_BACK_MIN_DELAY;
- uint32_t logic_port = hw->mac.dglort_map;
uint16_t buf_size;
- uint16_t queue_stride = 0;
+#ifndef RTE_FM10K_MANAGEMENT
/* enable RXINT for interrupt mode */
i = 0;
if (rte_intr_dp_is_en(intr_handle)) {
@@ -731,26 +775,36 @@ struct fm10k_xstats_name_off {
for (; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_RXINT(i),
3 << FM10K_RXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_rx_queue_num_set(unmap_hw, dev->data->nb_rx_queues);
+#endif
/* Setup RX queues */
for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef RTE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_rx_queues, &hw, &hw_queue_id);
+#endif
rxq = dev->data->rx_queues[i];
base_addr = rxq->hw_ring_phys_addr;
size = rxq->nb_desc * sizeof(union fm10k_rx_desc);
/* disable queue to avoid issues while updating state */
- ret = rx_queue_disable(hw, i);
+ ret = rx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Setup the Base and Length of the Rx Descriptor Ring */
- FM10K_WRITE_REG(hw, FM10K_RDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_RDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_RDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_RDLEN(hw_queue_id), size);
/* Configure the Rx buffer size for one buff without split */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
@@ -764,7 +818,7 @@ struct fm10k_xstats_name_off {
*/
buf_size -= FM10K_RX_DATABUF_ALIGN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id),
(buf_size >> FM10K_SRRCTL_BSIZEPKT_SHIFT) |
FM10K_SRRCTL_LOOPBACK_SUPPRESS);
@@ -774,9 +828,9 @@ struct fm10k_xstats_name_off {
rxq->offloads & DEV_RX_OFFLOAD_SCATTER) {
uint32_t reg;
dev->data->scattered_rx = 1;
- reg = FM10K_READ_REG(hw, FM10K_SRRCTL(i));
+ reg = FM10K_READ_REG(hw, FM10K_SRRCTL(hw_queue_id));
reg |= FM10K_SRRCTL_BUFFER_CHAINING_EN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i), reg);
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id), reg);
}
/* Enable drop on empty, it's RO for VF */
@@ -796,6 +850,7 @@ struct fm10k_xstats_name_off {
/* update RX_SGLORT for loopback suppress*/
if (hw->mac.type != fm10k_mac_pf)
return 0;
+#ifndef RTE_FM10K_MANAGEMENT
macvlan = FM10K_DEV_PRIVATE_TO_MACVLAN(dev->data->dev_private);
if (macvlan->nb_queue_pools)
queue_stride = dev->data->nb_rx_queues / macvlan->nb_queue_pools;
@@ -804,6 +859,7 @@ struct fm10k_xstats_name_off {
logic_port++;
FM10K_WRITE_REG(hw, FM10K_RX_SGLORT(i), logic_port);
}
+#endif
return 0;
}
@@ -812,12 +868,29 @@ struct fm10k_xstats_name_off {
fm10k_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ int ret;
+#endif
int err;
uint32_t reg;
struct fm10k_rx_queue *rxq;
+ uint16_t hw_queue_id = rx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ rx_queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and rx queue for:"
+ " %s %d", dev->device->name, rx_queue_id);
+ return -EIO;
+ } else if (ret != 1) /* reference port's queue don't need start */
+ return 0;
+#endif
+
rxq = dev->data->rx_queues[rx_queue_id];
err = rx_queue_reset(rxq);
if (err == -ENOMEM) {
@@ -836,23 +909,23 @@ struct fm10k_xstats_name_off {
* this comment and the following two register writes when the
* emulation platform is no longer being used.
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
/* Set PF ownership flag for PF devices */
- reg = FM10K_READ_REG(hw, FM10K_RXQCTL(rx_queue_id));
+ reg = FM10K_READ_REG(hw, FM10K_RXQCTL(hw_queue_id));
if (hw->mac.type == fm10k_mac_pf)
reg |= FM10K_RXQCTL_PF;
reg |= FM10K_RXQCTL_ENABLE;
/* enable RX queue */
- FM10K_WRITE_REG(hw, FM10K_RXQCTL(rx_queue_id), reg);
+ FM10K_WRITE_REG(hw, FM10K_RXQCTL(hw_queue_id), reg);
FM10K_WRITE_FLUSH(hw);
/* Setup the HW Rx Head and Tail Descriptor Pointers
* Note: this must be done AFTER the queue is enabled
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -879,21 +952,38 @@ struct fm10k_xstats_name_off {
fm10k_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ int ret;
+#endif
/** @todo - this should be defined in the shared code */
#define FM10K_TXDCTL_WRITE_BACK_MIN_DELAY 0x00010000
uint32_t txdctl = FM10K_TXDCTL_WRITE_BACK_MIN_DELAY;
struct fm10k_tx_queue *q = dev->data->tx_queues[tx_queue_id];
+ uint16_t hw_queue_id = tx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ tx_queue_id, dev->data->nb_tx_queues, &hw, &hw_queue_id);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and tx queue for:"
+ " %s %d", dev->device->name, tx_queue_id);
+ return -EIO;
+ } else if (ret != 1) {
+ return 0;
+ }
+#endif
+
q->ops->reset(q);
/* reset head and tail pointers */
- FM10K_WRITE_REG(hw, FM10K_TDH(tx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_TDT(tx_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDT(hw_queue_id), 0);
/* enable TX queue */
- FM10K_WRITE_REG(hw, FM10K_TXDCTL(tx_queue_id),
+ FM10K_WRITE_REG(hw, FM10K_TXDCTL(hw_queue_id),
FM10K_TXDCTL_ENABLE | txdctl);
FM10K_WRITE_FLUSH(hw);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -1084,9 +1174,25 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
int i, diag;
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+ int j, mapped_num;
+ uint32_t data;
+#endif
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_MANAGEMENT
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > FM10K_SW_PEP_MAP_NUM_MAX) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW for %s",
+ dev->device->name);
+ return -EIO;
+ }
+#endif
+
+
+#ifndef RTE_FM10K_MANAGEMENT
/* stop, init, then start the hw */
diag = fm10k_stop_hw(hw);
if (diag != FM10K_SUCCESS) {
@@ -1105,6 +1211,62 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
return -EIO;
}
+#else
+ for (j = 0; j < mapped_num; j++) {
+ struct rte_pci_device *pdev =
+ RTE_ETH_DEV_TO_PCI((struct rte_eth_dev *)
+ (fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[j])));
+ struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+ /* stop, init, then start the hw */
+ diag = fm10k_stop_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware stop failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_init_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware init failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_start_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
+ return -EIO;
+ }
+
+ /* Disable TXINT to avoid possible interrupt */
+ for (i = 0; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_TXINT(i),
+ 3 << FM10K_TXINT_TIMER_SHIFT);
+
+ /* enable RXINT for interrupt mode */
+ i = 0;
+ if (rte_intr_dp_is_en(intr_handle)) {
+ for (; i < dev->data->nb_rx_queues; i++) {
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_RXINT(i), Q2V(pdev, i));
+ if (mapped_hws[j]->mac.type == fm10k_mac_pf)
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_ITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ else
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_VFITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ }
+ }
+
+ /* Disable other RXINT to avoid possible interrupt */
+ for (; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_RXINT(i),
+ 3 << FM10K_RXINT_TIMER_SHIFT);
+ }
+#endif
diag = fm10k_dev_tx_init(dev);
if (diag) {
@@ -1156,10 +1318,26 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
}
}
+#ifndef RTE_FM10K_MANAGEMENT
/* Update default vlan when not in VMDQ mode */
if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+#endif
+#ifdef RTE_FM10K_MANAGEMENT
+ /* Admit all VLANs */
+ for (j = 0; j <= 64; j++) {
+ for (i = 0; i < FM10K_SW_VLAN_TABLE_ENTRIES; i++)
+ FM10K_WRITE_REG(hw,
+ FM10K_SW_VLAN_TABLE_ENTRY(j, i),
+ 0xffffffff);
+ }
+
+ /* Disable PEP loopback */
+ data = FM10K_READ_REG(hw, FM10K_CTRL_EXT);
+ data &= ~FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK;
+ FM10K_WRITE_REG(hw, FM10K_CTRL_EXT, data);
+#endif
fm10k_link_update(dev, 0);
return 0;
@@ -1327,12 +1505,24 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
struct fm10k_hw_stats *hw_stats =
FM10K_DEV_PRIVATE_TO_STATS(dev->data->dev_private);
int i;
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+ int mapped_num;
+ uint16_t hw_queue_id;
+#endif
PMD_INIT_FUNC_TRACE();
+ ipackets = 0;
+ opackets = 0;
+ ibytes = 0;
+ obytes = 0;
+ imissed = 0;
+
+#ifndef RTE_FM10K_MANAGEMENT
fm10k_update_hw_stats(hw, hw_stats);
- ipackets = opackets = ibytes = obytes = imissed = 0;
for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
(i < hw->mac.max_queues); ++i) {
stats->q_ipackets[i] = hw_stats->q[i].rx_packets.count;
@@ -1346,6 +1536,49 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
obytes += stats->q_obytes[i];
imissed += stats->q_errors[i];
}
+#else
+ /* Get statistics from mapped device's queue */
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(unmap_hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > FM10K_SW_PEP_MAP_NUM_MAX) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW for %s",
+ dev->device->name);
+ return -EIO;
+ }
+
+ for (i = 0; i < mapped_num; i++) {
+ struct rte_eth_dev *mydev =
+ fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[i]);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS(mydev->data->dev_private);
+ fm10k_update_hw_stats(mapped_hws[i], hw_stats);
+ }
+
+ if (mapped_num) {
+ for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
+ (i < unmap_hw->mac.max_queues); ++i) {
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, unmap_hw->mac.max_queues,
+ &hw, &hw_queue_id);
+ if (mapped_hws[1]) {
+ struct rte_eth_dev *mydev;
+ mydev = fm10k_switch_dpdk_port_rte_dev_get(hw);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS
+ (mydev->data->dev_private);
+ }
+ stats->q_ipackets[i] =
+ hw_stats->q[hw_queue_id].rx_packets.count;
+ stats->q_opackets[i] =
+ hw_stats->q[hw_queue_id].tx_packets.count;
+ stats->q_ibytes[i] =
+ hw_stats->q[hw_queue_id].rx_bytes.count;
+ stats->q_obytes[i] =
+ hw_stats->q[hw_queue_id].tx_bytes.count;
+ ipackets += stats->q_ipackets[i];
+ opackets += stats->q_opackets[i];
+ ibytes += stats->q_ibytes[i];
+ obytes += stats->q_obytes[i];
+ }
+ }
+#endif
stats->ipackets = ipackets;
stats->opackets = opackets;
stats->ibytes = ibytes;
@@ -1808,8 +2041,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_JUMBO_FRAME |
- DEV_RX_OFFLOAD_HEADER_SPLIT |
- DEV_RX_OFFLOAD_RSS_HASH);
+ DEV_RX_OFFLOAD_HEADER_SPLIT);
}
static int
@@ -1818,14 +2050,28 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
const struct rte_eth_rxconf *conf, struct rte_mempool *mp)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+#endif
struct fm10k_dev_info *dev_info =
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
struct fm10k_rx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id) < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and queue for:"
+ " %s %d", dev->device->name, queue_id);
+ return -EIO;
+ }
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads;
/* make sure the mempool element size can account for alignment. */
@@ -1871,7 +2117,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
q->port_id = dev->data->port_id;
q->queue_id = queue_id;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_RDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_RDT(hw_queue_id)];
q->offloads = offloads;
if (handle_rxconf(q, conf))
return -EINVAL;
@@ -2007,12 +2253,26 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
const struct rte_eth_txconf *conf)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+#endif
struct fm10k_tx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id) < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and queue for:"
+ " %s %d", dev->device->name, queue_id);
+ return -EIO;
+ }
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.txmode.offloads;
/* make sure a valid number of descriptors have been requested */
@@ -2054,7 +2314,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
q->offloads = offloads;
q->ops = &def_txq_ops;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_TDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_TDT(hw_queue_id)];
if (handle_txconf(q, conf))
return -EINVAL;
@@ -2284,6 +2544,77 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
return 0;
}
+#ifdef RTE_FM10K_MANAGEMENT
+static int
+fm10k_mirror_rule_set(struct rte_eth_dev *dev,
+ struct rte_eth_mirror_conf *mirror_conf,
+ uint8_t sw_id, uint8_t on)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG,
+ "Mirror set, switch %d to port %d attach vlan %d on %d",
+ sw_id, mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0], on);
+
+ if (on) {
+ if (fm10k_switch_mirror_set(hw,
+ mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0]) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ } else {
+ if (fm10k_switch_mirror_reset(hw) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t sw_id)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG, "Mirror reset, switch %d", sw_id);
+
+ fm10k_switch_mirror_reset(hw);
+
+ return 0;
+}
+
+static int
+fm10k_dev_filter_ctrl(struct rte_eth_dev *dev,
+ enum rte_filter_type filter_type,
+ enum rte_filter_op filter_op,
+ void *arg)
+{
+ int ret = 0;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ switch (filter_type) {
+ case RTE_ETH_FILTER_GENERIC:
+ if (filter_op != RTE_ETH_FILTER_GET)
+ return -EINVAL;
+ *(const void **)arg = fm10k_flow_ops_get();
+ break;
+ default:
+ PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
+ filter_type);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+#endif
+
static void
fm10k_dev_enable_intr_pf(struct rte_eth_dev *dev)
{
@@ -2592,6 +2923,9 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
int status_mbx;
s32 err;
+#ifdef RTE_FM10K_MANAGEMENT
+ uint32_t writeback = 0;
+#endif
if (hw->mac.type != fm10k_mac_pf)
return;
@@ -2605,11 +2939,20 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle switch up/down */
- if (cause & FM10K_EICR_SWITCHNOTREADY)
- PMD_INIT_LOG(ERR, "INT: Switch is not ready");
+ if (cause & FM10K_EICR_SWITCHNOTREADY) {
+ PMD_INIT_LOG(INFO, "INT: Switch is not ready");
+#ifdef RTE_FM10K_MANAGEMENT
+ hw->switch_ready = false;
+ writeback |= FM10K_EICR_SWITCHNOTREADY;
+#endif
+ }
if (cause & FM10K_EICR_SWITCHREADY) {
PMD_INIT_LOG(INFO, "INT: Switch is ready");
+#ifdef RTE_FM10K_MANAGEMENT
+ hw->switch_ready = true;
+ writeback |= FM10K_EICR_SWITCHREADY;
+#endif
if (dev_info->sm_down == 1) {
fm10k_mbx_lock(hw);
@@ -2660,6 +3003,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle mailbox message */
+#ifndef RTE_FM10K_MANAGEMENT
fm10k_mbx_lock(hw);
err = hw->mbx.ops.process(hw, &hw->mbx);
fm10k_mbx_unlock(hw);
@@ -2667,10 +3011,33 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
if (err == FM10K_ERR_RESET_REQUESTED) {
PMD_INIT_LOG(INFO, "INT: Switch is down");
dev_info->sm_down = 1;
- _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
NULL);
}
+#else
+ if (cause & FM10K_EICR_MAILBOX) {
+ fm10k_mbx_lock(hw);
+ err = hw->mbx.ops.process(hw, &hw->mbx);
+ fm10k_mbx_unlock(hw);
+ writeback |= FM10K_EICR_MAILBOX;
+ if (err == FM10K_ERR_RESET_REQUESTED) {
+ PMD_INIT_LOG(INFO, "INT: Switch is down");
+ dev_info->sm_down = 1;
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
+ NULL);
+ }
+ }
+
+ /* Handle switch interrupt */
+ if (cause & FM10K_SW_EICR_SWITCH_INT)
+ fm10k_switch_intr(hw);
+#endif
+
/* Handle SRAM error */
if (cause & FM10K_EICR_SRAMERROR) {
PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
@@ -2682,15 +3049,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
/* Todo: print out error message after shared code updates */
}
+#ifndef RTE_FM10K_MANAGEMENT
/* Clear these 3 events if having any */
cause &= FM10K_EICR_SWITCHNOTREADY | FM10K_EICR_MAILBOX |
FM10K_EICR_SWITCHREADY;
if (cause)
FM10K_WRITE_REG(hw, FM10K_EICR, cause);
+#else
+ if (writeback)
+ FM10K_WRITE_REG(hw, FM10K_EICR, writeback);
+#endif
/* Re-enable interrupt from device side */
- FM10K_WRITE_REG(hw, FM10K_ITR(0), FM10K_ITR_AUTOMASK |
+#ifndef RTE_FM10K_MANAGEMENT
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_ITR_AUTOMASK |
FM10K_ITR_MASK_CLEAR);
+#else
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_SW_ITR_AUTO_MASK |
+ FM10K_SW_MAKE_REG_FIELD(ITR_MASK, FM10K_SW_ITR_MASK_W_ENABLE));
+#endif
/* Re-enable interrupt from host side */
rte_intr_ack(dev->intr_handle);
}
@@ -2897,6 +3276,11 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
.reta_query = fm10k_reta_query,
.rss_hash_update = fm10k_rss_hash_update,
.rss_hash_conf_get = fm10k_rss_hash_conf_get,
+#ifdef RTE_FM10K_MANAGEMENT
+ .mirror_rule_set = fm10k_mirror_rule_set,
+ .mirror_rule_reset = fm10k_mirror_rule_reset,
+ .filter_ctrl = fm10k_dev_filter_ctrl,
+#endif
};
static int ftag_check_handler(__rte_unused const char *key,
@@ -3075,13 +3459,67 @@ static void __attribute__((cold))
info->sm_down = false;
}
+#ifdef RTE_FM10K_MANAGEMENT
+static int eth_fm10k_dev_init_hook(struct fm10k_hw *hw)
+{
+ int i;
+ struct rte_eth_dev *dev =
+ (struct rte_eth_dev *)fm10k_switch_dpdk_port_rte_dev_get(hw);
+
+ /*
+ * Below function will trigger operations on mailbox, acquire lock to
+ * avoid race condition from interrupt handler. Operations on mailbox
+ * FIFO will trigger interrupt to PF/SM, in which interrupt handler
+ * will handle and generate an interrupt to our side. Then, FIFO in
+ * mailbox will be touched.
+ */
+ if (hw->mac.dglort_map == FM10K_DGLORTMAP_NONE) {
+ PMD_INIT_LOG(ERR, "dglort_map is not ready");
+ return -1;
+ }
+
+ fm10k_mbx_lock(hw);
+ /* Enable port first */
+ hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+ MAX_LPORT_NUM, 1);
+ /* Set unicast mode by default. App can change to other mode in other
+ * API func.
+ */
+ hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+ FM10K_XCAST_MODE_NONE);
+ fm10k_mbx_unlock(hw);
+
+ /* Make sure default VID is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (hw->mac.default_vid)
+ break;
+ /* Delay some time to acquire async port VLAN info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (hw->mac.default_vid == 0)
+ hw->mac.default_vid = 1;
+ }
+
+ /* Add default mac address */
+ fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+ MAIN_VSI_POOL_NUMBER);
+
+ return 0;
+}
+#endif
+
static int
eth_fm10k_dev_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
- int diag, i;
+ int diag;
+#ifndef RTE_FM10K_MANAGEMENT
+ int i;
+#endif
struct fm10k_macvlan_filter_info *macvlan;
PMD_INIT_FUNC_TRACE();
@@ -3118,7 +3556,9 @@ static void __attribute__((cold))
" Try to blacklist unused devices.");
return -EIO;
}
-
+#ifdef RTE_FM10K_MANAGEMENT
+ hw->sw_addr = (void *)pdev->mem_resource[4].addr;
+#endif
/* Store fm10k_adapter pointer */
hw->back = dev->data->dev_private;
@@ -3129,6 +3569,25 @@ static void __attribute__((cold))
return -EIO;
}
+#ifdef RTE_FM10K_MANAGEMENT
+ if (hw->mac.type == fm10k_mac_pf) {
+ if (hw->hw_addr == NULL || hw->sw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ } else {
+ if (hw->hw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ }
+
+ /* Store fm10k_adapter pointer */
+ hw->back = dev->data->dev_private;
+#endif
+
/* Initialize parameters */
fm10k_params_init(dev);
@@ -3209,6 +3668,7 @@ static void __attribute__((cold))
hw->mac.ops.update_int_moderator(hw);
/* Make sure Switch Manager is ready before going forward. */
+#ifndef RTE_FM10K_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf) {
bool switch_ready = false;
@@ -3216,13 +3676,13 @@ static void __attribute__((cold))
fm10k_mbx_lock(hw);
hw->mac.ops.get_host_state(hw, &switch_ready);
fm10k_mbx_unlock(hw);
- if (switch_ready == true)
+ if (switch_ready)
break;
/* Delay some time to acquire async LPORT_MAP info. */
rte_delay_us(WAIT_SWITCH_MSG_US);
}
- if (switch_ready == false) {
+ if (switch_ready == 0) {
PMD_INIT_LOG(ERR, "switch is not ready");
return -1;
}
@@ -3268,11 +3728,25 @@ static void __attribute__((cold))
MAIN_VSI_POOL_NUMBER);
return 0;
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ bool master = FM10K_READ_REG(hw,
+ FM10K_CTRL) & FM10K_CTRL_BAR4_ALLOWED;
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 1, master, eth_fm10k_dev_init_hook);
+ } else {
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 0, false, eth_fm10k_dev_init_hook);
+ }
+#endif
}
static int
eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
{
+#ifdef RTE_FM10K_MANAGEMENT
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
PMD_INIT_FUNC_TRACE();
/* only uninitialize in the primary process */
@@ -3282,6 +3756,10 @@ static void __attribute__((cold))
/* safe to close dev here */
fm10k_dev_close(dev);
+#ifdef RTE_FM10K_MANAGEMENT
+ fm10k_switch_dpdk_port_stop(hw);
+#endif
+
return 0;
}
diff --git a/drivers/net/fm10k/switch/fm10k_switch.c b/drivers/net/fm10k/switch/fm10k_switch.c
index 009d62a..abe719d 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.c
+++ b/drivers/net/fm10k/switch/fm10k_switch.c
@@ -31,6 +31,10 @@
#define FM10K_LED_POLL_INTERVAL_MS 500
#define FM10K_LED_BLINKS_PER_SECOND 2
+/* Max try times to acquire switch status */
+#define FM10K_QUERY_SWITCH_STATE_TIMES 10
+/* Wait interval to get switch status */
+#define FM10K_WAIT_SWITCH_MSG_US 100000
/*
* use epl as external port map, only support QUAD_ON
@@ -1914,6 +1918,19 @@ struct fm10k_switch*
}
}
+ /* Make sure Switch Manager is ready before going forward. */
+ for (i = 0; i < FM10K_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (master_hw->switch_ready)
+ break;
+ /* Delay some time to acquire async LPORT_MAP info. */
+ rte_delay_us(FM10K_WAIT_SWITCH_MSG_US);
+ }
+
+ if (!master_hw->switch_ready) {
+ PMD_INIT_LOG(ERR, "switch is not ready");
+ return -1;
+ }
+
/* do initialize all ports, after switch is ready */
for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
uint32_t sglort;
@@ -1941,8 +1958,9 @@ struct fm10k_switch*
return -1;
}
hw->mac.dglort_map = sglort | 0xffff0000;
+ hw->switch_ready = master_hw->switch_ready;
+ func(hw);
}
- func(hw);
}
return 0;
}
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
index b4b363c..a2aa0d0 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.h
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -34,6 +34,7 @@
#define FM10K_SW_PEPS_MAX 9
#define FM10K_SW_PEPS_SUPPORTED 4
#define FM10K_SW_PEP_PORTS_MAX 4
+#define FM10K_SW_PEP_MAP_NUM_MAX 2
/*
* GLORT
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support
2020-02-25 11:28 ` Wang, Xiao W
2020-02-25 12:56 ` Xiaojun Liu
@ 2020-03-05 1:23 ` Xiaojun Liu
1 sibling, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-05 1:23 UTC (permalink / raw)
To: Wang, Xiao W, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E
Cc: dev, Ye, Xiaolong, Jeff Zheng
Hi Xiao,
Thank you! Did you received my new patches last weekend?
BTW, I updated to the latest dpdk as the code base.
Best regards,
Xiaojun
________________________________
From: Wang, Xiao W <xiao.w.wang@intel.com>
Sent: Tuesday, February 25, 2020 7:28 PM
To: Xiaojun Liu <xiaojun.liu@silicom.co.il>; Zhang, Qi Z <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller, Jacob E <jacob.e.keller@intel.com>
Cc: dev@dpdk.org <dev@dpdk.org>; Ye, Xiaolong <xiaolong.ye@intel.com>
Subject: RE: [PATCH v2 5/5] net/fm10k: add switch management support
Hi Xiaojun,
Gather some comments about the code changes to original file:
Based on the latest dpdk, "git am" will fail with the Makefile.
Add CONFIG_RTE_FM10K_MANAGEMENT=n as a default option into ./config is better. If user wants to turn on this switch feature, it's easier to just change "n" to "y".
No need to introduce another macro ENABLE_FM10K_MANAGEMENT, just use RTE_FM10K_MANAGEMENT to wrap your code.
For ethdev.c:
static int fm10k_switch_ready; This should be a per device value, right? I assume we may have >1 fm10k devices, then cannot share one state value.
DEV_RX_OFFLOAD_RSS_HASH is removed, it may affect current usage.
Log some debug info before return -EIO.
How about moving the commonly used variables like "struct fm10k_hw *hw" out of "ifdef else".
Put fm10k_link_update() at the end of dev_start().
For fm10k_stats_get(), it looks there's little code shared between #if and #else, you can just write a whole new function.
There's comment "It may not work for VF", have you addressed this uncertainty? We'd better not to leave this in upstream version.
BTW, when sending new versions, you need to document what has changed from last version in your cover letter, in below format:
V2:
* Fix what
* Fix what
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Thursday, February 20, 2020 10:00 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>;
> Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v2 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for switch management
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-03-16 6:37 ` Wang, Xiao W
0 siblings, 0 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-03-16 6:37 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiaojun,
Comments inline.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, February 28, 2020 4:38 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v1 1/5] net/fm10k: add basic functions for switch management
For new version patch, the version number should increment from previous one. Otherwise, it confuses reviewer.
>
> Add I2C to control the inside LED and PHY.
> All the operations of I2C are using fm10k I2C register.
> Add SBUS to communicate with spico(micro code in serdes)
> by using fm10k SBUS register. This is like I2C operations.
> Add registers defination, which include all the registers
> will be used in the driver. Add switch management log API.
> Add switch management structures. Modify Makefile to add
> new files building. Add CONFIG_RTE_FM10K_MANAGEMENT=n
> in config/common_linux.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> config/common_linux | 7 +
> drivers/net/fm10k/Makefile | 14 +
> drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> drivers/net/fm10k/switch/fm10k_i2c.c | 310 +++++
> drivers/net/fm10k/switch/fm10k_i2c.h | 54 +
> drivers/net/fm10k/switch/fm10k_regs.h | 2302
> +++++++++++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_sbus.c | 292 ++++
> drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
> 9 files changed, 3373 insertions(+)
> create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
>
> diff --git a/config/common_linux b/config/common_linux
> index 8168106..75a4fa8 100644
> --- a/config/common_linux
> +++ b/config/common_linux
> @@ -66,3 +66,10 @@ CONFIG_RTE_LIBRTE_HINIC_PMD=y
> # Hisilicon HNS3 PMD driver
> #
> CONFIG_RTE_LIBRTE_HNS3_PMD=y
> +
> +#
> +# FM10K switch management
> +#
> +CONFIG_RTE_FM10K_MANAGEMENT=n
CONFIG_RTE_FM10K_SWITCH_MANAGEMENT is more appropriate.
> +
> +
Do not add extra empty lines.
> diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
> index 29e659d..15ea187 100644
> --- a/drivers/net/fm10k/Makefile
> +++ b/drivers/net/fm10k/Makefile
> @@ -11,6 +11,9 @@ LIB = librte_pmd_fm10k.a
> CFLAGS += -O3
> CFLAGS += $(WERROR_FLAGS)
> CFLAGS += -DALLOW_EXPERIMENTAL_API
> +ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
> +CFLAGS += -DENABLE_FM10K_MANAGEMENT
> +endif
As I commented previously, no need to add another new MACRO. Just use RTE_FM10K_MANAGEMENT
>
> EXPORT_MAP := rte_pmd_fm10k_version.map
>
> @@ -49,6 +52,9 @@ endif
> LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
> LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash
> LDLIBS += -lrte_bus_pci
> +ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
> +LDLIBS += -lpthread
> +endif
>
> #
> # Add extra flags for base driver source files to disable warnings in them
> @@ -58,6 +64,10 @@ $(foreach obj, $(BASE_DRIVER_OBJS), $(eval
> CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER))
>
> VPATH += $(SRCDIR)/base
>
> +ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
> +VPATH += $(SRCDIR)/switch
> +endif
> +
> #
> # all source are stored in SRCS-y
> # base driver is based on the package of cid-fm10k.2017.01.24.tar.gz
> @@ -71,6 +81,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) +=
> fm10k_common.c
> SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_mbx.c
> SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
> SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
> +ifeq ($(CONFIG_RTE_FM10K_MANAGEMENT),y)
> +SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
> +SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
> +endif
> ifeq ($(CONFIG_RTE_ARCH_X86), y)
> SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
> endif
> diff --git a/drivers/net/fm10k/switch/fm10k_debug.h
> b/drivers/net/fm10k/switch/fm10k_debug.h
> new file mode 100644
> index 0000000..f7b5c06
> --- /dev/null
> +++ b/drivers/net/fm10k/switch/fm10k_debug.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 Silicom Ltd. Connectivity Solutions
> + */
> +
> +#ifndef _FM10K_DEBUG_H_
> +#define _FM10K_DEBUG_H_
> +
> +
> +#define FM10K_SW_ERR(...) PMD_INIT_LOG(ERR, __VA_ARGS__)
> +#define FM10K_SW_INFO(...) PMD_INIT_LOG(INFO, __VA_ARGS__)
> +#define FM10K_SW_TRACE(...) PMD_INIT_LOG(DEBUG, __VA_ARGS__)
> +
> +#define FM10K_SW_ASSERT(...) do {} while (0)
> +
> +#define FM10K_SW_STATS_TRACE_ENABLE 1
> +#define FM10K_SW_FFU_CONF_TRACE_ENABLE 0
> +#define FM10K_SW_MIRROR_TRACE_ENABLE 0
> +
> +#endif /* _FM10K_DEBUG_H_ */
> diff --git a/drivers/net/fm10k/switch/fm10k_i2c.c
> b/drivers/net/fm10k/switch/fm10k_i2c.c
> new file mode 100644
> index 0000000..28b0c34
> --- /dev/null
> +++ b/drivers/net/fm10k/switch/fm10k_i2c.c
> @@ -0,0 +1,310 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 Silicom Ltd. Connectivity Solutions
> + */
> +
> +#include <rte_malloc.h>
> +
Remove this extra empty line, and please re-check other places for similar issues.
> +#include <rte_time.h>
> +#include <rte_kvargs.h>
> +#include <rte_hash.h>
> +#include <rte_flow.h>
> +#include <rte_flow_driver.h>
> +#include <rte_tm_driver.h>
> +
> +#include "fm10k_debug.h"
> +#include "fm10k_i2c.h"
> +#include "fm10k_regs.h"
> +#include "fm10k_switch.h"
> +
[...]
> diff --git a/drivers/net/fm10k/switch/fm10k_i2c.h
> b/drivers/net/fm10k/switch/fm10k_i2c.h
> new file mode 100644
> index 0000000..f835afe
> --- /dev/null
> +++ b/drivers/net/fm10k/switch/fm10k_i2c.h
> @@ -0,0 +1,54 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 Silicom Ltd. Connectivity Solutions
> + */
> +
> +#ifndef _FM10K_SW_I2C_H_
> +#define _FM10K_SW_I2C_H_
> +
> +#include <semaphore.h>
> +#include <pthread.h>
Add an empty line here to separate CLIB headers and local headers. Please check all other places.
> +#include "rte_spinlock.h"
> +#include "fm10k_debug.h"
> +
[...]
> diff --git a/drivers/net/fm10k/switch/fm10k_sbus.c
> b/drivers/net/fm10k/switch/fm10k_sbus.c
> new file mode 100644
> index 0000000..d7d656e
> --- /dev/null
> +++ b/drivers/net/fm10k/switch/fm10k_sbus.c
> @@ -0,0 +1,292 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 Silicom Ltd. Connectivity Solutions
> + */
> +
> +#include <rte_malloc.h>
> +
> +#include <rte_time.h>
> +#include <rte_kvargs.h>
> +#include <rte_hash.h>
> +#include <rte_flow.h>
> +#include <rte_flow_driver.h>
> +#include <rte_tm_driver.h>
> +
> +#include "fm10k_debug.h"
> +#include "fm10k_regs.h"
> +#include "fm10k_sbus.h"
> +#include "fm10k_switch.h"
> +
> +#define FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX 4
> +#define FM10K_SW_SBUS_COMMAND_TIMEOUT_US 500000
> +#define FM10K_SW_SBUS_RING_DIVIDER 0x01
The above two lines are not aligned, though in outlook it may be. You can have a check here: http://patches.dpdk.org/patch/66130/
It's also not aligned in my "vim". Note: " set tabstop=8 " is recommended for dpdk.
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
@ 2020-03-16 6:48 ` Wang, Xiao W
0 siblings, 0 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-03-16 6:48 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiaojun,
Comments inline.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, February 28, 2020 4:38 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions
>
> Add epl serdes include loading spico,
> controlling pcsl, dma, dfe, ical. Add spico code.
> Add state machine for epl lane and port, it creates
> a pthread to handle the state changing event.
> Add external port management, which will use
> state machine to handle the event from lane and port.
> The lane state will change between DOWN, WAIT_PLL_LOCK,
> WAIT_SIGNAL_OK, WAIT_DFE_ICAL, WAIT_DFE_PCAL, UP.
> The port state will change between DOWN, WAIT_LANE_UP, UP.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
No need to add above section in every patch. I suggest to put it into doc/guides/nics/fm10k.rst,
and you can have a more detailed description about this big feature there.
Please also check the empty lines and alignment issues.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
@ 2020-03-16 7:04 ` Wang, Xiao W
0 siblings, 0 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-03-16 7:04 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi,
Comments inline.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, February 28, 2020 4:38 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file
> functions
>
> Add ffu to support offload flow into HW.
> It supports forward, mirror, push VLAN, pop VLAN.
> It also supports flowset for a group flow definition.
> The config file can configure debug log, port speed,
> epl port mapping dpdk port, flowset. All these configuration
> will be used by switch management.
> Statistics includes epl port, ffu rule, dpdk port, and error.
> All these statistics data are read from HW.
> Modify switch header file to support getting logical port
> and glort and device info.
>
If new features are added, please doc them in doc/guides/nics/features/fm10k.ini
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
[...]
> +
> + /* GLORT_DEST_TABLE
> + * Field Name Bit(s) Type Default
> + * DestMask 47:0 RW 0x0
> + * IP_MulticastIndex59:48 RW 0x0
> + * Reserved 63:60 RSV 0x0
> + */
Please fix the alignment issue around here.
> + temp64 = sw->mcast_dest_table_idx;
> + temp64 = temp64 << 48 | 1 << lport1 | 1 << lport2;
> + fm10k_write_switch_reg64(sw,
> + FM10K_SW_GLORT_DEST_TABLE
> + (sw->glort_dest_table_idx), temp64);
> + sw->glort_dest_table_idx++;
> +
> + /* MCAST_DEST_TABLE
> + * Field Name Bit(s) Type Default
> + * PortMask 47:0 RW 0x0
> + * LenTableIdx 61:48 RW 0x0
> + * Reserved 63:62 RSV 00b
> + */
> + temp64 = sw->mcast_len_table_idx;
> + temp64 = 1 << lport1 | 1 << lport2 | temp64 << 48;
> + fm10k_write_switch_reg64(sw,
> + FM10K_SW_SCHED_MCAST_DEST_TABLE
> + (sw->mcast_dest_table_idx++), temp64);
> +
> + /* MCAST_LEN_TABLE
> + * Field Name Bit(s) Type Default
> + * L3_McastIdx 14:0 RW 0x0
> + * L3_Repcnt 26:15 RW 0x0
> + * Reserved 31:27 RSV 0x0
> + */
> + temp64 =
> + sw->mcast_vlan_table_idx | 1 << 15;
> + fm10k_write_switch_reg64(sw,
> + FM10K_SW_SCHED_MCAST_LEN_TABLE
> + (sw->mcast_len_table_idx++), temp64);
> +
> + /* MCAST_VLAN_TABLE
> + * Field Name Bit(s) Type Default
> + * VID 11:0 RW 0x0
> + * DGLORT 27:12 RW 0x0
> + * ReplaceVID 28 RW 0b
> + * ReplaceDGLORT 29 RW 0b
> + * Reserved 31:30 RSV 00b
> + */
> + temp64 = vlan1 |
> + fm10k_switch_pf_glort_get
> + (lport1) << 12 | 1 << 28 | 1 << 29;
> + fm10k_write_switch_reg64(sw,
> + FM10K_SW_MOD_MCAST_VLAN_TABLE
> + (sw->mcast_vlan_table_idx++), temp64);
> + temp64 = vlan2 |
> + fm10k_switch_pf_glort_get
> + (lport2) << 12 | 1 << 28 | 1 << 29;
> + fm10k_write_switch_reg64(sw,
> + FM10K_SW_MOD_MCAST_VLAN_TABLE
> + (sw->mcast_vlan_table_idx++), temp64);
> +
> + return dglort;
[...]
> + for (i = 0;
> + i < sizeof(rx_port_cnt_map_table) /
> + sizeof(rx_port_cnt_map_table[0]);
> + i++) {
The style looks strange. No need to switch to a new line as " for (i = 0;" is so short.
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support Xiaojun Liu
@ 2020-03-16 7:34 ` Wang, Xiao W
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
1 sibling, 0 replies; 50+ messages in thread
From: Wang, Xiao W @ 2020-03-16 7:34 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiaojun,
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, February 28, 2020 4:38 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v1 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput.
>
> To enable the switch management, you need add
> CONFIG_RTE_FM10K_MANAGEMENT=y in
> config/common_linux when building.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> drivers/net/fm10k/base/fm10k_type.h | 1 +
> drivers/net/fm10k/fm10k_ethdev.c | 562
> +++++++++++++++++++++++++++++---
> drivers/net/fm10k/switch/fm10k_switch.c | 20 +-
> drivers/net/fm10k/switch/fm10k_switch.h | 1 +
> 4 files changed, 541 insertions(+), 43 deletions(-)
>
> diff --git a/drivers/net/fm10k/base/fm10k_type.h
> b/drivers/net/fm10k/base/fm10k_type.h
> index 84781ba..810e5ee 100644
> --- a/drivers/net/fm10k/base/fm10k_type.h
> +++ b/drivers/net/fm10k/base/fm10k_type.h
> @@ -717,6 +717,7 @@ struct fm10k_hw {
> u16 subsystem_vendor_id;
> u8 revision_id;
> u32 flags;
> + bool switch_ready;
> #define FM10K_HW_FLAG_CLOCK_OWNER BIT(0)
> };
>
> diff --git a/drivers/net/fm10k/fm10k_ethdev.c
> b/drivers/net/fm10k/fm10k_ethdev.c
> index 581c690..f11fa53 100644
> --- a/drivers/net/fm10k/fm10k_ethdev.c
> +++ b/drivers/net/fm10k/fm10k_ethdev.c
> @@ -13,6 +13,10 @@
>
> #include "fm10k.h"
> #include "base/fm10k_api.h"
> +#ifdef RTE_FM10K_MANAGEMENT
> +#include "switch/fm10k_regs.h"
> +#include "switch/fm10k_switch.h"
> +#endif
>
> /* Default delay to acquire mailbox lock */
> #define FM10K_MBXLOCK_DELAY_US 20
> @@ -461,9 +465,6 @@ struct fm10k_xstats_name_off {
>
> PMD_INIT_FUNC_TRACE();
>
> - if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG)
> - dev->data->dev_conf.rxmode.offloads |=
> DEV_RX_OFFLOAD_RSS_HASH;
> -
Why delete this? Will it break the existing usage?
> /* multipe queue mode checking */
> ret = fm10k_check_mq_mode(dev);
> if (ret != 0) {
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 0/5] support fm10k switch management
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support Xiaojun Liu
2020-03-16 7:34 ` Wang, Xiao W
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for " Xiaojun Liu
` (4 more replies)
1 sibling, 5 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
V3:
* Add I2C to control the inside LED and PHY.
* Add SBUS to communicate with spico.
* Add registers defination.
* Add switch management log API.
* Add switch management structures.
* Add epl serdes include loading spico,
controlling pcsl, dma, dfe, ical.
* Add spico code.
* Add state machine for epl lane and port.
* Add external port management.
* Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
* Add config file to configure debug log, port speed,
epl port mapping dpdk port, flowset.
* Add statistics includes epl port, ffu rule, dpdk port.
* Add flow interface to support offload flow into HW.
* Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
* Split dev_init to 2 parts in ethdev. First only register
the port in switch management; second init hook will be
called after all the pf are registered.
* Add switch interrupt support.
* Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset.
* Add fm10k_dev_filter_ctrl to support flow operation.
* Add dpdk port and pf mapping.
* Modify Makefile to add new files building
Xiaojun Liu (5):
net/fm10k: add basic functions for switch management
net/fm10k: add epl serdes and port control functions
net/fm10k: add ffu and statistics and config file functions
net/fm10k: add flow interface and switch management
net/fm10k: add switch management support
config/common_linux | 5 +
doc/guides/nics/fm10k.rst | 5 +
drivers/net/fm10k/Makefile | 20 +
drivers/net/fm10k/base/fm10k_type.h | 1 +
drivers/net/fm10k/fm10k_ethdev.c | 559 ++++-
drivers/net/fm10k/switch/fm10k_config.c | 857 ++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 ++
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 839 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_ffu.c | 1250 +++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 ++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_i2c.c | 309 +++
drivers/net/fm10k/switch/fm10k_i2c.h | 55 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 291 +++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 80 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
drivers/net/fm10k/switch/fm10k_stats.c | 1233 +++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++
drivers/net/fm10k/switch/fm10k_switch.c | 2382 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_switch.h | 476 +++++
29 files changed, 17329 insertions(+), 39 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for switch management
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-04-02 8:41 ` Wang, Xiao W
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
` (3 subsequent siblings)
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add I2C to control the inside LED and PHY.
All the operations of I2C are using fm10k I2C register.
Add SBUS to communicate with spico(micro code in serdes)
by using fm10k SBUS register. This is like I2C operations.
Add registers defination, which include all the registers
will be used in the driver. Add switch management log API.
Add switch management structures. Modify Makefile to add
new files building. Add CONFIG_RTE_FM10K_MANAGEMENT=n
in config/common_linux.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
config/common_linux | 5 +
drivers/net/fm10k/Makefile | 11 +
drivers/net/fm10k/switch/fm10k_debug.h | 19 +
drivers/net/fm10k/switch/fm10k_i2c.c | 309 +++++
drivers/net/fm10k/switch/fm10k_i2c.h | 55 +
drivers/net/fm10k/switch/fm10k_regs.h | 2302 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_sbus.c | 291 ++++
drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
9 files changed, 3367 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
diff --git a/config/common_linux b/config/common_linux
index 8168106..6e62b99 100644
--- a/config/common_linux
+++ b/config/common_linux
@@ -66,3 +66,8 @@ CONFIG_RTE_LIBRTE_HINIC_PMD=y
# Hisilicon HNS3 PMD driver
#
CONFIG_RTE_LIBRTE_HNS3_PMD=y
+
+#
+# FM10K switch management
+#
+CONFIG_RTE_FM10K_SWITCH_MANAGEMENT=n
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 29e659d..49bb554 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -49,6 +49,9 @@ endif
LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring
LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash
LDLIBS += -lrte_bus_pci
+ifeq ($(CONFIG_RTE_FM10K_SWITCH_MANAGEMENT),y)
+LDLIBS += -lpthread
+endif
#
# Add extra flags for base driver source files to disable warnings in them
@@ -58,6 +61,10 @@ $(foreach obj, $(BASE_DRIVER_OBJS), $(eval CFLAGS_$(obj)+=$(CFLAGS_BASE_DRIVER))
VPATH += $(SRCDIR)/base
+ifeq ($(CONFIG_RTE_FM10K_SWITCH_MANAGEMENT),y)
+VPATH += $(SRCDIR)/switch
+endif
+
#
# all source are stored in SRCS-y
# base driver is based on the package of cid-fm10k.2017.01.24.tar.gz
@@ -71,6 +78,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_common.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_mbx.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_vf.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
+ifeq ($(CONFIG_RTE_FM10K_SWITCH_MANAGEMENT),y)
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
endif
diff --git a/drivers/net/fm10k/switch/fm10k_debug.h b/drivers/net/fm10k/switch/fm10k_debug.h
new file mode 100644
index 0000000..f7b5c06
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_debug.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_DEBUG_H_
+#define _FM10K_DEBUG_H_
+
+
+#define FM10K_SW_ERR(...) PMD_INIT_LOG(ERR, __VA_ARGS__)
+#define FM10K_SW_INFO(...) PMD_INIT_LOG(INFO, __VA_ARGS__)
+#define FM10K_SW_TRACE(...) PMD_INIT_LOG(DEBUG, __VA_ARGS__)
+
+#define FM10K_SW_ASSERT(...) do {} while (0)
+
+#define FM10K_SW_STATS_TRACE_ENABLE 1
+#define FM10K_SW_FFU_CONF_TRACE_ENABLE 0
+#define FM10K_SW_MIRROR_TRACE_ENABLE 0
+
+#endif /* _FM10K_DEBUG_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.c b/drivers/net/fm10k/switch/fm10k_i2c.c
new file mode 100644
index 0000000..7d64899
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.c
@@ -0,0 +1,309 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+
+static void fm10k_i2c_init(struct fm10k_i2c *);
+
+struct fm10k_i2c *
+fm10k_i2c_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_i2c *i2c;
+
+ FM10K_SW_TRACE("i2c: attaching");
+
+ i2c = (struct fm10k_i2c *)rte_zmalloc("fm10k_i2c",
+ sizeof(struct fm10k_i2c), 0);
+ if (i2c == NULL) {
+ FM10K_SW_INFO("i2c: failed to allocate context");
+ goto fail;
+ }
+
+ i2c->sw = sw;
+ pthread_mutex_init(&i2c->req_lock, NULL);
+ pthread_mutex_init(&i2c->bus_lock, NULL);
+ sem_init(&i2c->req_cv, 0, 0);
+
+ fm10k_i2c_init(i2c);
+
+ FM10K_SW_TRACE("i2c: attach successful");
+ return i2c;
+fail:
+ if (i2c)
+ fm10k_i2c_detach(i2c);
+ return NULL;
+}
+
+void
+fm10k_i2c_detach(struct fm10k_i2c *i2c)
+{
+ FM10K_SW_TRACE("i2c: detaching");
+
+ rte_free(i2c);
+}
+
+static void
+fm10k_i2c_init(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_device_info *cfg = sw->info;
+ uint32_t freq = FM10K_SW_I2C_CFG_DIVIDER_400_KHZ;
+ uint32_t data;
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ freq = FM10K_SW_I2C_CFG_DIVIDER_100_KHZ;
+
+ /* clear any pending interrupt */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ /* 400 KHz, master mode, unmask interrupt */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CFG);
+ data &= ~FM10K_SW_I2C_CFG_SLAVE_ENABLE;
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ FM10K_SW_REPLACE_REG_FIELD(data, I2C_CFG_DIVIDER, freq, data);
+ data &= ~FM10K_SW_I2C_CFG_INTERRUPT_MASK;
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CFG, data);
+
+ if (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice) ==
+ FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4))
+ /* reset I2C */
+ fm10k_gpio_output_set(sw, 5, 1);
+}
+
+unsigned int
+fm10k_i2c_intr(struct fm10k_i2c *i2c)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ struct fm10k_i2c_req *req;
+ int i;
+ uint32_t data[3];
+ uint32_t ctrl;
+
+ req = i2c->cur_req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_I2C_CTRL);
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL,
+ FM10K_SW_I2C_CTRL_INTERRUPT_PENDING);
+
+ req->status = FM10K_SW_REG_FIELD(ctrl, I2C_CTRL_COMMAND_COMPLETED);
+
+ if ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->status == FM10K_SW_I2C_COMPLETION_NORMAL) {
+ for (i = 0; i < FM10K_SW_HOWMANY(req->read_len, 4, 4); i++)
+ data[i] = fm10k_read_switch_reg
+ (sw, FM10K_SW_I2C_DATA(i));
+
+ for (i = 0; i < req->read_len; i++)
+ req->msg[i] =
+ (data[i / 4] >> (24 - (i % 4) * 8)) & 0xff;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ sem_post(&i2c->req_cv);
+
+ return 1;
+}
+
+int
+fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req)
+{
+ struct fm10k_switch *sw = i2c->sw;
+ int i;
+ uint32_t ctrl;
+ uint32_t data[3];
+
+ if (((req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ req->write_len > FM10K_SW_I2C_MSG_MAX) ||
+ ((req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) &&
+ ((req->read_len == 0 ||
+ req->read_len > FM10K_SW_I2C_MSG_MAX))))
+ return (-1);
+
+ FM10K_SW_TRACE("i2c: initiating command %u", req->cmd);
+
+ ctrl =
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_ADDR, req->addr << 1) |
+ FM10K_SW_MAKE_REG_FIELD(I2C_CTRL_COMMAND, req->cmd);
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_WR ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD) {
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_W, req->write_len);
+
+ data[0] = 0;
+ data[1] = 0;
+ data[2] = 0;
+
+ for (i = 0; i < req->write_len; i++)
+ data[i / 4] |= req->msg[i] << (24 - (i % 4) * 8);
+
+ for (i = 0; i < FM10K_SW_HOWMANY(req->write_len, 4, 4); i++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_I2C_DATA(i), data[i]);
+ }
+
+ if (req->cmd == FM10K_SW_I2C_COMMAND_RD ||
+ req->cmd == FM10K_SW_I2C_COMMAND_WR_RD)
+ ctrl |= FM10K_SW_MAKE_REG_FIELD
+ (I2C_CTRL_LENGTH_R, req->read_len);
+
+ req->status = FM10K_SW_I2C_COMPLETION_RUNNING;
+ i2c->cur_req = req;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* zero command field */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, 0);
+ /* initiate command */
+ fm10k_write_switch_reg(sw, FM10K_SW_I2C_CTRL, ctrl);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ while (req->status == FM10K_SW_I2C_COMPLETION_RUNNING)
+ sem_wait(&i2c->req_cv);
+
+ return 0;
+}
+
+int
+fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_RD;
+ req.read_len = 1;
+ req.msg[0] = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR_RD;
+ req.write_len = 1;
+ req.read_len = 1;
+ req.msg[0] = reg;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ FM10K_SW_INFO("i2c read failed (%u)", req.status);
+ error = -1;
+ goto done;
+ }
+
+ *result = req.msg[0];
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 1;
+ req.msg[0] = data;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 2;
+ req.msg[0] = data0;
+ req.msg[1] = data1;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+int
+fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr)
+{
+ struct fm10k_i2c_req req;
+ int error;
+
+ req.addr = addr;
+ req.cmd = FM10K_SW_I2C_COMMAND_WR;
+ req.write_len = 0;
+
+ error = fm10k_i2c_exec(i2c, &req);
+ if (error)
+ goto done;
+
+ if (req.status != FM10K_SW_I2C_COMPLETION_NORMAL) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_i2c.h b/drivers/net/fm10k/switch/fm10k_i2c.h
new file mode 100644
index 0000000..4189da8
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_i2c.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_I2C_H_
+#define _FM10K_SW_I2C_H_
+
+#include <semaphore.h>
+#include <pthread.h>
+
+#include "rte_spinlock.h"
+#include "fm10k_debug.h"
+
+#define FM10K_SW_I2C_MSG_MAX 12
+
+struct fm10k_i2c_req {
+ uint8_t addr; /* 7-bit address */
+ uint8_t cmd; /* FM10K_SW_I2C_COMMAND_* */
+ uint8_t write_len;
+ uint8_t read_len;
+ uint8_t status; /* FM10K_SW_I2C_COMPLETION_ */
+ uint8_t msg[FM10K_SW_I2C_MSG_MAX];
+};
+
+struct fm10k_i2c {
+ struct fm10k_switch *sw;
+ pthread_mutex_t bus_lock;
+ pthread_mutex_t req_lock;
+ sem_t req_cv;
+ struct fm10k_i2c_req *cur_req;
+};
+
+#define FM10K_SW_I2C_LOCK(i2c_) \
+ pthread_mutex_lock(&(i2c_)->bus_lock)
+#define FM10K_SW_I2C_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&(i2c_)->bus_lock)
+
+#define FM10K_SW_I2C_REQ_LOCK(i2c_) \
+ pthread_mutex_lock(&((i2c_)->req_lock))
+#define FM10K_SW_I2C_REQ_UNLOCK(i2c_) \
+ pthread_mutex_unlock(&((i2c_)->req_lock))
+
+struct fm10k_i2c *fm10k_i2c_attach(struct fm10k_switch *sw);
+void fm10k_i2c_detach(struct fm10k_i2c *i2c);
+unsigned int fm10k_i2c_intr(struct fm10k_i2c *i2c);
+int fm10k_i2c_exec(struct fm10k_i2c *i2c, struct fm10k_i2c_req *req);
+int fm10k_i2c_read8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t *result);
+int fm10k_i2c_read8_ext(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t reg, uint8_t *result);
+int fm10k_i2c_write8(struct fm10k_i2c *i2c, uint8_t addr, uint8_t data);
+int fm10k_i2c_write16(struct fm10k_i2c *i2c,
+ uint8_t addr, uint8_t data0, uint8_t data1);
+int fm10k_i2c_probe(struct fm10k_i2c *i2c, uint8_t addr);
+
+#endif /* _FM10K_SW_I2C_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_regs.h b/drivers/net/fm10k/switch/fm10k_regs.h
new file mode 100644
index 0000000..2663773
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_regs.h
@@ -0,0 +1,2302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_REGS_H_
+#define _FM10K_REGS_H_
+
+#include <stdint.h>
+
+
+/* Convert a 32-bit word offset into a byte offset */
+#define FM10K_SW_REG_OFF(wo_) (wo_)
+#define FM10K_SW_MASK32(max_bit_, min_bit_) \
+ ((0xffffffffU << (min_bit_)) & \
+ (0xffffffffU >> (31 - (max_bit_))))
+#define FM10K_SW_REG_FIELD(r_, name_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) >> FM10K_SW_##name_##_lsb)
+#define FM10K_SW_REG_FIELD_IDX(r_, name_, i_, i1_, i2_) \
+ (((uint32_t)(r_) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i_), \
+ FM10K_SW_##name_##_lsb(i2_))) >> \
+ FM10K_SW_##name_##_lsb(i3_))
+#define FM10K_SW_MAKE_REG_FIELD(name_, v_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb))
+#define FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_, i1_, i2_) \
+ (((uint32_t)(v_) << FM10K_SW_##name_##_lsb(i_)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb, \
+ FM10K_SW_##name_##_lsb)) | \
+ FM10K_SW_MAKE_REG_FIELD(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb(i1_), \
+ FM10K_SW_##name_##_lsb(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX(name_, i_, v_)))
+#define FM10K_SW_MASK64(max_bit_, min_bit_) \
+ ((0xffffffffffffffffULL << (min_bit_)) & \
+ (0xffffffffffffffffULL >> (63 - (max_bit_))))
+#define FM10K_SW_REG_FIELD64(r_, name_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) >> \
+ FM10K_SW_##name_##_lsb64)
+#define FM10K_SW_REG_FIELD_IDX64(r_, name_, i_, i1_, i2_) \
+ (((uint64_t)(r_) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i_), \
+ FM10K_SW_##name_##_lsb64(i1_))) >> \
+ FM10K_SW_##name_##_lsb64(i2_))
+#define FM10K_SW_MAKE_REG_FIELD64(name_, v_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64))
+#define FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_, i1_, i2_) \
+ (((uint64_t)(v_) << FM10K_SW_##name_##_lsb64(i_)) & \
+ FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_)))
+#define FM10K_SW_REPLACE_REG_FIELD64(r_, name_, v_, r1_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64, \
+ FM10K_SW_##name_##_lsb64)) | \
+ FM10K_SW_MAKE_REG_FIELD64(name_, v_)))
+#define FM10K_SW_REPLACE_REG_FIELD_IDX64(r_, name_, i_, v_, r1_, i1_, i2_) \
+ (r_ = (((r1_) & \
+ ~FM10K_SW_MASK64(FM10K_SW_##name_##_msb64(i1_), \
+ FM10K_SW_##name_##_lsb64(i2_))) | \
+ FM10K_SW_MAKE_REG_FIELD_IDX64(name_, i_, v_)))
+
+/* These operate on arrays of 32-bit words */
+#define FM10K_SW_SET_ARRAY_BIT(a_, name_, b_, a1_) \
+ ((a_)[FM10K_SW_##name_##_bit / 32] = \
+ ((a1_)[FM10K_SW_##name_##_bit / 32] & \
+ ~(1 << (FM10K_SW_##name_##_bit % 32))) | \
+ (((b_) & 0x1) << (FM10K_SW_##name_##_bit % 32)))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_MAKE_ARRAY_FIELD(name_, v_) \
+ (((uint32_t)(v_) << (FM10K_SW_##name_##_lsb % 32)) & \
+ FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32))
+
+/* Does not support fields that cross 32-bit boundaries */
+#define FM10K_SW_REPLACE_ARRAY_FIELD(a_, name_, v_, a1_) \
+ ((a_)[FM10K_SW_##name_##_lsb / 32] = \
+ (((a1_)[FM10K_SW_##name_##_lsb / 32] & \
+ ~FM10K_SW_MASK32(FM10K_SW_##name_##_msb % 32, \
+ FM10K_SW_##name_##_lsb % 32)) | \
+ FM10K_SW_MAKE_ARRAY_FIELD(name_, v_)))
+
+
+/*
+ * BAR0 registers
+ */
+/* Interrupt throttle timer selection values */
+#define FM10K_SW_INT_TIMER_0 0
+#define FM10K_SW_INT_TIMER_1 1
+#define FM10K_SW_INT_TIMER_IMMEDIATE 2
+#define FM10K_SW_INT_TIMER_DISABLED 3
+
+#define FM10K_SW_CTRL FM10K_SW_REG_OFF(0x0)
+#define FM10K_SW_CTRL_BAR4_ALLOWED (1 << 2)
+#define FM10K_SW_CTRL_EXT FM10K_SW_REG_OFF(0x1)
+#define FM10K_SW_CTRL_EXT_NS_DIS (1 << 0)
+#define FM10K_SW_CTRL_EXT_RO_DIS (1 << 1)
+#define FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK (1 << 2)
+#define FM10K_SW_EXVET FM10K_SW_REG_OFF(0x2)
+#define FM10K_SW_GCR FM10K_SW_REG_OFF(0x3)
+#define FM10K_SW_FACTPS FM10K_SW_REG_OFF(0x4)
+#define FM10K_SW_GCR_EXT FM10K_SW_REG_OFF(0x5)
+#define FM10K_SW_EICR FM10K_SW_REG_OFF(0x6)
+#define FM10K_SW_EICR_PCA_FAULT_SHIFT 0
+#define FM10K_SW_EICR_PCA_FAULT \
+ (1 << FM10K_SW_EICR_PCA_FAULT_SHIFT)
+#define FM10K_SW_EICR_THI_FAULT_SHIFT 2
+#define FM10K_SW_EICR_THI_FAULT \
+ (1 << FM10K_SW_EICR_THI_FAULT_SHIFT)
+#define FM10K_SW_EICR_FUM_FAULT_SHIFT 5
+#define FM10K_SW_EICR_FUM_FAULT \
+ (1 << FM10K_SW_EICR_FUM_FAULT_SHIFT)
+#define FM10K_SW_EICR_MAILBOX_SHIFT 6
+#define FM10K_SW_EICR_MAILBOX \
+ (1 << FM10K_SW_EICR_MAILBOX_SHIFT)
+#define FM10K_SW_EICR_SWITCH_READY_SHIFT 7
+#define FM10K_SW_EICR_SWITCH_READY \
+ (1 << FM10K_SW_EICR_SWITCH_READY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_NREADY_SHIFT 8
+#define FM10K_SW_EICR_SWITCH_NREADY \
+ (1 << FM10K_SW_EICR_SWITCH_NREADY_SHIFT)
+#define FM10K_SW_EICR_SWITCH_INT_SHIFT 9
+#define FM10K_SW_EICR_SWITCH_INT \
+ (1 << FM10K_SW_EICR_SWITCH_INT_SHIFT)
+#define FM10K_SW_EICR_SRAM_ERROR_SHIFT 10
+#define FM10K_SW_EICR_SRAM_ERROR \
+ (1 << FM10K_SW_EICR_SRAM_ERROR_SHIFT)
+#define FM10K_SW_EICR_VFLR_SHIFT 11
+#define FM10K_SW_EICR_VFLR \
+ (1 << FM10K_SW_EICR_VFLR_SHIFT)
+#define FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT 12
+#define FM10K_SW_EICR_MAX_HOLD_TIME \
+ (1 << FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT)
+#define FM10K_SW_EIMR FM10K_SW_REG_OFF(0x7)
+#define FM10K_SW_EIMR_DISABLE_ALL 0x55555555
+#define FM10K_SW_EIMR_NO_CHANGE 0x0
+#define FM10K_SW_EIMR_DISABLE 0x1
+#define FM10K_SW_EIMR_ENABLE 0x2
+#define FM10K_SW_EIMR_FIELD(i_, v_) \
+ (FM10K_SW_EIMR_##v_ << (FM10K_SW_EICR_##i_##_SHIFT * 2))
+#define FM10K_SW_EIMR_ENABLED(e_, i_) \
+ ((((e_) >> (FM10K_SW_EICR_##i_##_SHIFT * 2)) & 0x3) == \
+ FM10K_SW_EIMR_ENABLE)
+#define FM10K_SW_PCA_FAULT FM10K_SW_REG_OFF(0x8)
+#define FM10K_SW_THI_FAULT FM10K_SW_REG_OFF(0x10)
+#define FM10K_SW_FUM_FAULT FM10K_SW_REG_OFF(0x1C)
+#define FM10K_SW_MAXHOLDQ(n_) FM10K_SW_REG_OFF(0x20 + (n_))
+#define FM10K_SW_MAXHOLDQ_ENTRIES 8
+#define FM10K_SW_SM_AREA FM10K_SW_REG_OFF(0x28)
+#define FM10K_SW_DGLORTMAP(n_) FM10K_SW_REG_OFF(0x30 + (n_))
+#define FM10K_SW_DGLORTMAP_ENTRIES 8
+#define FM10K_SW_DGLORTMAP_MATCH_ANY 0x00000000
+#define FM10K_SW_DGLORTMAP_MATCH_NONE 0x0000ffff
+#define FM10K_SW_DGLORTMAP_VALUE_lsb 0
+#define FM10K_SW_DGLORTMAP_VALUE_msb 15
+#define FM10K_SW_DGLORTMAP_MASK_lsb 16
+#define FM10K_SW_DGLORTMAP_MASK_msb 31
+#define FM10K_SW_DGLORTDEC(n_) FM10K_SW_REG_OFF(0x38 + (n_))
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_lsb 0
+#define FM10K_SW_DGLORTDEC_Q_LENGTH_msb 3
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_lsb 4
+#define FM10K_SW_DGLORTDEC_VSI_LENGTH_msb 6
+#define FM10K_SW_DGLORTDEC_VSI_BASE_lsb 7
+#define FM10K_SW_DGLORTDEC_VSI_BASE_msb 13
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_lsb 14
+#define FM10K_SW_DGLORTDEC_PC_LENGTH_msb 15
+#define FM10K_SW_DGLORTDEC_Q_BASE_lsb 16
+#define FM10K_SW_DGLORTDEC_Q_BASE_msb 23
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_MAX 7
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_lsb 24
+#define FM10K_SW_DGLORTDEC_RSS_LENGTH_msb 26
+#define FM10K_SW_DGLORTDEC_INNTER_RSS (1 << 27)
+#define FM10K_SW_TUNNEL_CFG FM10K_SW_REG_OFF(0x40)
+#define FM10K_SW_SWPRI_MAP(pri_) FM10K_SW_REG_OFF(0x50 + (pri_))
+#define FM10K_SW_RSSRK(f_, n_) \
+ FM10K_SW_REG_OFF(0x800 + 0x10 * (f_) + (n_))
+#define FM10K_SW_RSSRK_ENTRIES 10
+#define FM10K_SW_RETA(f_, n_) \
+ FM10K_SW_REG_OFF(0x1000 + 0x20 * (f_) + (n_))
+#define FM10K_SW_RETA_ENTRIES 32
+#define FM10K_SW_RETA_ENTRY(e0_, e1_, e2_, e3_) \
+ (((e0_) & 0xff) | (((e1_) & 0xff) << 8) | \
+ (((e2_) & 0xff) << 16) | (((e3_) & 0xff) << 24))
+#define FM10K_SW_TC_CREDIT(g_) FM10K_SW_REG_OFF(0x2000 + (g_))
+#define FM10K_SW_TC_MAXCREDIT(g_) FM10K_SW_REG_OFF(0x2040 + (g_))
+#define FM10K_SW_TC_RATE(g_) FM10K_SW_REG_OFF(0x2080 + (g_))
+#define FM10K_SW_TC_RATE_STATUS FM10K_SW_REG_OFF(0x20C0)
+#define FM10K_SW_PAUSE FM10K_SW_REG_OFF(0x20C2)
+#define FM10K_SW_DMA_CTRL FM10K_SW_REG_OFF(0x20C3)
+#define FM10K_SW_DMA_CTRL_TX_ENABLE (1 << 0)
+#define FM10K_SW_DMA_CTRL_TX_HOST_PENDING (1 << 1)
+#define FM10K_SW_DMA_CTRL_TX_DATA (1 << 2)
+#define FM10K_SW_DMA_CTRL_TX_ACTIVE (1 << 3)
+#define FM10K_SW_DMA_CTRL_RX_ENABLE (1 << 4)
+#define FM10K_SW_DMA_CTRL_RX_HOST_PENDING (1 << 5)
+#define FM10K_SW_DMA_CTRL_RX_DATA (1 << 6)
+#define FM10K_SW_DMA_CTRL_RX_ACTIVE (1 << 7)
+#define FM10K_SW_DMA_CTRL_RX_DESC_SIZE_32 (1 << 8)
+#define FM10K_SW_DMA_CTRL_MIN_MSS_lsb 9
+#define FM10K_SW_DMA_CTRL_MIN_MSS_msb 22
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_lsb 23
+#define FM10K_SW_DMA_CTRL_MAX_HOLD_TIME_msb 27
+#define FM10K_SW_DMA_CTRL_DATA_PATH_RESET (1 << 29)
+#define FM10K_SW_DMA_CTRL_MAX_QS_lsb 30
+#define FM10K_SW_DMA_CTRL_MAX_QS_msb 31
+#define FM10K_SW_DMA_CTRL_MAX_QS_256 0
+#define FM10K_SW_DMA_CTRL_MAX_QS_128 1
+#define FM10K_SW_DMA_CTRL_MAX_QS_64 2
+#define FM10K_SW_DMA_CTRL2 FM10K_SW_REG_OFF(0x20C4)
+#define FM10K_SW_DTXTCPFLGL FM10K_SW_REG_OFF(0x20C5)
+#define FM10K_SW_DTXTCPFLGH FM10K_SW_REG_OFF(0x20C6)
+#define FM10K_SW_TPH_CTRL FM10K_SW_REG_OFF(0x20C7)
+#define FM10K_SW_MRQC(f_) \
+ FM10K_SW_REG_OFF(0x2100 + (f_))
+#define FM10K_SW_MRQC_TCPIPV4 (1 << 0)
+#define FM10K_SW_MRQC_IPV4 (1 << 1)
+#define FM10K_SW_MRQC_IPV6 (1 << 4)
+#define FM10K_SW_MRQC_TCPIPV6 (1 << 5)
+#define FM10K_SW_MRQC_UDPIPV4 (1 << 6)
+#define FM10K_SW_MRQC_UDPIPV6 (1 << 7)
+#define FM10K_SW_TQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x2800 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_RQMAP(nvf_, vf_, vq_, vf1_, vq1_) \
+ FM10K_SW_REG_OFF(0x3000 + (((nvf_) & ~0x7) ? \
+ ((vf_) * 32 + ((vq_) & 0x1f)) : ((vf1_) * 256 + (vq1_))))
+#define FM10K_SW_STATS_TIMEOUT FM10K_SW_REG_OFF(0x3800)
+#define FM10K_SW_STATS_UR FM10K_SW_REG_OFF(0x3801)
+#define FM10K_SW_STATS_CA FM10K_SW_REG_OFF(0x3802)
+#define FM10K_SW_STATS_UM FM10K_SW_REG_OFF(0x3803)
+#define FM10K_SW_STATS_XEC FM10K_SW_REG_OFF(0x3804)
+#define FM10K_SW_STATS_VLAN_DROP FM10K_SW_REG_OFF(0x3805)
+#define FM10K_SW_STATS_LOOPBACK_DROP FM10K_SW_REG_OFF(0x3806)
+#define FM10K_SW_STATS_NODESC_DROP FM10K_SW_REG_OFF(0x3807)
+#define FM10K_SW_RRTIME_CFG FM10K_SW_REG_OFF(0x3808)
+#define FM10K_SW_RRTIME_LIMIT(n_) \
+ FM10K_SW_REG_OFF(0x380C + 0x40 * (n_))
+#define FM10K_SW_RRTIME_COUNT(n_) \
+ FM10K_SW_REG_OFF(0x3810 + 0x40 * (n_))
+#define FM10K_SW_SYSTIME FM10K_SW_REG_OFF(0x3814)
+#define FM10K_SW_SYSTIME0 FM10K_SW_REG_OFF(0x3816)
+#define FM10K_SW_SYSTIME_CFG FM10K_SW_REG_OFF(0x3818)
+#define FM10K_SW_PFVFBME FM10K_SW_REG_OFF(0x381A)
+#define FM10K_SW_PHYADDR FM10K_SW_REG_OFF(0x381C)
+#define FM10K_SW_RDBAL(q_) \
+ FM10K_SW_REG_OFF(0x4000 + 0x40 * (q_))
+#define FM10K_SW_RDBAH(q_) \
+ FM10K_SW_REG_OFF(0x4001 + 0x40 * (q_))
+#define FM10K_SW_RDLEN(q_) \
+ FM10K_SW_REG_OFF(0x4002 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x4003 + 0x40 * (q_))
+#define FM10K_SW_TPH_RXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_RXCTRL_HEADER_TPHEN (1 << 6)
+#define FM10K_SW_TPH_RXCTRL_PAYLOAD_TPHEN (1 << 7)
+#define FM10K_SW_TPH_RXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_RXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_RXCTRL_DATA_WRITE_RO_EN (1 << 13)
+#define FM10K_SW_TPH_RXCTRL_REP_HEADER_RO_EN (1 << 15)
+#define FM10K_SW_RDH(q_) \
+ FM10K_SW_REG_OFF(0x4004 + 0x40 * (q_))
+#define FM10K_SW_RDT(q_) \
+ FM10K_SW_REG_OFF(0x4005 + 0x40 * (q_))
+#define FM10K_SW_RDT_RDT_lsb 0
+#define FM10K_SW_RDT_RDT_msb 15
+#define FM10K_SW_RXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x4006 + 0x40 * (q_))
+#define FM10K_SW_RXQCTL_ENABLE (1 << 0)
+#define FM10K_SW_RXQCTL_VF_lsb 2
+#define FM10K_SW_RXQCTL_VF_msb 7
+#define FM10K_SW_RXQCTL_OWNED_BY_VF (1 << 8)
+#define FM10K_SW_RXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x4007 + 0x40 * (q_))
+#define FM10K_SW_RXDCTL_MAX_TIME_lsb 0
+#define FM10K_SW_RXDCTL_MAX_TIME_msb 7
+#define FM10K_SW_RXDCTL_WRITE_BACK_IMM (1 << 8)
+#define FM10K_SW_RXDCTL_DROP_ON_EMPTY (1 << 9)
+#define FM10K_SW_RXDCTL_WRITE_RSS_HASH (1 << 10)
+#define FM10K_SW_RXINT(q_) \
+ FM10K_SW_REG_OFF(0x4008 + 0x40 * (q_))
+#define FM10K_SW_RXINT_INT_lsb 0
+#define FM10K_SW_RXINT_INT_msb 7
+#define FM10K_SW_RXINT_INT_TIMER_lsb 8
+#define FM10K_SW_RXINT_INT_TIMER_msb 9
+#define FM10K_SW_SRRCTL(q_) \
+ FM10K_SW_REG_OFF(0x4009 + 0x40 * (q_))
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_lsb 0
+#define FM10K_SW_SRRCTL_BSIZE_PACKET_msb 7
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_lsb 8
+#define FM10K_SW_SRRCTL_BSIZE_HEADER_msb 13
+#define FM10K_SW_SRRCTL_DESC_TYPE_lsb 14
+#define FM10K_SW_SRRCTL_DESC_TYPE_msb 15
+#define FM10K_SW_SRRCTL_DESC_TYPE_NO_SPLIT 0
+#define FM10K_SW_SRRCTL_DESC_TYPE_HDR_SPLIT 1
+#define FM10K_SW_SRRCTL_DESC_TYPE_SIZE_SPLIT 2
+#define FM10K_SW_SRRCTL_PSR_TYPE_lsb 16
+#define FM10K_SW_SRRCTL_PSR_TYPE_msb 29
+#define FM10K_SW_SRRCTL_LOOPBACK_SUPPRESS (1 << 30)
+#define FM10K_SW_SRRCTL_BUFFER_CHAINING_EN (1 << 31)
+#define FM10K_SW_QPRC(q_) \
+ FM10K_SW_REG_OFF(0x400A + 0x40 * (q_))
+#define FM10K_SW_QPRDC(q_) \
+ FM10K_SW_REG_OFF(0x400B + 0x40 * (q_))
+#define FM10K_SW_QBRC_L(q_) \
+ FM10K_SW_REG_OFF(0x400C + 0x40 * (q_))
+#define FM10K_SW_QBRC_H(q_) \
+ FM10K_SW_REG_OFF(0x400D + 0x40 * (q_))
+#define FM10K_SW_RX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x400E + 0x40 * (q_))
+#define FM10K_SW_TDBAL(q_) \
+ FM10K_SW_REG_OFF(0x8000 + 0x40 * (q_))
+#define FM10K_SW_TDBAH(q_) \
+ FM10K_SW_REG_OFF(0x8001 + 0x40 * (q_))
+#define FM10K_SW_TDLEN(q_) \
+ FM10K_SW_REG_OFF(0x8002 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL(q_) \
+ FM10K_SW_REG_OFF(0x8003 + 0x40 * (q_))
+#define FM10K_SW_TPH_TXCTRL_DESC_TPHEN (1 << 5)
+#define FM10K_SW_TPH_TXCTRL_DESC_READ_RO_EN (1 << 9)
+#define FM10K_SW_TPH_TXCTRL_DESC_WRITE_RO_EN (1 << 11)
+#define FM10K_SW_TPH_TXCTRL_DATA_READ_RO_EN (1 << 13)
+#define FM10K_SW_TDH(q_) \
+ FM10K_SW_REG_OFF(0x8004 + 0x40 * (q_))
+#define FM10K_SW_TDT(q_) \
+ FM10K_SW_REG_OFF(0x8005 + 0x40 * (q_))
+#define FM10K_SW_TDT_TDT_lsb 0
+#define FM10K_SW_TDT_TDT_msb 15
+#define FM10K_SW_TXDCTL(q_) \
+ FM10K_SW_REG_OFF(0x8006 + 0x40 * (q_))
+#define FM10K_SW_TXDCTL_PTHRESH_lsb 0
+#define FM10K_SW_TXDCTL_PTHRESH_msb 6
+#define FM10K_SW_TXDCTL_HTHRESH_lsb 7
+#define FM10K_SW_TXDCTL_HTHRESH_msb 13
+#define FM10K_SW_TXDCTL_ENABLE (1 << 14)
+#define FM10K_SW_TXDCTL_MAX_TIME_lsb 16
+#define FM10K_SW_TXDCTL_MAX_TIME_msb 27
+#define FM10K_SW_TXDCTL_PUSH_DESC (1 << 28)
+#define FM10K_SW_TXQCTL(q_) \
+ FM10K_SW_REG_OFF(0x8007 + 0x40 * (q_))
+#define FM10K_SW_TXQCTL_VF_lsb 0
+#define FM10K_SW_TXQCTL_VF_msb 5
+#define FM10K_SW_TXQCTL_OWNED_BY_VF (1 << 6)
+#define FM10K_SW_TXQCTL_PC_lsb 7
+#define FM10K_SW_TXQCTL_PC_msb 9
+#define FM10K_SW_TXQCTL_TC_lsb 10
+#define FM10K_SW_TXQCTL_TC_msb 15
+#define FM10K_SW_TXQCTL_VID_lsb 16
+#define FM10K_SW_TXQCTL_VID_msb 27
+#define FM10K_SW_TXQCTL_UNLIMITED_BW (1 << 28)
+#define FM10K_SW_TXQCTL_PUSH_MODE_DIS (1 << 29)
+#define FM10K_SW_TXINT(q_) \
+ FM10K_SW_REG_OFF(0x8008 + 0x40 * (q_))
+#define FM10K_SW_TXINT_INT_lsb 0
+#define FM10K_SW_TXINT_INT_msb 7
+#define FM10K_SW_TXINT_INT_TIMER_lsb 8
+#define FM10K_SW_TXINT_INT_TIMER_msb 9
+#define FM10K_SW_QPTC(q_) \
+ FM10K_SW_REG_OFF(0x8009 + 0x40 * (q_))
+#define FM10K_SW_QBTC_L(q_) \
+ FM10K_SW_REG_OFF(0x800A + 0x40 * (q_))
+#define FM10K_SW_QBTC_H(q_) \
+ FM10K_SW_REG_OFF(0x800B + 0x40 * (q_))
+#define FM10K_SW_TQDLOC(q_) \
+ FM10K_SW_REG_OFF(0x800C + 0x40 * (q_))
+#define FM10K_SW_TQDLOC_BASE_lsb 0
+#define FM10K_SW_TQDLOC_BASE_msb 15
+#define FM10K_SW_TQDLOC_SIZE_lsb 16
+#define FM10K_SW_TQDLOC_SIZE_msb 19
+#define FM10K_SW_TQDLOC_SIZE_32 5
+#define FM10K_SW_TQDLOC_SIZE_64 6
+#define FM10K_SW_TQDLOC_SIZE_128 7
+#define FM10K_SW_TX_SGLORT(q_) \
+ FM10K_SW_REG_OFF(0x800D + 0x40 * (q_))
+#define FM10K_SW_TX_SGLORT_SGLORT_lsb 0
+#define FM10K_SW_TX_SGLORT_SGLORT_msb 15
+#define FM10K_SW_PFVTCTL(q_) \
+ FM10K_SW_REG_OFF(0x800E + 0x40 * (q_))
+#define FM10K_SW_TX_DESC(q_, d_, w_) \
+ FM10K_SW_REG_OFF(0x40000 + \
+ 0x400 * (q_) + 0x4 * (d_) + (w_))
+#define FM10K_SW_PBACL(n_) \
+ FM10K_SW_REG_OFF(0x10000 + (n_))
+#define FM10K_SW_INT_MAP(n_) \
+ FM10K_SW_REG_OFF(0x10080 + (n_))
+#define FM10K_SW_INT_MAP_ENTRIES 8
+#define FM10K_SW_INT_MAP_INDEX_MAILBOX 0
+#define FM10K_SW_INT_MAP_INDEX_FAULT 1
+#define FM10K_SW_INT_MAP_INDEX_SWITCH_UP_DOWN 2
+#define FM10K_SW_INT_MAP_INDEX_SWITCH 3
+#define FM10K_SW_INT_MAP_INDEX_SRAM 4
+#define FM10K_SW_INT_MAP_INDEX_VFLR 5
+#define FM10K_SW_INT_MAP_INDEX_MAX_HOLD_TIME 6
+#define FM10K_SW_INT_MAP_INT_lsb 0
+#define FM10K_SW_INT_MAP_INT_msb 7
+#define FM10K_SW_INT_MAP_INT_TIMER_lsb 8
+#define FM10K_SW_INT_MAP_INT_TIMER_msb 9
+#define FM10K_SW_MSIX_VECTOR(v_) \
+ FM10K_SW_REG_OFF(0x11000 + 0x4 * (v_))
+#define FM10K_SW_INT_CTRL \
+ FM10K_SW_REG_OFF(0x12000)
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_lsb 0
+#define FM10K_SW_INT_CTRL_NEXT_VECTOR_msb 9
+#define FM10K_SW_INT_CTRL_ENABLE_MODERATOR (1 << 10)
+#define FM10K_SW_ITR(v_) \
+ FM10K_SW_REG_OFF(0x12400 + (v_))
+
+/*
+ * Interrupt throttle timer intervals in microseconds. These provide the
+ * direct values for programming the ITR interval field when using a Gen3
+ * PCLK, otherwise they need to be scaled appropriately.
+ */
+#define FM10K_SW_ITR_INTERVAL_20K 50
+#define FM10K_SW_ITR_INTERVAL_40K 25
+#define FM10K_SW_ITR_INTERVAL_0_lsb 0
+#define FM10K_SW_ITR_INTERVAL_0_msb 11
+#define FM10K_SW_ITR_INTERVAL_1_lsb 12
+#define FM10K_SW_ITR_INTERVAL_1_msb 23
+#define FM10K_SW_ITR_TIMER_0_EXPIRED (1 << 24)
+#define FM10K_SW_ITR_TIMER_1_EXPIRED (1 << 25)
+#define FM10K_SW_ITR_PENDING_0 (1 << 26)
+#define FM10K_SW_ITR_PENDING_1 (1 << 27)
+#define FM10K_SW_ITR_PENDING_2 (1 << 28)
+#define FM10K_SW_ITR_AUTO_MASK (1 << 29)
+#define FM10K_SW_ITR_MASK_lsb 30
+#define FM10K_SW_ITR_MASK_msb 31
+#define FM10K_SW_ITR_MASK_R_ENABLED 0
+#define FM10K_SW_ITR_MASK_R_BLOCKED 1
+#define FM10K_SW_ITR_MASK_W_KEEP 0
+#define FM10K_SW_ITR_MASK_W_BLOCK 1
+#define FM10K_SW_ITR_MASK_W_ENABLE 2
+#define FM10K_SW_ITR2(v_) \
+ FM10K_SW_REG_OFF(0x12800 + 0x2 * (v_))
+#define FM10K_SW_IP FM10K_SW_REG_OFF(0x13000)
+#define FM10K_SW_IP_HOT_RESET (1 << 0)
+#define FM10K_SW_IP_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IP_MAILBOX (1 << 2)
+#define FM10K_SW_IP_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IP_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IP_PFLR (1 << 5)
+#define FM10K_SW_IP_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IP_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IP_NOT_IN_RESET (1 << 8)
+#define FM10K_SW_IP_TIMEOUT (1 << 9)
+#define FM10K_SW_IP_VFLR (1 << 10)
+#define FM10K_SW_IM FM10K_SW_REG_OFF(0x13001)
+#define FM10K_SW_IM_ALL FM10K_SW_MASK32(10, 0)
+#define FM10K_SW_IM_HOT_RESET (1 << 0)
+#define FM10K_SW_IM_DEVICE_STATE_CHANGE (1 << 1)
+#define FM10K_SW_IM_MAILBOX (1 << 2)
+#define FM10K_SW_IM_VPD_REQUEST (1 << 3)
+#define FM10K_SW_IM_SRAM_ERROR (1 << 4)
+#define FM10K_SW_IM_PFLR (1 << 5)
+#define FM10K_SW_IM_DATA_PATH_RESET (1 << 6)
+#define FM10K_SW_IM_OUT_OF_RESET (1 << 7)
+#define FM10K_SW_IM_TIMEOUT (1 << 9)
+#define FM10K_SW_IM_VFLR (1 << 10)
+#define FM10K_SW_IB FM10K_SW_REG_OFF(0x13002)
+#define FM10K_SW_SRAM_IP FM10K_SW_REG_OFF(0x13003)
+#define FM10K_SW_SRAM_IM FM10K_SW_REG_OFF(0x13004)
+#define FM10K_SW_VLAN_TABLE(f_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_))
+#define FM10K_SW_VLAN_TABLE_ENTRIES 128
+#define FM10K_SW_VLAN_TABLE_ENTRY(f_, n_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + (n_))
+#define FM10K_SW_VLAN_TABLE_VLAN_ENTRY(f_, vl_) \
+ FM10K_SW_REG_OFF(0x14000 + 0x80 * (f_) + ((vl_) >> 5))
+#define FM10K_SW_VLAN_TABLE_VLAN_BIT(vl_) (1 << ((vl_) & 0x1f))
+#define FM10K_SW_MBMEM(n_) \
+ FM10K_SW_REG_OFF(0x18000 + (n_))
+#define FM10K_SW_MBX(vf_) \
+ FM10K_SW_REG_OFF(0x18800 + (vf_))
+#define FM10K_SW_MBICR(vf_) \
+ FM10K_SW_REG_OFF(0x18840 + ((vf_) >> 5))
+#define FM10K_SW_GMBX FM10K_SW_REG_OFF(0x18842)
+#define FM10K_SW_GMBX_GLOBAL_REQ (1 << 1)
+#define FM10K_SW_GMBX_GLOBAL_ACK (1 << 2)
+#define FM10K_SW_GMBX_PF_REQ_INTERRUPT (1 << 3)
+#define FM10K_SW_GMBX_PF_ACK_INTERRUPT (1 << 4)
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_lsb 5
+#define FM10K_SW_GMBX_PF_INTERRUPT_ENABLE_msb 6
+#define FM10K_SW_GMBX_INTERRUPT_NO_CHANGE 0
+#define FM10K_SW_GMBX_INTERRUPT_ENABLE 1
+#define FM10K_SW_GMBX_INTERRUPT_DISABLE 2
+#define FM10K_SW_GMBX_PF_REQ (1 << 7)
+#define FM10K_SW_GMBX_PF_ACK (1 << 8)
+#define FM10K_SW_GMBX_GLOBAL_REQ_INTERRUPT (1 << 9)
+#define FM10K_SW_GMBX_GLOBAL_ACK_INTERRUPT (1 << 10)
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_lsb 11
+#define FM10K_SW_GMBX_GLOBAL_INTERRUPT_ENABLE_msb 12
+#define FM10K_SW_PFVFLRE FM10K_SW_REG_OFF(0x18844)
+#define FM10K_SW_PFVFLREC FM10K_SW_REG_OFF(0x18846)
+#define FM10K_SW_TEST_CFG0 FM10K_SW_REG_OFF(0x18849)
+#define FM10K_SW_INT_SRAM_CTRL FM10K_SW_REG_OFF(0x18850)
+#define FM10K_SW_FUM_SRAM_CTRL FM10K_SW_REG_OFF(0x18852)
+#define FM10K_SW_PCA_SRAM_CTRL FM10K_SW_REG_OFF(0x18854)
+#define FM10K_SW_PP_SRAM_CTRL FM10K_SW_REG_OFF(0x18858)
+#define FM10K_SW_PCW_SRAM_CTRL FM10K_SW_REG_OFF(0x1885C)
+#define FM10K_SW_RHI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18872)
+#define FM10K_SW_RHI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18860)
+#define FM10K_SW_THI_SRAM_CTRL1 FM10K_SW_REG_OFF(0x18864)
+#define FM10K_SW_THI_SRAM_CTRL2 FM10K_SW_REG_OFF(0x18868)
+#define FM10K_SW_TIMEOUT_CFG FM10K_SW_REG_OFF(0x1886B)
+#define FM10K_SW_LVMMC FM10K_SW_REG_OFF(0x18880)
+#define FM10K_SW_LVMMI FM10K_SW_REG_OFF(0x18881)
+#define FM10K_SW_HOST_MISC FM10K_SW_REG_OFF(0x19000)
+#define FM10K_SW_HOST_LANE_CTRL FM10K_SW_REG_OFF(0x19001)
+#define FM10K_SW_SERDES_CTRL(l_) \
+ FM10K_SW_REG_OFF(0x19010 + 0x2 * (l_))
+
+/*
+ * BAR4 registers
+ */
+
+/*
+ * Access to non-master PEP registers via BAR4
+ */
+#define FM10K_SW_PCIE_GLOBAL(p_, r_) \
+ (FM10K_SW_REG_OFF(((p_) + 1) << 20) | FM10K_SW_##r_)
+
+/*
+ * SBUS register fields for use with both the PCIE and EPL SBUS interfaces.
+ */
+#define FM10K_SW_SBUS_CFG_SBUS_CONTROLLER_RESET (1 << 0)
+#define FM10K_SW_SBUS_CFG_ROM_ENABLE (1 << 1)
+#define FM10K_SW_SBUS_CFG_ROM_BUSY (1 << 2)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_PASS (1 << 3)
+#define FM10K_SW_SBUS_CFG_BIST_DONE_FAIL (1 << 4)
+
+#define FM10K_SW_SBUS_COMMAND_REGISTER_lsb 0
+#define FM10K_SW_SBUS_COMMAND_REGISTER_msb 7
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_lsb 8
+#define FM10K_SW_SBUS_COMMAND_ADDRESS_msb 15
+#define FM10K_SW_SBUS_COMMAND_OP_lsb 16
+#define FM10K_SW_SBUS_COMMAND_OP_msb 23
+#define FM10K_SW_SBUS_OP_RESET 0x20
+#define FM10K_SW_SBUS_OP_WRITE 0x21
+#define FM10K_SW_SBUS_OP_READ 0x22
+#define FM10K_SW_SBUS_COMMAND_EXECUTE (1 << 24)
+#define FM10K_SW_SBUS_COMMAND_BUSY (1 << 25)
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_lsb 26
+#define FM10K_SW_SBUS_COMMAND_RESULT_CODE_msb 28
+#define FM10K_SW_SBUS_RESULT_RESET 0x00
+#define FM10K_SW_SBUS_RESULT_WRITE 0x01
+#define FM10K_SW_SBUS_RESULT_READ 0x04
+
+#define FM10K_SW_SBUS_ADDR_EPL_RMON2 1
+#define FM10K_SW_SBUS_ADDR_EPL_LANE(e_, l_) (4 * (e_) + (l_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_SERDES(s_) ((s_) + 2)
+#define FM10K_SW_SBUS_ADDR_EPL_RMON3 38
+#define FM10K_SW_SBUS_ADDR_EPL_PMRO 39
+
+/* Common to both EPL and PCIE
+ */
+#define FM10K_SW_SBUS_ADDR_SPICO 253
+#define FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER 254
+#define FM10K_SW_SBUS_ADDR_BROADCAST 0xFF
+
+
+#define FM10K_SW_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x000000)
+#define FM10K_SW_MGMT_REG(wo_) \
+ (FM10K_SW_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_FATAL_CODE FM10K_SW_MGMT_REG(0x0)
+#define FM10K_SW_LAST_FATAL_CODE FM10K_SW_MGMT_REG(0x1)
+#define FM10K_SW_FATAL_COUNT FM10K_SW_MGMT_REG(0x2)
+#define FM10K_SW_SOFT_RESET FM10K_SW_MGMT_REG(0x3)
+#define FM10K_SW_SOFT_RESET_COLD_RESET (1 << 0)
+#define FM10K_SW_SOFT_RESET_EPL_RESET (1 << 1)
+#define FM10K_SW_SOFT_RESET_SWITCH_RESET (1 << 2)
+#define FM10K_SW_SOFT_RESET_SWITCH_READY (1 << 3)
+#define FM10K_SW_SOFT_RESET_PCIE_RESET(p_) (1 << ((p_) + 4))
+#define FM10K_SW_SOFT_RESET_PCIE_ACTIVE(p_) (1 << ((p_) + 13))
+#define FM10K_SW_DEVICE_CFG FM10K_SW_MGMT_REG(0x4)
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS 4
+#define FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(p_) (1 << (p_))
+#define FM10K_SW_DEVICE_CFG_PCIE_100G_DIS (1 << 4)
+#define FM10K_SW_DEVICE_CFG_FEATURE_lsb 5
+#define FM10K_SW_DEVICE_CFG_FEATURE_msb 6
+#define FM10K_SW_DEVICE_CFG_PCIE_FULL 0
+#define FM10K_SW_DEVICE_CFG_PCIE_HALF 1
+#define FM10K_SW_DEVICE_CFG_PCIE_BASIC 2
+#define FM10K_SW_DEVICE_CFG_PCIE_EN(p_) (1 << (7 + (p_)))
+#define FM10K_SW_RESET_CFG FM10K_SW_MGMT_REG(0x5)
+#define FM10K_SW_WATCHDOG_CFG FM10K_SW_MGMT_REG(0x6)
+#define FM10K_SW_MGMT_SCRATCH(n_) FM10K_SW_MGMT_REG(0x8 + (n_))
+#define FM10K_SW_VITAL_PRODUCT_DATA FM10K_SW_MGMT_REG(0x304)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x400)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_lsb64 0
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM_msb64 8
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_lsb64 9
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE_msb64 17
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_lsb64 18
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL_msb64 26
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_lsb64 27
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL_msb64 28
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CORE (1ULL << 29)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SOFTWARE (1ULL << 30)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_GPIO (1ULL << 31)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C (1ULL << 32)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_MDIO (1ULL << 33)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_CRM (1ULL << 34)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_TAIL (1ULL << 35)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FH_HEAD (1ULL << 36)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_PINS (1ULL << 39)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_FIBM (1ULL << 40)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_BSM (1ULL << 41)
+#define FM10K_SW_GLOBAL_INTERRUPT_DETECT_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_INT \
+ FM10K_SW_MGMT_REG(0x402)
+#define FM10K_SW_INTERRUPT_MASK_PCIE(p_) \
+ FM10K_SW_MGMT_REG(0x420 + 0x2 * (p_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_lsb64 0
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM_msb64 8
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_BSM(n_) (1ULL << (n_))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_lsb64 9
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE_msb64 17
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PCIE(n_) \
+ (1ULL << ((n_) + 9))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_lsb64 18
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL_msb64 26
+#define FM10K_SW_INTERRUPT_MASK_PCIE_EPL(n_) \
+ (1ULL << ((n_) + 18))
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_lsb64 27
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL_msb64 28
+#define FM10K_SW_INTERRUPT_MASK_PCIE_TUNNEL(n_) \
+ (1ULL << ((n_) + 27))
+
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CORE (1ULL << 29)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SOFTWARE (1ULL << 30)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_GPIO (1ULL << 31)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_I2C (1ULL << 32)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_MDIO (1ULL << 33)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_CRM (1ULL << 34)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_TAIL (1ULL << 35)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FH_HEAD (1ULL << 36)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_EPL (1ULL << 37)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_SBUS_PCIE (1ULL << 38)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_PINS (1ULL << 39)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_FIBM (1ULL << 40)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_BSM (1ULL << 41)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_XCLK (1ULL << 42)
+#define FM10K_SW_INTERRUPT_MASK_PCIE_ALL FM10K_SW_MASK64(42, 0)
+#define FM10K_SW_INTERRUPT_MASK_FIBM FM10K_SW_MGMT_REG(0x440)
+#define FM10K_SW_INTERRUPT_MASK_BSM FM10K_SW_MGMT_REG(0x442)
+#define FM10K_SW_CORE_INTERRUPT_DETECT FM10K_SW_MGMT_REG(0x444)
+#define FM10K_SW_CORE_INTERRUPT_MASK FM10K_SW_MGMT_REG(0x445)
+#define FM10K_SW_SRAM_ERR_IP FM10K_SW_MGMT_REG(0x446)
+#define FM10K_SW_SRAM_ERR_IM FM10K_SW_MGMT_REG(0x448)
+#define FM10K_SW_PINS_STAT FM10K_SW_MGMT_REG(0x44A)
+#define FM10K_SW_PINS_IP FM10K_SW_MGMT_REG(0x44B)
+#define FM10K_SW_PINS_IM FM10K_SW_MGMT_REG(0x44C)
+#define FM10K_SW_SW_IP FM10K_SW_MGMT_REG(0x44D)
+#define FM10K_SW_SW_IM FM10K_SW_MGMT_REG(0x44E)
+#define FM10K_SW_SW_TEST_AND_SET FM10K_SW_MGMT_REG(0x44F)
+#define FM10K_SW_LSM_CLKOBS_CTRL FM10K_SW_MGMT_REG(0x450)
+#define FM10K_SW_CHIP_VERSION FM10K_SW_MGMT_REG(0x452)
+#define FM10K_SW_BSM_SCRATCH(n_) FM10K_SW_MGMT_REG(0x800 + (n_))
+#define FM10K_SW_BSM_CTRL FM10K_SW_MGMT_REG(0xC00)
+#define FM10K_SW_BSM_ARGS FM10K_SW_MGMT_REG(0xC01)
+#define FM10K_SW_BSM_ADDR_OFFSET(n_) FM10K_SW_MGMT_REG(0xC04 + (n_))
+#define FM10K_SW_BSM_COUNTER(n_) FM10K_SW_MGMT_REG(0xC08 + (n_))
+#define FM10K_SW_BSM_SRAM_CTRL FM10K_SW_MGMT_REG(0xC0A)
+#define FM10K_SW_BSM_IP FM10K_SW_MGMT_REG(0xC0B)
+#define FM10K_SW_BSM_IM FM10K_SW_MGMT_REG(0xC0C)
+#define FM10K_SW_PIN_STRAP_STAT FM10K_SW_MGMT_REG(0xC0D)
+#define FM10K_SW_FUSE_DATA_0 FM10K_SW_MGMT_REG(0xC0E)
+#define FM10K_SW_FUSE_SKU_lsb 11
+#define FM10K_SW_FUSE_SKU_msb 15
+#define FM10K_SW_FUSE_SKU_FM10840 0
+#define FM10K_SW_FUSE_SKU_FM10420 1
+#define FM10K_SW_FUSE_SKU_FM10064 2
+#define FM10K_SW_FUSE_DATA_1 FM10K_SW_MGMT_REG(0xC0F)
+#define FM10K_SW_BIST_CTRL FM10K_SW_MGMT_REG(0xC10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_MODE_EPL (1ULL << 41)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC (1ULL << 42)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL (1ULL << 43)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_BSM (1ULL << 44)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_CRM (1ULL << 45)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_FIBM (1ULL << 46)
+#define FM10K_SW_BIST_CTRL_BIST_MODE_SBM (1ULL << 47)
+#define FM10K_SW_REI_CTRL FM10K_SW_MGMT_REG(0xC12)
+#define FM10K_SW_REI_STAT FM10K_SW_MGMT_REG(0xC13)
+#define FM10K_SW_GPIO_CFG FM10K_SW_MGMT_REG(0xC15)
+#define FM10K_SW_GPIO_DATA FM10K_SW_MGMT_REG(0xC16)
+#define FM10K_SW_GPIO_IP FM10K_SW_MGMT_REG(0xC17)
+#define FM10K_SW_GPIO_IM FM10K_SW_MGMT_REG(0xC18)
+#define FM10K_SW_I2C_CFG FM10K_SW_MGMT_REG(0xC19)
+#define FM10K_SW_I2C_CFG_SLAVE_ENABLE (1 << 0)
+#define FM10K_SW_I2C_CFG_ADDR_lsb 1
+#define FM10K_SW_I2C_CFG_ADDR_msb 7
+#define FM10K_SW_I2C_CFG_DIVIDER_lsb 8
+#define FM10K_SW_I2C_CFG_DIVIDER_msb 19
+#define FM10K_SW_I2C_CFG_DIVIDER_100_KHZ 52
+#define FM10K_SW_I2C_CFG_DIVIDER_400_KHZ 10
+#define FM10K_SW_I2C_CFG_INTERRUPT_MASK (1 << 20)
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_lsb 21
+#define FM10K_SW_I2C_CFG_DEBOUNCE_FILTER_COUNT_LIMIT_msb 27
+#define FM10K_SW_I2C_DATA(n_) FM10K_SW_MGMT_REG(0xC1C + (n_))
+#define FM10K_SW_I2C_CTRL FM10K_SW_MGMT_REG(0xC20)
+#define FM10K_SW_I2C_CTRL_ADDR_lsb 0
+#define FM10K_SW_I2C_CTRL_ADDR_msb 7
+#define FM10K_SW_I2C_CTRL_COMMAND_lsb 8
+#define FM10K_SW_I2C_CTRL_COMMAND_msb 9
+#define FM10K_SW_I2C_COMMAND_NULL 0
+#define FM10K_SW_I2C_COMMAND_WR 1
+#define FM10K_SW_I2C_COMMAND_WR_RD 2
+#define FM10K_SW_I2C_COMMAND_RD 3
+#define FM10K_SW_I2C_CTRL_LENGTH_W_lsb 10
+#define FM10K_SW_I2C_CTRL_LENGTH_W_msb 13
+#define FM10K_SW_I2C_CTRL_LENGTH_R_lsb 14
+#define FM10K_SW_I2C_CTRL_LENGTH_R_msb 17
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_lsb 18
+#define FM10K_SW_I2C_CTRL_LENGTH_SENT_msb 21
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_lsb 22
+#define FM10K_SW_I2C_CTRL_COMMAND_COMPLETED_msb 25
+#define FM10K_SW_I2C_COMPLETION_RUNNING 0
+#define FM10K_SW_I2C_COMPLETION_NORMAL 1
+#define FM10K_SW_I2C_COMPLETION_PREMATURE 2
+#define FM10K_SW_I2C_COMPLETION_NO_DEVICE 3
+#define FM10K_SW_I2C_COMPLETION_TIMEOUT 4
+#define FM10K_SW_I2C_COMPLETION_LOST_ARB 5
+#define FM10K_SW_I2C_COMPLETION_BUS_WAIT 6
+#define FM10K_SW_I2C_COMPLETION_INVALID 7
+#define FM10K_SW_I2C_CTRL_INTERRUPT_PENDING (1 << 26)
+#define FM10K_SW_MDIO_CFG FM10K_SW_MGMT_REG(0xC22)
+#define FM10K_SW_MDIO_DATA FM10K_SW_MGMT_REG(0xC23)
+#define FM10K_SW_MDIO_CTRL FM10K_SW_MGMT_REG(0xC24)
+#define FM10K_SW_SPI_TX_DATA FM10K_SW_MGMT_REG(0xC26)
+#define FM10K_SW_SPI_RX_DATA FM10K_SW_MGMT_REG(0xC27)
+#define FM10K_SW_SPI_HEADER FM10K_SW_MGMT_REG(0xC28)
+#define FM10K_SW_SPI_CTRL FM10K_SW_MGMT_REG(0xC29)
+#define FM10K_SW_LED_CFG FM10K_SW_MGMT_REG(0xC2B)
+#define FM10K_SW_SCAN_DATA_IN FM10K_SW_MGMT_REG(0xC2D)
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_lsb 0
+#define FM10K_SW_SCAN_DATA_IN_SCAN_DATA_msb 24
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
+#define FM10K_SW_SCAN_DATA_IN_INJECT (1 << 28)
+#define FM10K_SW_SCAN_DATA_IN_DRAIN (1 << 29)
+#define FM10K_SW_SCAN_DATA_IN_PASSTHRU (1 << 30)
+#define FM10K_SW_SCAN_DATA_IN_SINGLE (1 << 31)
+#define FM10K_SW_CRM_DATA(m_, n_) \
+ FM10K_SW_MGMT_REG(0x1000 + 0x2 * (m_) + (n_))
+
+#define FM10K_SW_CRM_CTRL FM10K_SW_MGMT_REG(0x2000)
+#define FM10K_SW_CRM_STATUS FM10K_SW_MGMT_REG(0x2001)
+#define FM10K_SW_CRM_TIME FM10K_SW_MGMT_REG(0x2002)
+#define FM10K_SW_CRM_SRAM_CTRL FM10K_SW_MGMT_REG(0x2004)
+#define FM10K_SW_CRM_IP FM10K_SW_MGMT_REG(0x2008)
+#define FM10K_SW_CRM_IM FM10K_SW_MGMT_REG(0x200C)
+#define FM10K_SW_CRM_COMMAND(n_) FM10K_SW_MGMT_REG(0x2080 + 0x2 * (n_))
+#define FM10K_SW_CRM_REGISTER(n_) FM10K_SW_MGMT_REG(0x2100 + 0x2 * (n_))
+#define FM10K_SW_CRM_PERIOD(n_) FM10K_SW_MGMT_REG(0x2180 + 0x2 * (n_))
+#define FM10K_SW_CRM_PARAM(n_) FM10K_SW_MGMT_REG(0x2200 + (n_))
+#define FM10K_SW_PLL_PCIE_CTRL FM10K_SW_MGMT_REG(0x2241)
+#define FM10K_SW_PLL_PCIE_STAT FM10K_SW_MGMT_REG(0x2242)
+#define FM10K_SW_SBUS_PCIE_CFG FM10K_SW_MGMT_REG(0x2243)
+#define FM10K_SW_SBUS_PCIE_COMMAND FM10K_SW_MGMT_REG(0x2244)
+#define FM10K_SW_SBUS_PCIE_REQUEST FM10K_SW_MGMT_REG(0x2245)
+#define FM10K_SW_SBUS_PCIE_RESPONSE FM10K_SW_MGMT_REG(0x2246)
+#define FM10K_SW_SBUS_PCIE_SPICO_IN FM10K_SW_MGMT_REG(0x2247)
+#define FM10K_SW_SBUS_PCIE_SPICO_OUT FM10K_SW_MGMT_REG(0x2248)
+#define FM10K_SW_SBUS_PCIE_IP FM10K_SW_MGMT_REG(0x2249)
+#define FM10K_SW_SBUS_PCIE_IM FM10K_SW_MGMT_REG(0x224A)
+#define FM10K_SW_MGMT_SYSTIME_CFG FM10K_SW_MGMT_REG(0x224C)
+#define FM10K_SW_MGMT_SYSTIME FM10K_SW_MGMT_REG(0x224E)
+#define FM10K_SW_MGMT_SYSTIME0 FM10K_SW_MGMT_REG(0x2250)
+#define FM10K_SW_SYSTIME_PULSE_(n_) FM10K_SW_MGMT_REG(0x2252 + (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_LO(n_) FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_))
+#define FM10K_SW_SYSTIME_CAPTURE_HI(n_) \
+ FM10K_SW_MGMT_REG(0x2258 + 0x2 * (n_) + 0x1)
+#define FM10K_SW_PCIE_XPLL_CTRL FM10K_SW_MGMT_REG(0x3000)
+#define FM10K_SW_PCIE_CLK_CTRL FM10K_SW_MGMT_REG(0x3001)
+#define FM10K_SW_PCIE_CLK_CTRL2 FM10K_SW_MGMT_REG(0x3002)
+#define FM10K_SW_PCIE_CLKMON_RATIO_CFG FM10K_SW_MGMT_REG(0x3003)
+#define FM10K_SW_PCIE_CLKMON_TOLERANCE_CFG FM10K_SW_MGMT_REG(0x3004)
+#define FM10K_SW_PCIE_CLKMON_DEADLINES_CFG FM10K_SW_MGMT_REG(0x3005)
+#define FM10K_SW_PCIE_CLK_STAT FM10K_SW_MGMT_REG(0x3006)
+#define FM10K_SW_PCIE_CLK_IP FM10K_SW_MGMT_REG(0x3007)
+#define FM10K_SW_PCIE_CLK_IM FM10K_SW_MGMT_REG(0x3008)
+#define FM10K_SW_PCIE_WARM_RESET_DELAY FM10K_SW_MGMT_REG(0x3009)
+#define FM10K_SW_EPL_BASE FM10K_SW_REG_OFF(0x0E0000)
+#define FM10K_SW_EPL_PORT_REG(p_, r_) \
+ (FM10K_SW_EPL_BASE + FM10K_SW_REG_OFF(0x400 * (p_) + (r_)))
+#define FM10K_SW_EPL_LANE_REG(p_, l_, r_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x80 * (l_) + (r_))
+
+/* EPL enumerated types
+ */
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX 0
+#define FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1 1
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_XX_L4 2
+#define FM10K_SW_EPL_QPL_MODE_XX_XX_L4_XX 3
+#define FM10K_SW_EPL_QPL_MODE_XX_L4_XX_XX 4
+#define FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX 5
+#define FM10K_SW_EPL_PCS_SEL_DISABLE 0
+#define FM10K_SW_EPL_PCS_SEL_AN_73 1
+#define FM10K_SW_EPL_PCS_SEL_SGMII_10 2
+#define FM10K_SW_EPL_PCS_SEL_SGMII_100 3
+#define FM10K_SW_EPL_PCS_SEL_SGMII_1000 4
+#define FM10K_SW_EPL_PCS_SEL_1000BASEX 5
+#define FM10K_SW_EPL_PCS_SEL_10GBASER 6
+#define FM10K_SW_EPL_PCS_SEL_40GBASER 7
+#define FM10K_SW_EPL_PCS_SEL_100GBASER 8
+
+#define FM10K_SW_LANE_OVERRIDE_NORMAL 0
+#define FM10K_SW_LANE_OVERRIDE_FORCE_GOOD 1
+#define FM10K_SW_LANE_OVERRIDE_FORCE_BAD 2
+
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSTHRU 0
+#define FM10K_SW_TX_MAX_FCS_MODE_PASSSTHRU_CHECK 1
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_GOOD 2
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_BAD 3
+#define FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL 4
+
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_DRAIN 0
+#define FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL 1
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_NORMAL 2
+#define FM10K_SW_TX_MAC_DRAIN_MODE_HOLD_HOLD 3
+
+#define FM10K_SW_TX_MAC_FAULT_MODE_NORMAL 0
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_IDLE 1
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LOCAL_FAULT 2
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_REMOTE_FAULT 3
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_LINK_INTERRUPTION 4
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_OK 5
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_ERROR 6
+#define FM10K_SW_TX_MAC_FAULT_MODE_FORCE_USER_VAL 7
+
+#define FM10K_SW_EPL_IP(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x300)
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_lsb 0
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT_msb 3
+#define FM10K_SW_EPL_IP_AN_PORT_INTERRUPT(n_) (1 << ((n_) + 0))
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_lsb 4
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT_msb 7
+#define FM10K_SW_EPL_IP_LINK_PORT_INTERRUPT(n_) (1 << ((n_) + 4))
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_lsb 8
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT_msb 11
+#define FM10K_SW_EPL_IP_SERDES_INTERRUPT(n_) (1 << ((n_) + 8))
+#define FM10K_SW_EPL_IP_ERROR_INTERRUPT (1 << 12)
+
+#define FM10K_SW_EPL_ERROR_IP(p_) FM10K_SW_EPL_PORT_REG((p_), 0x301)
+#define FM10K_SW_EPL_ERROR_IM(p_) FM10K_SW_EPL_PORT_REG((p_), 0x302)
+#define FM10K_SW_EPL_ERROR_IM_ALL FM10K_SW_MASK32(8, 0)
+#define FM10K_SW_EPL_BIST_STATUS(p_) FM10K_SW_EPL_PORT_REG((p_), 0x303)
+#define FM10K_SW_EPL_CFG_A(p_) FM10K_SW_EPL_PORT_REG((p_), 0x304)
+#define FM10K_SW_EPL_CFG_A_SPEED_UP (1 << 0)
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_lsb 1
+#define FM10K_SW_EPL_CFG_A_TIMEOUT_msb 6
+#define FM10K_SW_EPL_CFG_A_ACTIVE(p_) (1 << ((p_) + 7))
+#define FM10K_SW_EPL_CFG_A_ACTIVE_QUAD (0xf << 7)
+
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_lsb 11
+#define FM10K_SW_EPL_CFG_A_SKEW_TOLERANCE_msb 16
+
+#define FM10K_SW_EPL_CFG_B(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x305)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_lsb(p_) ((p_) * 4)
+#define FM10K_SW_EPL_CFG_B_PCS_SEL_msb(p_) (((p_) * 4) + 3)
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_lsb 16
+#define FM10K_SW_EPL_CFG_B_QPL_MODE_msb 18
+#define FM10K_SW_EPL_LED_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x306)
+
+#define FM10K_SW_EPL_LED_STATUS_PORT_RESET(n_) (1 << ((n_) * 6 + 0))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP(n_) (1 << ((n_) * 6 + 1))
+#define FM10K_SW_EPL_LED_STATUS_PORT_LOCAL_FAULT(n_) (1 << ((n_) * 6 + 2))
+#define FM10K_SW_EPL_LED_STATUS_PORT_REMOTE_FAULT(n_) (1 << ((n_) * 6 + 3))
+#define FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING(n_) (1 << ((n_) * 6 + 4))
+#define FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING(n_) (1 << ((n_) * 6 + 5))
+
+#define FM10K_SW_EPL_FIFO_ERROR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x307)
+#define FM10K_SW_EPL_TX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x308)
+#define FM10K_SW_EPL_TX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x309)
+#define FM10K_SW_EPL_TX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30A)
+#define FM10K_SW_EPL_TX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30B)
+#define FM10K_SW_EPL_TX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30C)
+#define FM10K_SW_EPL_TX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30D)
+#define FM10K_SW_EPL_RX_FIFO_RD_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30E)
+#define FM10K_SW_EPL_RX_FIFO_WR_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x30F)
+#define FM10K_SW_EPL_RX_FIFO_A_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x310)
+#define FM10K_SW_EPL_RX_FIFO_B_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x311)
+#define FM10K_SW_EPL_RX_FIFO_C_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x312)
+#define FM10K_SW_EPL_RX_FIFO_D_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x313)
+#define FM10K_SW_PCS_ML_BASER_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x314)
+#define FM10K_SW_PCS_ML_BASER_RX_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x315)
+#define FM10K_SW_PCS_100GBASER_BIP_0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x317)
+#define FM10K_SW_PCS_100GBASER_BIP_1(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x318)
+#define FM10K_SW_PCS_100GBASER_BIP_2(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x319)
+#define FM10K_SW_PCS_100GBASER_BIP_3(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31A)
+#define FM10K_SW_PCS_100GBASER_BIP_4(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31B)
+#define FM10K_SW_PCS_100GBASER_BIP_5(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31C)
+#define FM10K_SW_PCS_100GBASER_BIP_6(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31D)
+#define FM10K_SW_PCS_100GBASER_BIP_7(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31E)
+#define FM10K_SW_PCS_100GBASER_BIP_8(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x31F)
+#define FM10K_SW_PCS_100GBASER_BIP_9(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x320)
+#define FM10K_SW_PCS_100GBASER_BIP_10(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x321)
+#define FM10K_SW_PCS_100GBASER_BIP_11(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x322)
+#define FM10K_SW_PCS_100GBASER_BIP_12(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x323)
+#define FM10K_SW_PCS_100GBASER_BIP_13(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x324)
+#define FM10K_SW_PCS_100GBASER_BIP_14(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x325)
+#define FM10K_SW_PCS_100GBASER_BIP_15(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x326)
+#define FM10K_SW_PCS_100GBASER_BIP_16(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x327)
+#define FM10K_SW_PCS_100GBASER_BIP_17(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x328)
+#define FM10K_SW_PCS_100GBASER_BIP_18(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x329)
+#define FM10K_SW_PCS_100GBASER_BIP_19(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32A)
+#define FM10K_SW_PCS_100GBASER_BLOCK_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32B)
+#define FM10K_SW_PCS_100GBASER_AMPS_LOCK(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32C)
+#define FM10K_SW_RS_FEC_UNCORRECTED(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x33D)
+#define FM10K_SW_RS_FEC_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x32E)
+#define FM10K_SW_RS_FEC_STATUS(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x330)
+#define FM10K_SW_EPL_SYSTIME(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x331)
+#define FM10K_SW_EPL_SYSTIME0(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x332)
+#define FM10K_SW_EPL_SYSTIME_CFG(p_) \
+ FM10K_SW_EPL_PORT_REG((p_), 0x333)
+#define FM10K_SW_PORT_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x0)
+#define FM10K_SW_AN_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1)
+
+#define FM10K_SW_AN_IM_ALL FM10K_SW_MASK32(18, 0)
+#define FM10K_SW_LINK_IM(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2)
+#define FM10K_SW_LINK_IM_ALL FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_AN_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3)
+#define FM10K_SW_LINK_IP(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4)
+#define FM10K_SW_MP_CFG(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_73_PAGE_RX(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_LINK_RULES(p_, l_) FM10K_SW_EPL_LANE_REG((p_), (l_), 0xC)
+
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_lsb 0
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_UP_msb 3
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_lsb 4
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_UP_msb 8
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_lsb 9
+#define FM10K_SW_LINK_RULES_FAULT_TIME_SCALE_DOWN_msb 12
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_lsb 13
+#define FM10K_SW_LINK_RULES_FAULT_TICKS_DOWN_msb 17
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_lsb 18
+#define FM10K_SW_LINK_RULES_HEARTBEAT_TIME_SCALE_msb 21
+#define FM10K_SW_MAC_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x10)
+#define FM10K_SW_MAC_CFG_ARRAY_SIZE 8
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_lsb 0
+#define FM10K_SW_MAC_CFG_TX_ANTI_BUBBLE_WATERMARK_msb 5
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_lsb 6
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_WATERMARK_msb 9
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_lsb 10
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_FAST_INC_msb 17
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_lsb 18
+#define FM10K_SW_MAC_CFG_TX_RATE_FIFO_SLOW_INC_msb 25
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_lsb 26
+#define FM10K_SW_MAC_CFG_TX_IDLE_MIN_IFG_BYTES_msb 31
+
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_lsb 32
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT_msb 47
+#define FM10K_SW_MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE_bit 48
+
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_lsb 49
+#define FM10K_SW_MAC_CFG_TX_FAULT_MODE_msb 51
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_lsb 52
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIME_SCALE_msb 55
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_lsb 56
+#define FM10K_SW_MAC_CFG_TX_PC_ACT_TIMEOUT_msb 63
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_lsb 64
+#define FM10K_SW_MAC_CFG_TX_DRAIN_MODE_msb 65
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_lsb 66
+#define FM10K_SW_MAC_CFG_TX_MIN_COLUMNS_msb 71
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_lsb 72
+#define FM10K_SW_MAC_CFG_TX_SEG_MIN_SPACING_msb 83
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_lsb 84
+#define FM10K_SW_MAC_CFG_TX_SEG_MAX_CREDIT_msb 95
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_lsb 96
+#define FM10K_SW_MAC_CFG_TX_SEG_SIZE_msb 107
+#define FM10K_SW_MAC_CFG_TX_LP_IDLE_REQUEST_bit 108
+#define FM10K_SW_MAC_CFG_TX_LPI_AUTOMATIC_bit 109
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_lsb 110
+#define FM10K_SW_MAC_CFG_TX_LPI_TIMEOUT_msb 117
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_lsb 118
+#define FM10K_SW_MAC_CFG_TX_LPI_TIME_SCALE_msb 119
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_lsb 120
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIMEOUT_msb 127
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_lsb 128
+#define FM10K_SW_MAC_CFG_TX_LPI_HOLD_TIME_SCALE_msb 129
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_lsb 130
+#define FM10K_SW_MAC_CFG_TX_FCS_MODE_msb 132
+#define FM10K_SW_MAC_CFG_TX_OBEY_LINT_bit 133
+#define FM10K_SW_MAC_CFG_TX_IDLE_ENABLE_DIC_bit 134
+#define FM10K_SW_MAC_CFG_CJPAT_ENABLE_bit 135
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_lsb 136
+#define FM10K_SW_MAC_CFG_RX_MIN_FRAME_LENGTH_msb 143
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_lsb 144
+#define FM10K_SW_MAC_CFG_RX_MAX_FRAME_LENGTH_msb 159
+#define FM10K_SW_MAC_CFG_START_CHAR_D_lsb 160
+#define FM10K_SW_MAC_CFG_START_CHAR_D_msb 167
+#define FM10K_SW_MAC_CFG_IEEE_1588_ENABLE_bit 168
+#define FM10K_SW_MAC_CFG_FCS_START_bit 169
+#define FM10K_SW_MAC_CFG_PREAMBLE_MODE_bit 170
+#define FM10K_SW_MAC_CFG_COUNTER_WRAP_bit 171
+#define FM10K_SW_MAC_CFG_LINK_FAULT_DISABLE_bit 172
+#define FM10K_SW_MAC_CFG_RX_DRAIN_bit 173
+#define FM10K_SW_MAC_CFG_RX_FCS_FORCE_BAD_bit 174
+#define FM10K_SW_MAC_CFG_RX_IGNORE_CODE_ERRORS_bit 175
+#define FM10K_SW_MAC_CFG_RX_IGNORE_UNDERSIZE_ERRORS_bit 176
+#define FM10K_SW_MAC_CFG_RX_IGNORE_OVERSIZE_ERRORS_bit 177
+#define FM10K_SW_MAC_CFG_RX_IGNORE_FCS_ERRORS_bit 178
+#define FM10K_SW_MAC_CFG_RX_IGNORE_PREAMBLE_ERRORS_bit 179
+#define FM10K_SW_MAC_CFG_RX_IGNORE_IFG_ERRORS_bit 180
+#define FM10K_SW_MAC_CFG_ERR_WRITE_lsb 181
+#define FM10K_SW_MAC_CFG_ERR_WRITE_msb 182
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_lsb 183
+#define FM10K_SW_MAC_CFG_RX_MIN_EVENT_RATE_msb 186
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_lsb 187
+#define FM10K_SW_MAC_CFG_RX_PC_REQUEST_msb 191
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_lsb 192
+#define FM10K_SW_MAC_CFG_RX_PC_SEG_SIZE_msb 196
+
+#define FM10K_SW_TX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1A)
+#define FM10K_SW_RX_SEQUENCE(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1C)
+#define FM10K_SW_MAC_1588_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x1E)
+#define FM10K_SW_WAKE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x20)
+#define FM10K_SW_MAC_OVERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x21)
+#define FM10K_SW_MAC_JABBER_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x22)
+#define FM10K_SW_MAC_UNDERSIZE_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x23)
+#define FM10K_SW_MAC_RUNT_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x24)
+#define FM10K_SW_MAC_OVERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x25)
+#define FM10K_SW_MAC_UNDERRUN_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x26)
+#define FM10K_SW_MAC_CODE_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x27)
+#define FM10K_SW_EPL_TX_FRAME_ERROR_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x28)
+#define FM10K_SW_MAC_LINK_COUNTER(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x29)
+#define FM10K_SW_PCS_1000BASEX_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2A)
+#define FM10K_SW_PCS_1000BASEX_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2B)
+#define FM10K_SW_PCS_1000BASEX_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2C)
+#define FM10K_SW_PCS_10GBASER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2D)
+#define FM10K_SW_PCS_10GBASER_RX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2E)
+#define FM10K_SW_PCS_10GBASER_TX_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x2F)
+#define FM10K_SW_AN_37_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x30)
+#define FM10K_SW_AN_37_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_37_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_37_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_SGMII_AN_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_SGMII_AN_TX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_TX_CONFIG_LOOPBACK(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_SGMII_AN_RX_CONFIG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x8)
+#define FM10K_SW_AN_37_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x32)
+#define FM10K_SW_AN_73_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x33)
+#define FM10K_SW_AN_73_TIMER_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x34)
+#define FM10K_SW_AN_73_BASE_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_BASE_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_NEXT_PAGE_TX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x6)
+#define FM10K_SW_AN_73_NEXT_PAGE_RX(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0xA)
+#define FM10K_SW_AN_73_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x36)
+#define FM10K_SW_AN_73_TX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x38)
+#define FM10K_SW_AN_73_RX_LCW(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3A)
+#define FM10K_SW_PCSL_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x3C)
+
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_lsb 0
+#define FM10K_SW_PCSL_CFG_SLIP_TIME_msb 7
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE (1 << 8)
+#define FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL (1 << 9)
+#define FM10K_SW_PCSL_CFG_RX_GB_NARROW (1 << 10)
+#define FM10K_SW_PCSL_CFG_TX_GB_NARROW (1 << 11)
+
+#define FM10K_SW_MP_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_1000BASEX_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_PCS_10GBASER_EEE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x40)
+#define FM10K_SW_MP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_40GBASER_RX_BIP_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_PCS_10GBASER_RX_BER_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_DISPARITY_ERROR_8B10B(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x44)
+#define FM10K_SW_LANE_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x45)
+#define FM10K_SW_LANE_SERDES_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x46)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x47)
+
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_OVERRIDE_msb 1
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT (1 << 5)
+
+#define FM10K_SW_LANE_ACTIVITY_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x48)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x49)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_lsb 0
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE_msb 1
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK (1 << 2)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY (1 << 3)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY (1 << 4)
+#define FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT (1 << 5)
+#define FM10K_SW_LANE_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4A)
+#define FM10K_SW_LANE_SERDES_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4B)
+
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_lsb 0
+#define FM10K_SW_LANE_SERDES_STATUS_ANALOG_TO_CORE_msb 7
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_lsb 8
+#define FM10K_SW_LANE_SERDES_STATUS_CORE_STATUS_msb 23
+#define FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK (1 << 12)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_RDY (1 << 24)
+#define FM10K_SW_LANE_SERDES_STATUS_TX_RDY (1 << 25)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_IDLE_DETECT (1 << 26)
+#define FM10K_SW_LANE_SERDES_STATUS_RX_ACTIVITY (1 << 27)
+#define FM10K_SW_SERDES_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4C)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IM_CORE_STATUS_4
+#define FM10K_SW_SERDES_IM_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IM_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IM_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IM_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IM_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IM_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IM_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IM_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IM_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IM_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IM_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IM_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IM_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IM_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IM_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_SERDES_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4D)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_0 (1 << 0)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_1 (1 << 1)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_2 (1 << 2)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_3 (1 << 3)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_4 (1 << 4)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_OK \
+ FM10K_SW_SERDES_IP_CORE_STATUS_4
+#define FM10K_SW_SERDES_IP_CORE_STATUS_5 (1 << 5)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_6 (1 << 6)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_7 (1 << 7)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_8 (1 << 8)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_9 (1 << 9)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_10 (1 << 10)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_11 (1 << 11)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_12 (1 << 12)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_13 (1 << 13)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_14 (1 << 14)
+#define FM10K_SW_SERDES_IP_CORE_STATUS_15 (1 << 15)
+#define FM10K_SW_SERDES_IP_TX_RDY (1 << 16)
+#define FM10K_SW_SERDES_IP_RX_ENERGY_DETECT (1 << 17)
+#define FM10K_SW_SERDES_IP_RX_SIGNAL_DETECT (1 << 18)
+#define FM10K_SW_SERDES_IP_RX_RDY (1 << 19)
+#define FM10K_SW_SERDES_IP_RX_ACTIVITY (1 << 20)
+#define FM10K_SW_SERDES_IP_RX_IDLE_DETECT (1 << 21)
+#define FM10K_SW_SERDES_IP_SAI_COMPLETE (1 << 22)
+#define FM10K_SW_SERDES_IP_SAI_REQUEST_ERROR (1 << 23)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_U_ERR (1 << 24)
+#define FM10K_SW_SERDES_IP_TX_CDC_FIFO_ERROR (1 << 25)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_U_ERR (1 << 26)
+#define FM10K_SW_SERDES_IP_RX_CDC_FIFO_ERROR (1 << 27)
+#define FM10K_SW_SERDES_IP_SLIP_REQUEST (1 << 28)
+#define FM10K_SW_SERDES_IP_ANALOG_IP (1 << 29)
+#define FM10K_SW_SERDES_IP_ALL \
+ FM10K_SW_MASK32(29, 0)
+#define FM10K_SW_LANE_ANALOG_IM(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4E)
+#define FM10K_SW_LANE_ANALOG_IP(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x4F)
+#define FM10K_SW_LANE_SAI_CFG(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x50)
+#define FM10K_SW_LANE_SAI_CFG_CODE_lsb64 0
+#define FM10K_SW_LANE_SAI_CFG_CODE_msb64 15
+#define FM10K_SW_LANE_SAI_CFG_DATA_lsb64 16
+#define FM10K_SW_LANE_SAI_CFG_DATA_msb64 31
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_lsb64 32
+#define FM10K_SW_LANE_SAI_CFG_RESULT_PATTERN_msb64 47
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_lsb64 48
+#define FM10K_SW_LANE_SAI_CFG_RESULT_MODE_msb64 49
+#define FM10K_SW_LANE_SAI_CFG_REQUEST (1ULL << 50)
+#define FM10K_SW_LANE_SAI_STATUS(p_, l_) \
+ FM10K_SW_EPL_LANE_REG((p_), (l_), 0x52)
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_lsb 0
+#define FM10K_SW_LANE_SAI_STATUS_RESULT_msb 15
+#define FM10K_SW_LANE_SAI_STATUS_COMPLETE (1 << 16)
+#define FM10K_SW_LANE_SAI_STATUS_ACCESS_REQUEST (1 << 17)
+#define FM10K_SW_LANE_SAI_STATUS_IN_PROGRESS (1 << 18)
+#define FM10K_SW_LANE_SAI_STATUS_BUSY (1 << 19)
+#define FM10K_SW_LANE_SAI_STATUS_REQUEST_ERROR (1 << 20)
+
+#define FM10K_SW_PORTS_MGMT_BASE \
+ FM10K_SW_REG_OFF(0x0E8000)
+#define FM10K_SW_PORTS_MGMT_REG(wo_) \
+ (FM10K_SW_PORTS_MGMT_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_PLL_EPL_CTRL FM10K_SW_PORTS_MGMT_REG(0x0)
+#define FM10K_SW_PLL_EPL_STAT FM10K_SW_PORTS_MGMT_REG(0x1)
+#define FM10K_SW_PLL_FABRIC_CTRL FM10K_SW_PORTS_MGMT_REG(0x2)
+
+#define FM10K_SW_PLL_FABRIC_REFDIV_lsb 3
+#define FM10K_SW_PLL_FABRIC_REFDIV_msb 8
+#define FM10K_SW_PLL_FABRIC_FBDIV4_lsb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV4_msb 9
+#define FM10K_SW_PLL_FABRIC_FBDIV255_lsb 10
+#define FM10K_SW_PLL_FABRIC_FBDIV255_msb 17
+#define FM10K_SW_PLL_FABRIC_OUTDIV_lsb 18
+#define FM10K_SW_PLL_FABRIC_OUTDIV_msb 23
+#define FM10K_SW_PLL_FABRIC_STAT FM10K_SW_PORTS_MGMT_REG(0x3)
+#define FM10K_SW_PLL_FABRIC_LOCK FM10K_SW_PORTS_MGMT_REG(0x4)
+#define FM10K_SW_PLL_FABRIC_FREQSEL_lsb 4
+#define FM10K_SW_PLL_FABRIC_FREQSEL_msb 7
+#define FM10K_SW_PLL_FABRIC_FREQSEL_CTRL 0
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F600 1
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F500 2
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F400 3
+#define FM10K_SW_PLL_FABRIC_FREQSEL_F300 4
+#define FM10K_SW_SBUS_EPL_CFG FM10K_SW_PORTS_MGMT_REG(0x5)
+#define FM10K_SW_SBUS_EPL_COMMAND FM10K_SW_PORTS_MGMT_REG(0x6)
+#define FM10K_SW_SBUS_EPL_REQUEST FM10K_SW_PORTS_MGMT_REG(0x7)
+#define FM10K_SW_SBUS_EPL_RESPONSE FM10K_SW_PORTS_MGMT_REG(0x8)
+#define FM10K_SW_SBUS_EPL_SPICO_IN FM10K_SW_PORTS_MGMT_REG(0x9)
+#define FM10K_SW_SBUS_EPL_SPICO_OUT FM10K_SW_PORTS_MGMT_REG(0xA)
+#define FM10K_SW_SBUS_EPL_IP FM10K_SW_PORTS_MGMT_REG(0xB)
+#define FM10K_SW_SBUS_EPL_IM FM10K_SW_PORTS_MGMT_REG(0xC)
+
+#define FM10K_SW_ETHCLK_CFG(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0xE + (n_))
+#define FM10K_SW_ETHCLK_RATIO(n_) \
+ FM10K_SW_PORTS_MGMT_REG(0x10 + (n_))
+#define FM10K_SW_PM_CLKOBS_CTRL FM10K_SW_PORTS_MGMT_REG(0x12)
+
+#define FM10K_SW_PCIE_CFG_BASE FM10K_SW_REG_OFF(0x120000)
+#define FM10K_SW_PCIE_CFG_REG(wo_) \
+ (FM10K_SW_PCIE_CFG_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * These register offsets can also be passed to fm10k_read_config(), which
+ * will mask off the upper bits
+ */
+#define FM10K_SW_PCIE_CFG_ID FM10K_SW_PCIE_CFG_REG(0x0)
+#define FM10K_SW_PCIE_CFG_CMD FM10K_SW_PCIE_CFG_REG(0x1)
+#define FM10K_SW_PCIE_CFG_1 FM10K_SW_PCIE_CFG_REG(0x2)
+#define FM10K_SW_PCIE_CFG_2 FM10K_SW_PCIE_CFG_REG(0x3)
+#define FM10K_SW_PCIE_CFG_BAR0 FM10K_SW_PCIE_CFG_REG(0x4)
+#define FM10K_SW_PCIE_CFG_BAR1 FM10K_SW_PCIE_CFG_REG(0x5)
+#define FM10K_SW_PCIE_CFG_BAR2 FM10K_SW_PCIE_CFG_REG(0x6)
+#define FM10K_SW_PCIE_CFG_BAR3 FM10K_SW_PCIE_CFG_REG(0x7)
+#define FM10K_SW_PCIE_CFG_BAR4 FM10K_SW_PCIE_CFG_REG(0x8)
+#define FM10K_SW_PCIE_CFG_BAR5 FM10K_SW_PCIE_CFG_REG(0x9)
+#define FM10K_SW_PCIE_CFG_CARDBUS FM10K_SW_PCIE_CFG_REG(0xA)
+#define FM10K_SW_PCIE_CFG_SUBID FM10K_SW_PCIE_CFG_REG(0xB)
+#define FM10K_SW_PCIE_CFG_EXP_ROM FM10K_SW_PCIE_CFG_REG(0xC)
+#define FM10K_SW_PCIE_CFG_CAP_PTR FM10K_SW_PCIE_CFG_REG(0xD)
+#define FM10K_SW_PCIE_CFG_RSVD FM10K_SW_PCIE_CFG_REG(0xE)
+#define FM10K_SW_PCIE_CFG_INT FM10K_SW_PCIE_CFG_REG(0xF)
+#define FM10K_SW_PCIE_CFG_PM_CAP FM10K_SW_PCIE_CFG_REG(0x10)
+#define FM10K_SW_PCIE_CFG_PM_CTRL FM10K_SW_PCIE_CFG_REG(0x11)
+#define FM10K_SW_PCIE_CFG_PCIE_CAP FM10K_SW_PCIE_CFG_REG(0x1C)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP FM10K_SW_PCIE_CFG_REG(0x1D)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL FM10K_SW_PCIE_CFG_REG(0x1E)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CAP FM10K_SW_PCIE_CFG_REG(0x1F)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL FM10K_SW_PCIE_CFG_REG(0x20)
+
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_lsb 16
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_msb 19
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_2P5 1
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_5 2
+#define FM10K_SW_PCIE_CFG_LINK_SPEED_8 3
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_lsb 20
+#define FM10K_SW_PCIE_CFG_LINK_WIDTH_msb 24
+
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CAP2 FM10K_SW_PCIE_CFG_REG(0x25)
+#define FM10K_SW_PCIE_CFG_PCIE_DEV_CTRL2 FM10K_SW_PCIE_CFG_REG(0x26)
+#define FM10K_SW_PCIE_CFG_PCIE_LINK_CTRL2 FM10K_SW_PCIE_CFG_REG(0x28)
+#define FM10K_SW_PCIE_CFG_MSIX_CAP FM10K_SW_PCIE_CFG_REG(0x2C)
+#define FM10K_SW_PCIE_CFG_MSIX_TABLE_OFFSET FM10K_SW_PCIE_CFG_REG(0x2D)
+#define FM10K_SW_PCIE_CFG_MSIX_PBA FM10K_SW_PCIE_CFG_REG(0x2E)
+#define FM10K_SW_PCIE_CFG_VPD_CAP FM10K_SW_PCIE_CFG_REG(0x34)
+#define FM10K_SW_PCIE_CFG_VPD_DATA FM10K_SW_PCIE_CFG_REG(0x35)
+#define FM10K_SW_PCIE_CFG_AER_HDR FM10K_SW_PCIE_CFG_REG(0x40)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_STATUS FM10K_SW_PCIE_CFG_REG(0x41)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_MASK FM10K_SW_PCIE_CFG_REG(0x42)
+#define FM10K_SW_PCIE_CFG_AER_UNERR_SEVERITY FM10K_SW_PCIE_CFG_REG(0x43)
+#define FM10K_SW_PCIE_CFG_AER_COERR_STATUS FM10K_SW_PCIE_CFG_REG(0x44)
+#define FM10K_SW_PCIE_CFG_AER_COERR_MASK FM10K_SW_PCIE_CFG_REG(0x45)
+#define FM10K_SW_PCIE_CFG_AER_CTRL FM10K_SW_PCIE_CFG_REG(0x46)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG0 FM10K_SW_PCIE_CFG_REG(0x47)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG1 FM10K_SW_PCIE_CFG_REG(0x48)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG2 FM10K_SW_PCIE_CFG_REG(0x49)
+#define FM10K_SW_PCIE_CFG_AER_HEADER_LOG3 FM10K_SW_PCIE_CFG_REG(0x4A)
+#define FM10K_SW_PCIE_CFG_SPD_HDR FM10K_SW_PCIE_CFG_REG(0x52)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_L FM10K_SW_PCIE_CFG_REG(0x53)
+#define FM10K_SW_PCIE_CFG_SPD_NUMBER_H FM10K_SW_PCIE_CFG_REG(0x54)
+#define FM10K_SW_PCIE_CFG_ARI_HDR FM10K_SW_PCIE_CFG_REG(0x56)
+#define FM10K_SW_PCIE_CFG_ARI_CTRL FM10K_SW_PCIE_CFG_REG(0x57)
+#define FM10K_SW_PCIE_CFG_SPCIE_HDR FM10K_SW_PCIE_CFG_REG(0x5A)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_CTRL3 FM10K_SW_PCIE_CFG_REG(0x5B)
+#define FM10K_SW_PCIE_CFG_SPCIE_ERR_STS FM10K_SW_PCIE_CFG_REG(0x5C)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ01 FM10K_SW_PCIE_CFG_REG(0x5D)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ23 FM10K_SW_PCIE_CFG_REG(0x5E)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ45 FM10K_SW_PCIE_CFG_REG(0x5F)
+#define FM10K_SW_PCIE_CFG_SPCIE_LINK_EQ67 FM10K_SW_PCIE_CFG_REG(0x60)
+#define FM10K_SW_PCIE_CFG_SRIOV_HDR FM10K_SW_PCIE_CFG_REG(0x62)
+#define FM10K_SW_PCIE_CFG_SRIOV_CAP FM10K_SW_PCIE_CFG_REG(0x63)
+#define FM10K_SW_PCIE_CFG_SRIOV_CTRL FM10K_SW_PCIE_CFG_REG(0x64)
+#define FM10K_SW_PCIE_CFG_SRIOV_CFG FM10K_SW_PCIE_CFG_REG(0x65)
+#define FM10K_SW_PCIE_CFG_SRIOV_NUM FM10K_SW_PCIE_CFG_REG(0x66)
+#define FM10K_SW_PCIE_CFG_SRIOV_MAP FM10K_SW_PCIE_CFG_REG(0x67)
+#define FM10K_SW_PCIE_CFG_SRIOV_DEVID FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_SUP FM10K_SW_PCIE_CFG_REG(0x69)
+#define FM10K_SW_PCIE_CFG_SRIOV_PAGE_CFG FM10K_SW_PCIE_CFG_REG(0x6A)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR0 FM10K_SW_PCIE_CFG_REG(0x6B)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR1 FM10K_SW_PCIE_CFG_REG(0x6C)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR2 FM10K_SW_PCIE_CFG_REG(0x6D)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR3 FM10K_SW_PCIE_CFG_REG(0x6E)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR4 FM10K_SW_PCIE_CFG_REG(0x6F)
+#define FM10K_SW_PCIE_CFG_SRIOV_BAR5 FM10K_SW_PCIE_CFG_REG(0x70)
+#define FM10K_SW_PCIE_CFG_SRIOV_MIG FM10K_SW_PCIE_CFG_REG(0x71)
+#define FM10K_SW_PCIE_CFG_TPH_HDR FM10K_SW_PCIE_CFG_REG(0x72)
+#define FM10K_SW_PCIE_CFG_TPH_CAP FM10K_SW_PCIE_CFG_REG(0x73)
+#define FM10K_SW_PCIE_CFG_TPH_CTRL FM10K_SW_PCIE_CFG_REG(0x74)
+#define FM10K_SW_PCIE_CFG_ACS_HDR FM10K_SW_PCIE_CFG_REG(0x76)
+#define FM10K_SW_PCIE_CFG_ACS_CAP FM10K_SW_PCIE_CFG_REG(0x77)
+#define FM10K_SW_PCIE_PORTLOGIC FM10K_SW_PCIE_CFG_REG(0x1C0)
+#define FM10K_SW_PCIE_PORTLOGIC_LINK_STATE FM10K_SW_PCIE_CFG_REG(0x1CA)
+
+#define FM10K_SW_FFU_BASE FM10K_SW_REG_OFF(0xC00000)
+#define FM10K_SW_FFU_REG(wo_) \
+ (FM10K_SW_FFU_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_FFU_NUM_SLICES 32
+#define FM10K_SW_FFU_NUM_SCENARIOS 32
+
+/* FFU enumerated types */
+#define FM10K_SW_FFU_MUX_SEL_MAP_DIP_MAP_SIP 0
+#define FM10K_SW_FFU_MUX_SEL_MAP_DMAC_MAP_SMAC 1
+#define FM10K_SW_FFU_MUX_SEL_MAP_PROT_MAP_LENGTH 2
+#define FM10K_SW_FFU_MUX_SEL_MAP_SRC_MAP_TYPE 3
+#define FM10K_SW_FFU_MUX_SEL_USER 4
+#define FM10K_SW_FFU_MUX_SEL_FTYPE_SWPRI 5
+#define FM10K_SW_FFU_MUX_SEL_IPMISC 6
+#define FM10K_SW_FFU_MUX_SEL_TOS 7
+#define FM10K_SW_FFU_MUX_SEL_PROT 8
+#define FM10K_SW_FFU_MUX_SEL_TTL 9
+#define FM10K_SW_FFU_MUX_SEL_SRC_PORT 10
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID_11_8 11
+#define FM10K_SW_FFU_MUX_SEL_VID_7_0 12
+#define FM10K_SW_FFU_MUX_SEL_RXTAG 13
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_15_0 14
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_31_16 15
+#define FM10K_SW_FFU_MUX_SEL_L2_DMAC_47_32 16
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_15_0 17
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_31_16 18
+#define FM10K_SW_FFU_MUX_SEL_L2_SMAC_47_32 19
+#define FM10K_SW_FFU_MUX_SEL_DGLORT 20
+#define FM10K_SW_FFU_MUX_SEL_SGLORT 21
+#define FM10K_SW_FFU_MUX_SEL_VPRI_VID 22
+#define FM10K_SW_FFU_MUX_SEL_VPRI2_VID2 23
+#define FM10K_SW_FFU_MUX_SEL_L2_TYPE 24
+#define FM10K_SW_FFU_MUX_SEL_L4_DST 25
+#define FM10K_SW_FFU_MUX_SEL_L4_SRC 26
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_DST 27
+#define FM10K_SW_FFU_MUX_SEL_MAP_L4_SRC 28
+#define FM10K_SW_FFU_MUX_SEL_L4A 29
+#define FM10K_SW_FFU_MUX_SEL_L4B 30
+#define FM10K_SW_FFU_MUX_SEL_L4C 31
+#define FM10K_SW_FFU_MUX_SEL_L4D 32
+#define FM10K_SW_FFU_MUX_SEL_MAP_VPRI1_VID1 33
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_31_0 34
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_63_32 35
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_95_64 36
+#define FM10K_SW_FFU_MUX_SEL_L3_DIP_127_96 37
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_31_0 38
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_63_32 39
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64 40
+#define FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96 41
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_INDEX_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_COUNT_msb 19
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_ARP_TYPE_EXP (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_ROUTE_GLORT_FLOOD_SET (1 << 20)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_MASK_msb 7
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_lsb 8
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_BYTE_DATA_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_BITS_SUB_CMD_msb 20
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_VLAN_msb 11
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_lsb 16
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_TX_TAG_msb 17
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_VLAN_SET_PRI (1 << 19)
+
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_lsb 0
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_DSCP_msb 5
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_lsb 12
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_PRI_msb 15
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_DSCP (1 << 17)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_VPRI (1 << 18)
+#define FM10K_SW_FFU_SLICE_SRAM_SET_PRI_SET_PRI (1 << 19)
+
+/* FFU Registers */
+#define FM10K_SW_FFU_SLICE_TCAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x2000 * (sl_) + 0x4 * (n_))
+#define FM10K_SW_FFU_SLICE_TCAM_ENTRIES 1024
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_lsb64 0
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_msb64 31
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_lsb64 32
+#define FM10K_SW_FFU_SLICE_TCAM_KEY_TOP_msb64 39
+#define FM10K_SW_FFU_SLICE_SRAM(sl_, n_) \
+ FM10K_SW_FFU_REG(0x1000 + 0x2000 * (sl_) + 0x2 * (n_))
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_lsb64 21
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_msb64 22
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_ARP 0
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT 1
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_BIT_SET 2
+#define FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET 3
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_lsb64 23
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_INDEX_msb64 34
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_lsb64 35
+#define FM10K_SW_FFU_SLICE_SRAM_COUNTER_BANK_msb64 36
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_lsb64 37
+#define FM10K_SW_FFU_SLICE_SRAM_PRECEDENCE_msb64 39
+#define FM10K_SW_FFU_SLICE_VALID(sl_) \
+ FM10K_SW_FFU_REG(0x1800 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_VALID_SCENARIO(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_FFU_SLICE_CASCADE_ACTION(sl_) \
+ FM10K_SW_FFU_REG(0x1804 + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG(sl_, scen_) \
+ FM10K_SW_FFU_REG(0x1840 + 0x2 * (scen_) + 0x2000 * (sl_))
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_lsb64 0
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_0_msb64 5
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_lsb64 6
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_1_msb64 11
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_lsb64 12
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_2_msb64 17
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_lsb64 18
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_3_msb64 23
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_lsb64 24
+#define FM10K_SW_FFU_SLICE_CFG_SELECT_TOP_msb64 29
+#define FM10K_SW_FFU_SLICE_CFG_START_COMPARE (1ULL << 30)
+#define FM10K_SW_FFU_SLICE_CFG_START_ACTION (1ULL << 31)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_LOW (1ULL << 32)
+#define FM10K_SW_FFU_SLICE_CFG_VALID_HIGH (1ULL << 33)
+#define FM10K_SW_FFU_SLICE_CFG_CASE_lsb64 34
+#define FM10K_SW_FFU_SLICE_CFG_CASE_msb64 37
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_lsb64 38
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_msb64 39
+
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_NOT_MAPPED 0
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_LOW_NIBBLE 1
+#define FM10K_SW_FFU_SLICE_CFG_CASE_LOCATION_TOP_HIGH_NIBBLE 2
+
+#define FM10K_SW_FFU_MASTER_VALID \
+ FM10K_SW_FFU_REG(0x40000)
+#define FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(s_) (1ULL << (s_))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_SLICES_VALID FM10K_SW_MASK64(31, 0)
+#define FM10K_SW_FFU_MASTER_VALID_CHUNK_VALID(c_) (1ULL << ((c_) + 32))
+#define FM10K_SW_FFU_MASTER_VALID_ALL_CHUNKS_VALID FM10K_SW_MASK64(63, 32)
+
+#define FM10K_SW_L2LOOKUP_BASE \
+ FM10K_SW_REG_OFF(0xC80000)
+#define FM10K_SW_L2LOOKUP_REG(wo_) \
+ (FM10K_SW_L2LOOKUP_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_MA_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x10000 * (t_) + 0x4 * (n_))
+#define FM10K_SW_INGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x20000 + 0x4 * (v_))
+#define FM10K_SW_INGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative
+ * to the start of the upper 64-bit word
+ */
+#define FM10K_SW_INGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_COUNTER_INDEX_msb64 (89 - 64)
+#define FM10K_SW_INGRESS_VID_TABLE_REFLECT (1ULL << (90 - 64))
+#define FM10K_SW_INGRESS_VID_TABLE_TRAP_IGMP (1ULL << (91 - 64))
+#define FM10K_SW_EGRESS_VID_TABLE(v_) \
+ FM10K_SW_L2LOOKUP_REG(0x24000 + 0x4 * (v_))
+#define FM10K_SW_EGRESS_VID_TABLE_ENTRIES 4096
+#define FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP(l_) (1ULL << (l_))
+
+/* note these bit positions are relative to
+ * the start of the upper 64-bit word
+ */
+#define FM10K_SW_EGRESS_VID_TABLE_FID_lsb64 (64 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_FID_msb64 (75 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_lsb64 (76 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MST_INDEX_msb64 (83 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_lsb64 (84 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_MTU_INDEX_msb64 (86 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_lsb64 (87 - 64)
+#define FM10K_SW_EGRESS_VID_TABLE_TRIG_ID_msb64 (92 - 64)
+
+#define FM10K_SW_MA_USED_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28000 + 0x200 * (t_) + (n_))
+#define FM10K_SW_INGRESS_MST_TABLE(t_, n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28400 + 0x200 * (t_) + 0x2 * (n_))
+
+#define FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE 24
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) \
+ (((l_) % FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) * 2)
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_msb64(l_) \
+ (FM10K_SW_INGRESS_MST_TABLE_STP_STATE_lsb64(l_) + 1)
+
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_DISABLE 0
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LISTENING 1
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_LEARNING 2
+#define FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD 3
+
+#define FM10K_SW_EGRESS_MST_TABLE(t_) \
+ FM10K_SW_L2LOOKUP_REG(0x28800 + 0x2 * (t_))
+#define FM10K_SW_EGRESS_MST_TABLE_FORWARDING(l_) (1ULL << (l_))
+
+#define FM10K_SW_MA_TABLE_CFG_1 \
+ FM10K_SW_L2LOOKUP_REG(0x28A00)
+#define FM10K_SW_MA_TABLE_CFG_2 \
+ FM10K_SW_L2LOOKUP_REG(0x28A02)
+#define FM10K_SW_MTU_TABLE(n_) \
+ FM10K_SW_L2LOOKUP_REG(0x28A08 + (n_))
+#define FM10K_SW_IEEE_RESERVED_MAC_ACTION \
+ FM10K_SW_L2LOOKUP_REG(0x28A10)
+#define FM10K_SW_IEEE_RESERVED_MAC_TRAP_PRIORITY \
+ FM10K_SW_L2LOOKUP_REG(0x28A14)
+#define FM10K_SW_IEEE_RESERVED_MAC_CFG \
+ FM10K_SW_L2LOOKUP_REG(0x28A16)
+
+
+#define FM10K_SW_GLORT_BASE FM10K_SW_REG_OFF(0xCE0000)
+#define FM10K_SW_GLORT_REG(wo_) \
+ (FM10K_SW_GLORT_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_GLORT_CAM_ENTRIES 256
+#define FM10K_SW_GLORT_DEST_TABLE(n_) \
+ FM10K_SW_GLORT_REG(0x0 + 0x2 * (n_))
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_lsb64 0
+#define FM10K_SW_GLORT_DEST_TABLE_MASK_msb64 47
+
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_lsb64 48
+#define FM10K_SW_GLORT_DEST_TABLE_IP_MCAST_IDX_msb64 59
+
+#define FM10K_SW_GLORT_CAM(n_) \
+ FM10K_SW_GLORT_REG(0x2000 + (n_))
+#define FM10K_SW_GLORT_CAM_MATCH_ANY 0x00000000
+#define FM10K_SW_GLORT_CAM_MATCH_NONE FM10K_SW_MASK32(31, 0)
+#define FM10K_SW_GLORT_CAM_KEY_lsb 0
+#define FM10K_SW_GLORT_CAM_KEY_msb 15
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_lsb 16
+#define FM10K_SW_GLORT_CAM_KEY_INVERT_msb 31
+#define FM10K_SW_GLORT_RAM(n_) \
+ FM10K_SW_GLORT_REG(0x2200 + 0x2 * (n_))
+#define FM10K_SW_GLORT_RAM_STRICT_lsb64 0
+#define FM10K_SW_GLORT_RAM_STRICT_msb64 1
+#define FM10K_SW_GLORT_RAM_STRICT_FTYPE 0
+#define FM10K_SW_GLORT_RAM_STRICT_RSVD 1
+#define FM10K_SW_GLORT_RAM_STRICT_HASHED 2
+#define FM10K_SW_GLORT_RAM_STRICT_STRICT 3
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_lsb64 2
+#define FM10K_SW_GLORT_RAM_DEST_INDEX_msb64 13
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_lsb64 14
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_A_msb64 21
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_lsb64 22
+#define FM10K_SW_GLORT_RAM_RANGE_SUBIDX_B_msb64 29
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_lsb64 30
+#define FM10K_SW_GLORT_RAM_DEST_COUNT_msb64 33
+#define FM10K_SW_GLORT_RAM_HASH_ROTATION (1ULL << 34)
+
+
+#define FM10K_SW_PARSER_BASE FM10K_SW_REG_OFF(0xCF0000)
+#define FM10K_SW_PARSER_REG(wo_) \
+ (FM10K_SW_PARSER_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_PARSER_PORT_CFG_1(p_) \
+ FM10K_SW_PARSER_REG(0x0 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_1_FTAG (1ULL << 0)
+
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(v_) (1ULL << (1 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(v_) (1ULL << (5 + (v_)))
+#define FM10K_SW_PARSER_PORT_CFG_1_VLAN2_FIRST (1ULL << 9)
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_lsb 10
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID_msb 21
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_lsb 22
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_lsb 26
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VID2_msb 37
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_lsb 38
+#define FM10K_SW_PARSER_PORT_CFG_1_DEFAULT_VPRI2_msb 41
+#define FM10K_SW_PARSER_PORT_CFG_1_USE_DEFAULT_VLAN (1ULL << 42)
+#define FM10K_SW_PARSER_PORT_CFG_2(p_) \
+ FM10K_SW_PARSER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_lsb 0
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_1_msb 3
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_lsb 4
+#define FM10K_SW_PARSER_PORT_CFG_2_CUSTOM_TAG_2_msb 7
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_MPLS (1ULL << 8)
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_lsb 9
+#define FM10K_SW_PARSER_PORT_CFG_2_STORE_MPLS_msb 11
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3 (1ULL << 12)
+#define FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4 (1ULL << 13)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV4_OPTIONS (1ULL << 14)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_HOP_BY_HOP (1ULL << 15)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_ROUTING (1ULL << 16)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_FRAG (1ULL << 17)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_DEST (1ULL << 18)
+#define FM10K_SW_PARSER_PORT_CFG_2_FLAG_IPV6_AUTH (1ULL << 19)
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_lsb 20
+#define FM10K_SW_PARSER_PORT_CFG_2_DEFAULT_DSCP_msb 25
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_TAGGED (1ULL << 26)
+#define FM10K_SW_PARSER_PORT_CFG_2_DROP_UNTAGGED (1ULL << 27)
+#define FM10K_SW_PARSER_PORT_CFG_2_USE_DEFAULT_DSCP (1ULL << 28)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_VLAN (1ULL << 29)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_DSCP (1ULL << 30)
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_FROM_ISL (1ULL << 31)
+
+#define FM10K_SW_PARSER_PORT_CFG_2_SWITCH_PRI_PREFER_DSCP (1ULL << 32)
+
+#define FM10K_SW_PARSER_PORT_CFG_3(p_) \
+ FM10K_SW_PARSER_REG(0x100 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_ISL(p_) \
+ FM10K_SW_PARSER_REG(0x180 + (p_))
+
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_lsb 0
+#define FM10K_SW_PORT_CFG_ISL_SGLORT_msb 15
+#define FM10K_SW_PORT_CFG_ISL_USR_lsb 16
+#define FM10K_SW_PORT_CFG_ISL_USR_msb 23
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_lsb 24
+#define FM10K_SW_PORT_CFG_ISL_DEFAULT_PRI_msb 27
+
+#define FM10K_SW_PARSER_VLAN_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C0 + (n_))
+#define FM10K_SW_PARSER_CUSTOM_TAG(n_) \
+ FM10K_SW_PARSER_REG(0x1C4 + (n_))
+#define FM10K_SW_PARSER_MPLS_TAG \
+ FM10K_SW_PARSER_REG(0x1C8)
+#define FM10K_SW_PARSER_DI_CFG(n_) \
+ FM10K_SW_PARSER_REG(0x1D0 + 0x2 * (n_))
+#define FM10K_SW_RX_VPRI_MAP(p_) \
+ FM10K_SW_PARSER_REG(0x200 + 0x2 * (p_))
+#define FM10K_SW_DSCP_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x280 + (pri_))
+#define FM10K_SW_VPRI_PRI_MAP(pri_) \
+ FM10K_SW_PARSER_REG(0x2C0 + (pri_))
+
+#define FM10K_SW_HANDLER_BASE FM10K_SW_REG_OFF(0xD50000)
+#define FM10K_SW_HANDLER_REG(wo_) \
+ (FM10K_SW_HANDLER_BASE + FM10K_SW_REG_OFF(wo_))
+
+#define FM10K_SW_SYS_CFG_1 \
+ FM10K_SW_HANDLER_REG(0x0)
+#define FM10K_SW_SYS_CFG_1_DROP_PAUSE (1 << 0)
+#define FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS (1 << 1)
+#define FM10K_SW_SYS_CFG_1_ENABLE_TRAP_PLUS_LOG (1 << 2)
+#define FM10K_SW_SYS_CFG_1_DROP_INVALID_SMAC (1 << 3)
+#define FM10K_SW_SYS_CFG_1_DROP_MAC_CTRL_ETHERTYPE (1 << 4)
+
+#define FM10K_SW_CPU_MAC FM10K_SW_HANDLER_REG(0x2)
+#define FM10K_SW_SYS_CFG_ROUTER FM10K_SW_HANDLER_REG(0x4)
+#define FM10K_SW_L34_HASH_CFG FM10K_SW_HANDLER_REG(0x5)
+#define FM10K_SW_L34_HASH_CFG_SYMMETRIC (1 << 0)
+#define FM10K_SW_L34_HASH_CFG_USE_SIP (1 << 1)
+#define FM10K_SW_L34_HASH_CFG_USE_DIP (1 << 2)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT (1 << 3)
+#define FM10K_SW_L34_HASH_CFG_USE_TCP (1 << 4)
+#define FM10K_SW_L34_HASH_CFG_USE_UDP (1 << 5)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT1 (1 << 6)
+#define FM10K_SW_L34_HASH_CFG_USE_PROT2 (1 << 7)
+#define FM10K_SW_L34_HASH_CFG_USE_L4SRC (1 << 8)
+#define FM10K_SW_L34_HASH_CFG_USE_L4DST (1 << 9)
+
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_lsb 10
+#define FM10K_SW_L34_HASH_CFG_ECMP_ROTATION_msb 11
+#define FM10K_SW_L34_HASH_CFG_PROT1_lsb 16
+#define FM10K_SW_L34_HASH_CFG_PROT1_msb 23
+#define FM10K_SW_L34_HASH_CFG_PROT2_lsb 24
+#define FM10K_SW_L34_HASH_CFG_PROT2_msb 31
+
+#define FM10K_SW_L34_FLOW_HASH_CFG_1 FM10K_SW_HANDLER_REG(0x6)
+#define FM10K_SW_L34_FLOW_HASH_CFG_2 FM10K_SW_HANDLER_REG(0x7)
+#define FM10K_SW_L234_HASH_CFG FM10K_SW_HANDLER_REG(0x8)
+#define FM10K_SW_L234_HASH_CFG_USE_L2_IF_IP (1 << 0)
+#define FM10K_SW_L234_HASH_CFG_USE_L34 (1 << 1)
+#define FM10K_SW_L234_HASH_CFG_SYMMETRIC (1 << 2)
+#define FM10K_SW_L234_HASH_CFG_USE_DMAC (1 << 3)
+#define FM10K_SW_L234_HASH_CFG_USE_SMAC (1 << 4)
+#define FM10K_SW_L234_HASH_CFG_USE_TYPE (1 << 5)
+#define FM10K_SW_L234_HASH_CFG_USE_VPRI (1 << 6)
+#define FM10K_SW_L234_HASH_CFG_USE_VID (1 << 8)
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_lsb 8
+#define FM10K_SW_L234_HASH_CFG_ROTATION_A_msb 9
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_lsb 10
+#define FM10K_SW_L234_HASH_CFG_ROTATION_B_msb 11
+#define FM10K_SW_CPU_TRAP_MASK_FH FM10K_SW_HANDLER_REG(0xA)
+#define FM10K_SW_TRAP_GLORT FM10K_SW_HANDLER_REG(0xC)
+#define FM10K_SW_RX_MIRROR_CFG FM10K_SW_HANDLER_REG(0xD)
+#define FM10K_SW_LOG_MIRROR_PROFILE FM10K_SW_HANDLER_REG(0xE)
+
+#define FM10K_SW_FH_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_HANDLER_REG(0x40 + (n_))
+#define FM10K_SW_PORT_CFG_2(p_) \
+ FM10K_SW_HANDLER_REG(0x80 + 0x2 * (p_))
+#define FM10K_SW_PORT_CFG_3(p_) \
+ FM10K_SW_HANDLER_REG(0x100 + (p_))
+#define FM10K_SW_FH_LOOPBACK_SUPPRESS(p_) \
+ FM10K_SW_HANDLER_REG(0x140 + (p_))
+#define FM10K_SW_FH_HEAD_IP FM10K_SW_HANDLER_REG(0x180)
+#define FM10K_SW_FH_HEAD_IM FM10K_SW_HANDLER_REG(0x182)
+#define FM10K_SW_PARSER_EARLY_SRAM_CTRL FM10K_SW_HANDLER_REG(0x184)
+#define FM10K_SW_PARSER_LATE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x185)
+#define FM10K_SW_MAPPER_SRAM_CTRL FM10K_SW_HANDLER_REG(0x186)
+#define FM10K_SW_FFU_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1A0 + 0x4 * (n_))
+#define FM10K_SW_ARP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C0)
+#define FM10K_SW_VLAN_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C1)
+#define FM10K_SW_MA_TABLE_SRAM_CTRL(n_) \
+ FM10K_SW_HANDLER_REG(0x1C4 + 0x2 * (n_))
+#define FM10K_SW_FID_GLORT_LOOKUP_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1C8)
+#define FM10K_SW_GLORT_RAM_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CA)
+#define FM10K_SW_GLORT_TABLE_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CB)
+#define FM10K_SW_FH_HEAD_OUTPUT_FIFO_SRAM_CTRL FM10K_SW_HANDLER_REG(0x1CC)
+
+#define FM10K_SW_LAG_BASE FM10K_SW_REG_OFF(0xD90000)
+#define FM10K_SW_LAG_REG(wo_) \
+ (FM10K_SW_LAG_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_LAG_CFG(l_) FM10K_SW_LAG_REG(l_)
+#define FM10K_SW_LAG_CFG_LAG_SIZE_lsb 0
+#define FM10K_SW_LAG_CFG_LAG_SIZE_msb 3
+#define FM10K_SW_LAG_CFG_INDEX_lsb 4
+#define FM10K_SW_LAG_CFG_INDEX_msb 7
+#define FM10K_SW_LAG_CFG_HASH_ROTATION (1 << 8)
+#define FM10K_SW_LAG_CFG_IN_LAG (1 << 9)
+
+#define FM10K_SW_CANONICAL_GLORT_CAM(n_) \
+ FM10K_SW_LAG_REG((n_) + 0x40)
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_lsb 0
+#define FM10K_SW_CANONICAL_GLORT_CAM_LAG_GLORT_msb 15
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_lsb 16
+#define FM10K_SW_CANONICAL_GLORT_CAM_MASK_SIZE_msb 19
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_lsb 20
+#define FM10K_SW_CANONICAL_GLORT_CAM_PORT_FIELD_SIZE_msb 22
+
+#define FM10K_SW_RX_STATS_BASE \
+ FM10K_SW_REG_OFF(0xE00000)
+#define FM10K_SW_RX_STATS_REG(wo_) \
+ (FM10K_SW_RX_STATS_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_RX_STATS_BANK(b_, i_) \
+ FM10K_SW_RX_STATS_REG(0x1000 * (b_) + 0x4 * (i_))
+#define FM10K_SW_RX_STATS_BANK_1_INDEX 0
+#define FM10K_SW_RX_STATS_BANK_2_INDEX 1
+#define FM10K_SW_RX_STATS_BANK_3_INDEX 2
+#define FM10K_SW_RX_STATS_BANK_4_INDEX 3
+#define FM10K_SW_RX_STATS_BANK_5_INDEX 4
+#define FM10K_SW_RX_STATS_BANK_6_INDEX 5
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_UCAST 0
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_MCAST 1
+#define FM10K_SW_RX_STATS_BANK_1_NON_IP_L2_BCAST 2
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_UCAST 3
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_MCAST 4
+#define FM10K_SW_RX_STATS_BANK_1_IPV4_L2_BCAST 5
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_UCAST 6
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_MCAST 7
+#define FM10K_SW_RX_STATS_BANK_1_IPV6_L2_BCAST 8
+#define FM10K_SW_RX_STATS_BANK_1_IEEE802_3_PAUSE 9
+#define FM10K_SW_RX_STATS_BANK_1_CLASS_BASED_PAUSE 10
+#define FM10K_SW_RX_STATS_BANK_1_FRAMING_ERR 11
+#define FM10K_SW_RX_STATS_BANK_1_FCS_ERR 12
+#define FM10K_SW_RX_STATS_BANK_2_LEN_LT_64 0
+#define FM10K_SW_RX_STATS_BANK_2_LEN_EQ_64 1
+#define FM10K_SW_RX_STATS_BANK_2_LEN_65_127 2
+#define FM10K_SW_RX_STATS_BANK_2_LEN_128_255 3
+#define FM10K_SW_RX_STATS_BANK_2_LEN_256_511 4
+#define FM10K_SW_RX_STATS_BANK_2_LEN_512_1023 5
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1024_1522 6
+#define FM10K_SW_RX_STATS_BANK_2_LEN_1523_2047 7
+#define FM10K_SW_RX_STATS_BANK_2_LEN_2048_4095 8
+#define FM10K_SW_RX_STATS_BANK_2_LEN_4096_8191 9
+#define FM10K_SW_RX_STATS_BANK_2_LEN_8192_10239 10
+#define FM10K_SW_RX_STATS_BANK_2_LEN_GE_10240 11
+#define FM10K_SW_RX_STATS_BANK_3_PRI(p_) (p_)
+#define FM10K_SW_RX_STATS_BANK_4_FID_FORWARDED 0
+#define FM10K_SW_RX_STATS_BANK_4_FLOOD_FORWARDED 1
+#define FM10K_SW_RX_STATS_BANK_4_SPECIALLY_HANDLED 2
+#define FM10K_SW_RX_STATS_BANK_4_PARSER_ERROR_DROP 3
+#define FM10K_SW_RX_STATS_BANK_4_ECC_ERROR_DROP 4
+#define FM10K_SW_RX_STATS_BANK_4_TRAPPED 5
+#define FM10K_SW_RX_STATS_BANK_4_PAUSE_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_4_STP_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_4_SECURITY_VIOLATIONS 8
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_TAG_DROPS 9
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_INGRESS_DROPS 10
+#define FM10K_SW_RX_STATS_BANK_4_VLAN_EGRESS_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_4_GLORT_MISS_DROPS 12
+#define FM10K_SW_RX_STATS_BANK_4_FFU_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_4_TRIGGER_DROPS 14
+#define FM10K_SW_RX_STATS_BANK_5_POLICER_DROPS 0
+#define FM10K_SW_RX_STATS_BANK_5_TTL_DROPS 1
+#define FM10K_SW_RX_STATS_BANK_5_CM_PRIV_DROPS 2
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP0_DROPS 3
+#define FM10K_SW_RX_STATS_BANK_5_CM_SMP1_DROPS 4
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG0_DROPS 5
+#define FM10K_SW_RX_STATS_BANK_5_CM_RX_HOG1_DROPS 6
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG0_DROPS 7
+#define FM10K_SW_RX_STATS_BANK_5_CM_TX_HOG1_DROPS 8
+#define FM10K_SW_RX_STATS_BANK_5_TRIGGER_REDIRECTS 10
+#define FM10K_SW_RX_STATS_BANK_5_FLOOD_CONTROL_DROPS 11
+#define FM10K_SW_RX_STATS_BANK_5_GLORT_FORWARDED 12
+#define FM10K_SW_RX_STATS_BANK_5_LOOPBACK_SUPP_DROPS 13
+#define FM10K_SW_RX_STATS_BANK_5_OTHER_DROPS 14
+#define FM10K_SW_RX_PORT_STAT(b_, s_, p_, f_) \
+ (FM10K_SW_RX_STATS_BANK(FM10K_SW_RX_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_RX_STATS_##b_##_##s_) + \
+ ((f_) ? 0 : FM10K_SW_REG_OFF(2)))
+
+#define FM10K_SW_RX_STATS_CFG(p_) \
+ FM10K_SW_RX_STATS_REG(0x10000 + (p_))
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_lsb 0
+#define FM10K_SW_RX_STATS_CFG_PER_FRAME_ADJUSTMENT_msb 7
+#define FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS 0x00003f00
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_1 (1 << 8)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_2 (1 << 9)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_3 (1 << 10)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_4 (1 << 11)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_5 (1 << 12)
+#define FM10K_SW_RX_STATS_CFG_ENABLE_BANK_6 (1 << 13)
+#define FM10K_SW_RX_STATS_CFG_SWITCH_PRI (1 << 14)
+
+#define FM10K_SW_HANDLER_TAIL_BASE \
+ FM10K_SW_REG_OFF(0xE30000)
+#define FM10K_SW_HANDLER_TAIL_REG(wo_) \
+ (FM10K_SW_HANDLER_TAIL_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_SAF_MATRIX(l_) \
+ FM10K_SW_HANDLER_TAIL_REG(0x2 * (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF(l_) (1ULL << (l_))
+#define FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS FM10K_SW_MASK64(47, 0)
+#define FM10K_SW_SAF_MATRIX_IGNORE_ERROR (1ULL << 50)
+
+#define FM10K_SW_FRAME_TIME_OUT FM10K_SW_HANDLER_TAIL_REG(0x80)
+#define FM10K_SW_SAF_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x81)
+#define FM10K_SW_EGRESS_PAUSE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x82)
+#define FM10K_SW_RX_STATS_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x84)
+#define FM10K_SW_POLICER_USAGE_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x88)
+#define FM10K_SW_TCN_SRAM_CTRL FM10K_SW_HANDLER_TAIL_REG(0x8C)
+#define FM10K_SW_FH_TAIL_IP FM10K_SW_HANDLER_TAIL_REG(0x8D)
+#define FM10K_SW_FH_TAIL_IM FM10K_SW_HANDLER_TAIL_REG(0x8E)
+#define FM10K_SW_TAIL_PERMIT_MGMT FM10K_SW_HANDLER_TAIL_REG(0x8F)
+#define FM10K_SW_TAIL_FORCE_IDLE FM10K_SW_HANDLER_TAIL_REG(0x90)
+
+
+#define FM10K_SW_CM_USAGE_BASE FM10K_SW_REG_OFF(0xE60000)
+#define FM10K_SW_CM_USAGE_REG(wo_) \
+ (FM10K_SW_CM_USAGE_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_CM_SWEEPER_SWITCH_PRI_TO_TC FM10K_SW_CM_USAGE_REG(0x0)
+#define FM10K_SW_CM_SWEEPER_TC_TO_SMP FM10K_SW_CM_USAGE_REG(0x2)
+#define FM10K_SW_CM_TX_TC_PRIVATE_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x200 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_TX_TC_HOG_WM(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x400 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_PAUSE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x600 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_PRIVATE_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x680 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_RX_SMP_HOG_WM(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x700 + 0x2 * (l_) + (s_))
+#define FM10K_SW_CM_PAUSE_RESEND_INTERVAL(l_) \
+ FM10K_SW_CM_USAGE_REG(0x780 + (l_))
+#define FM10K_SW_CM_PAUSE_BASE_FREQ \
+ FM10K_SW_CM_USAGE_REG(0x7C0)
+#define FM10K_SW_CM_PAUSE_CFG(l_) \
+ FM10K_SW_CM_USAGE_REG(0x800 + (l_))
+#define FM10K_SW_CM_SHARED_WM(p_) \
+ FM10K_SW_CM_USAGE_REG(0x840 + (p_))
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_WM(s_) \
+ FM10K_SW_CM_USAGE_REG(0x850 + (s_))
+#define FM10K_SW_CM_GLOBAL_WM FM10K_SW_CM_USAGE_REG(0x852)
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_lsb 0
+#define FM10K_SW_CM_GLOBAL_WM_WATERMARK_msb 14
+#define FM10K_SW_CM_GLOBAL_CFG FM10K_SW_CM_USAGE_REG(0x853)
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_lsb 0
+#define FM10K_SW_CM_GLOBAL_CFG_IFG_PENALTY_msb 7
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_ON (1 << 8)
+#define FM10K_SW_CM_GLOBAL_CFG_FORCE_PAUSE_OFF (1 << 9)
+#define FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN (1 << 10)
+
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN (1 << 11)
+#define FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN (1 << 12)
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_lsb 13
+#define FM10K_SW_CM_GLOBAL_CFG_NUM_SWEEPER_PORTS_msb 18
+
+#define FM10K_SW_CM_TC_PC_MAP(l_) FM10K_SW_CM_USAGE_REG(0x880 + (l_))
+#define FM10K_SW_CM_PC_SMP_MAP(l_) FM10K_SW_CM_USAGE_REG(0x8C0 + (l_))
+#define FM10K_SW_CM_SOFTDROP_WM(p_) FM10K_SW_CM_USAGE_REG(0x900 + (p_))
+
+#define FM10K_SW_CM_SHARED_SMP_PAUSE_CFG(s_) \
+ FM10K_SW_CM_USAGE_REG(0x910 + 0x2 * (s_))
+#define FM10K_SW_TX_RATE_LIM_CFG(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xA00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_TX_RATE_LIM_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0xC00 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_BSG_MAP(l_) \
+ FM10K_SW_CM_USAGE_REG(0xE00 + (l_))
+#define FM10K_SW_CM_TX_TC_USAGE(l_, c_) \
+ FM10K_SW_CM_USAGE_REG(0x1000 + 0x8 * (l_) + (c_))
+#define FM10K_SW_CM_RX_SMP_USAGE(l_, s_) \
+ FM10K_SW_CM_USAGE_REG(0x1200 + 0x2 * (l_) + (s_))
+#define FM10K_SW_MCAST_EPOCH_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1280 + (s_))
+#define FM10K_SW_CM_SHARED_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1282 + (s_))
+#define FM10K_SW_CM_SMP_USAGE(s_) \
+ FM10K_SW_CM_USAGE_REG(0x1284 + (s_))
+#define FM10K_SW_CM_GLOBAL_USAGE FM10K_SW_CM_USAGE_REG(0x1286)
+
+#define FM10K_SW_CM_PAUSE_GEN_STATE(l_) \
+ FM10K_SW_CM_USAGE_REG(0x12C0 + (l_))
+#define FM10K_SW_CM_PAUSE_RCV_TIMER FM10K_SW_CM_USAGE_REG(0x1300)
+#define FM10K_SW_CM_PAUSE_RCV_PORT_TIMER(l_) \
+ FM10K_SW_CM_USAGE_REG(0x1340 + (l_))
+
+/* FM10K_SW_CM_EGRESS_PAUSE_COUNT is also
+ * known as FM10K_SW_CM_PAUSE_RCV_STATE
+ */
+#define FM10K_SW_CM_EGRESS_PAUSE_COUNT(l_, dw_) \
+ FM10K_SW_CM_USAGE_REG(0x1400 + 0x4 * (l_) + 0x2 * (dw_))
+
+#define FM10K_SW_MOD_BASE FM10K_SW_REG_OFF(0xE80000)
+#define FM10K_SW_MOD_REG(wo_) \
+ (FM10K_SW_MOD_BASE + FM10K_SW_REG_OFF(wo_))
+#define FM10K_SW_MOD_MAX_MGMT_WAIT_CYCLE FM10K_SW_MOD_REG(0xF8)
+#define FM10K_SW_MOD_IP FM10K_SW_MOD_REG(0x106)
+#define FM10K_SW_MOD_IM FM10K_SW_MOD_REG(0x108)
+#define FM10K_SW_MOD_SRAM_BIST_OUT FM10K_SW_MOD_REG(0x10A)
+#define FM10K_SW_MOD_SRAM_ERROR_WRITE FM10K_SW_MOD_REG(0x10C)
+#define FM10K_SW_MOD_PAUSE_SMAC FM10K_SW_MOD_REG(0x10E)
+#define FM10K_SW_MOD_ROUTER_SMAC(n_) \
+ FM10K_SW_MOD_REG(0x120 + 0x2 * (n_))
+#define FM10K_SW_MOD_MCAST_VLAN_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x10000 + 0x2 * (n_))
+#define FM10K_SW_MOD_VLAN_TAG_VID1_MAP(v_) \
+ FM10K_SW_MOD_REG(0x20000 + 0x2 * (v_))
+#define FM10K_SW_MOD_VID2_MAP(v_) \
+ FM10K_SW_MOD_REG(0x22000 + 0x2 * (v_))
+#define FM10K_SW_MOD_MIRROR_PROFILE_TABLE(n_) \
+ FM10K_SW_MOD_REG(0x24000 + 0x2 * (n_))
+#define FM10K_SW_MOD_PER_PORT_CFG_1(p_) \
+ FM10K_SW_MOD_REG(0x24080 + 0x2 * (p_))
+#define FM10K_SW_MOD_PER_PORT_CFG_2(p_) \
+ FM10K_SW_MOD_REG(0x24100 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_lsb64 0
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIRROR_TRUNCATION_LEN_msb64 5
+
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP1_UPDATE (1ULL << 6)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_PCP2_UPDATE (1ULL << 7)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI1_UPDATE (1ULL << 8)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DEI2_UPDATE (1ULL << 9)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_lsb64 10
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN1_ETYPE_msb64 11
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_lsb64 12
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN2_ETYPE_msb64 13
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DMAC_ROUTING (1ULL << 14)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_SMAC_ROUTING (1ULL << 15)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_TTL_DECREMENT (1ULL << 16)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_ENABLE_DSCP_MODIFICATION (1ULL << 17)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_FTAG (1ULL << 18)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VID2_FIRST (1ULL << 19)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_lsb64 20
+#define FM10K_SW_MOD_PER_PORT_CFG_2_VLAN_TAGGING_msb64 22
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_lsb64 23
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_PRI_EN_VEC_msb64 30
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_TYPE (1ULL << 31)
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_lsb64 32
+#define FM10K_SW_MOD_PER_PORT_CFG_2_TX_PAUSE_VALUE_msb64 47
+#define FM10K_SW_MOD_PER_PORT_CFG_2_MIN_FRAME_SIZE (1ULL << 48)
+
+#define FM10K_SW_MOD_VPRI1_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24180 + 0x2 * (p_))
+#define FM10K_SW_MOD_VPRI2_MAP(p_) \
+ FM10K_SW_MOD_REG(0x24200 + 0x2 * (p_))
+#define FM10K_SW_MOD_VLAN_ETYPE(n_) \
+ FM10K_SW_MOD_REG(0x24280 + 0x2 * (n_))
+#define FM10K_SW_MOD_STATS_CFG(p_) \
+ FM10K_SW_MOD_REG(0x24300 + 0x2 * (p_))
+
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 (1 << 0)
+#define FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8 (1 << 1)
+#define FM10K_SW_MOD_STATS_BANK_FRAME(b_, i_) \
+ FM10K_SW_MOD_REG(0x25000 + 0x800 * (b_) + 0x2 * (i_))
+#define FM10K_SW_MOD_STATS_BANK_BYTE(b_, i_) \
+ FM10K_SW_MOD_REG(0x26000 + 0x800 * (b_) + 0x2 * (i_))
+
+#define FM10K_SW_MOD_STATS_BANK_7_INDEX 0
+#define FM10K_SW_MOD_STATS_BANK_8_INDEX 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_UCAST 0
+#define FM10K_SW_MOD_STATS_BANK_7_L2_MCAST 1
+#define FM10K_SW_MOD_STATS_BANK_7_L2_BCAST 2
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR 3
+#define FM10K_SW_MOD_STATS_BANK_7_TIMEOUT_DROP 4
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ERROR_DROP 5
+#define FM10K_SW_MOD_STATS_BANK_7_TX_ECC_DROP 6
+#define FM10K_SW_MOD_STATS_BANK_7_LOOPBACK_DROP 7
+#define FM10K_SW_MOD_STATS_BANK_7_TTL1_DROP 8
+#define FM10K_SW_MOD_STATS_BANK_7_PAUSE 9
+#define FM10K_SW_MOD_STATS_BANK_7_CB_PAUSE 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_LT_64 0
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_EQ_64 1
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_65_127 2
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_128_255 3
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_256_511 4
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_512_1023 5
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1024_1522 6
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_1523_2047 7
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_2048_4095 8
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_4096_8191 9
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_8192_10239 10
+#define FM10K_SW_MOD_STATS_BANK_8_LEN_GE_10240 11
+#define FM10K_SW_TX_PORT_STAT(b_, s_, p_, f_, p1_) \
+ ((f_) ? \
+ FM10K_SW_MOD_STATS_BANK_FRAME(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p_) + FM10K_SW_MOD_STATS_##b_##_##s_) : \
+ FM10K_SW_MOD_STATS_BANK_BYTE(FM10K_SW_MOD_STATS_##b_##_INDEX, \
+ 16 * (p1_) + FM10K_SW_MOD_STATS_##b_##_##s_))
+
+#define FM10K_SW_SCHED_BASE FM10K_SW_REG_OFF(0xF00000)
+#define FM10K_SW_SCHED_REG(wo_) \
+ (FM10K_SW_SCHED_BASE + FM10K_SW_REG_OFF(wo_))
+
+/*
+ * Schedule fields for both FM10K_SW_SCHED_RX_SCHEDULE and
+ * FM10K_SW_SCHED_TX_SCHEDULE
+ */
+#define FM10K_SW_SCHED_SCHEDULE_ENTRY(p_, l_, q_) \
+ (FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_PHYS_PORT, (p_)) | \
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SCHEDULE_LOG_PORT, (l_)) | \
+ ((q_) ? FM10K_SW_SCHED_SCHEDULE_QUAD : 0))
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_lsb 0
+#define FM10K_SW_SCHED_SCHEDULE_PHYS_PORT_msb 7
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_lsb 8
+#define FM10K_SW_SCHED_SCHEDULE_LOG_PORT_msb 13
+#define FM10K_SW_SCHED_SCHEDULE_QUAD (1 << 14)
+#define FM10K_SW_SCHED_SCHEDULE_COLOR (1 << 15)
+#define FM10K_SW_SCHED_SCHEDULE_IDLE (1 << 16)
+
+#define FM10K_SW_SCHED_RX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20000 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_TX_SCHEDULE(p_, n_) \
+ FM10K_SW_SCHED_REG(0x20400 + 0x200 * (p_) + (n_))
+#define FM10K_SW_SCHED_SCHEDULE_CTRL \
+ FM10K_SW_SCHED_REG(0x20800)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE (1 << 0)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_PAGE (1 << 1)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_lsb 2
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_RX_MAX_INDEX_msb 10
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE (1 << 11)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_PAGE (1 << 12)
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_lsb 13
+#define FM10K_SW_SCHED_SCHEDULE_CTRL_TX_MAX_INDEX_msb 21
+#define FM10K_SW_SCHED_CONFIG_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x20801)
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x20840 + (l_))
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_lsb 0
+#define FM10K_SW_SCHED_SSCHED_RX_PERPORT_NEXT_msb 15
+#define FM10K_SW_SCHED_SSCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30050)
+#define FM10K_SW_SCHED_ESCHED_CFG_1(l_) \
+ FM10K_SW_SCHED_REG(0x30080 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_2(l_) \
+ FM10K_SW_SCHED_REG(0x300C0 + (l_))
+#define FM10K_SW_SCHED_ESCHED_CFG_3(l_) \
+ FM10K_SW_SCHED_REG(0x30100 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_ESCHED \
+ FM10K_SW_SCHED_REG(0x30240)
+#define FM10K_SW_SCHED_ESCHED_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30242)
+#define FM10K_SW_SCHED_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x30244)
+#define FM10K_SW_SCHED_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_FREELIST_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30246)
+#define FM10K_SW_SCHED_MONITOR_DRR_Q_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x30400 + (q_))
+#define FM10K_SW_SCHED_MONITOR_DRR_CFG_PERPORT(l_) \
+ FM10K_SW_SCHED_REG(0x30600 + (l_))
+#define FM10K_SW_SCHED_MGMT_TIMER_MONITOR \
+ FM10K_SW_SCHED_REG(0x30A40)
+#define FM10K_SW_SCHED_MONITOR_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x30A41)
+#define FM10K_SW_SCHED_MCAST_LEN_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x34000 + (n_))
+#define FM10K_SW_SCHED_MCAST_DEST_TABLE(n_) \
+ FM10K_SW_SCHED_REG(0x38000 + 0x2 * (n_))
+#define FM10K_SW_SCHED_MCAST_LIMITED_SKEW_MULTICAST \
+ FM10K_SW_SCHED_REG(0x3A000)
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(n_) \
+ FM10K_SW_SCHED_REG(0x60400 + 0x2 * (n_))
+
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES 8
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_lsb 0
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE_msb 9
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_lsb 10
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE_msb 19
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_lsb 20
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_HEAD_IDX_msb 24
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_lsb 25
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_TAIL_IDX_msb 29
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_lsb 30
+#define FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_NEXT_PAGE_msb 39
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x60410)
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES 1024
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_RXQ_FREELIST_INIT_ADDRESS_msb 9
+
+#define FM10K_SW_SCHED_MGMT_TIMER_RXQ \
+ FM10K_SW_SCHED_REG(0x60412)
+#define FM10K_SW_SCHED_MGMT_TIMER_LG \
+ FM10K_SW_SCHED_REG(0x60413)
+#define FM10K_SW_SCHED_RXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x60414)
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60600 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL0_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60800 + (q_))
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_lsb 0
+#define FM10K_SW_SCHED_TXQ_TAIL1_PERQ_TAIL_msb 15
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ(q_) \
+ FM10K_SW_SCHED_REG(0x60A00 + (q_))
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES 384
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_lsb 0
+#define FM10K_SW_SCHED_TXQ_HEAD_PERQ_HEAD_msb 15
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT \
+ FM10K_SW_SCHED_REG(0x62000)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES (24 * 1024)
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_lsb 0
+#define FM10K_SW_SCHED_TXQ_FREELIST_INIT_ADDRESS_msb 15
+#define FM10K_SW_SCHED_TXQ_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62002)
+#define FM10K_SW_SCHED_FIFO_SRAM_CTRL \
+ FM10K_SW_SCHED_REG(0x62004)
+#define FM10K_SW_SCHED_IP \
+ FM10K_SW_SCHED_REG(0x62006)
+#define FM10K_SW_SCHED_IM \
+ FM10K_SW_SCHED_REG(0x62007)
+
+#endif /* _FM10K_REGS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.c b/drivers/net/fm10k/switch/fm10k_sbus.c
new file mode 100644
index 0000000..4ab30ff
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.c
@@ -0,0 +1,291 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+#define FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX 4
+#define FM10K_SW_SBUS_COMMAND_TIMEOUT_US 500000
+#define FM10K_SW_SBUS_RING_DIVIDER 0x01
+
+static int fm10k_sbus_init(struct fm10k_sbus *);
+static int fm10k_sbus_exec(struct fm10k_sbus *, struct fm10k_sbus_req *);
+static int fm10k_sbus_reset_all(struct fm10k_sbus *);
+static int fm10k_sbus_sbm_reset(struct fm10k_sbus *);
+
+struct fm10k_sbus *
+fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name,
+ unsigned int cfg_reg)
+{
+ struct fm10k_sbus *sb;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: attaching", name);
+
+ sb = (struct fm10k_sbus *)rte_zmalloc
+ ("fm10k_sbus", sizeof(struct fm10k_sbus), 0);
+ if (sb == NULL) {
+ FM10K_SW_INFO("sbus %s: failed to allocate context", name);
+ goto fail;
+ }
+
+ sb->sw = sw;
+ sb->name = name;
+ pthread_mutex_init(&sb->lock, NULL);
+
+ sb->cfg_reg = cfg_reg;
+ sb->cmd_reg = cfg_reg + FM10K_SW_REG_OFF(1);
+ sb->req_reg = cfg_reg + FM10K_SW_REG_OFF(2);
+ sb->resp_reg = cfg_reg + FM10K_SW_REG_OFF(3);
+
+ error = fm10k_sbus_init(sb);
+ if (error)
+ goto fail;
+
+ FM10K_SW_TRACE("sbus %s: attach successful", name);
+ return sb;
+fail:
+ if (sb)
+ fm10k_sbus_detach(sb);
+ return NULL;
+}
+
+void
+fm10k_sbus_detach(struct fm10k_sbus *sb)
+{
+ FM10K_SW_TRACE("sbus %s: detaching", sb->name);
+
+ rte_free(sb);
+}
+
+static int
+fm10k_sbus_init(struct fm10k_sbus *sb)
+{
+ uint32_t data;
+ int error;
+
+ error = fm10k_sbus_sbm_reset(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_reset_all(sb);
+ if (error)
+ goto done;
+
+ /* set clock to REFCLK/2 */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ FM10K_SW_SBUS_RING_DIVIDER);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to set ring divider", sb->name);
+ goto done;
+ }
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SBUS_CONTROLLER, 0x0a,
+ &data);
+ if (error) {
+ FM10K_SW_ERR("sbus %s: failed to read back ring divider",
+ sb->name);
+ goto done;
+ }
+ if (data != FM10K_SW_SBUS_RING_DIVIDER) {
+ FM10K_SW_ERR("sbus %s: ring divider "
+ "verify failed (expected %u, got %u)",
+ sb->name, FM10K_SW_SBUS_RING_DIVIDER, data);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return error;
+}
+
+static int
+fm10k_sbus_exec(struct fm10k_sbus *sb, struct fm10k_sbus_req *req)
+{
+ struct fm10k_switch *sw = sb->sw;
+ unsigned int total_usecs;
+ unsigned int drop_lock;
+ unsigned int delay_time;
+ int error = 0;
+ uint32_t data;
+ uint8_t expected_result;
+
+ FM10K_SW_SBUS_LOCK(sb);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (data & FM10K_SW_SBUS_COMMAND_BUSY) {
+ FM10K_SW_INFO("sbus %s: bus busy (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+
+ switch (req->op) {
+ case FM10K_SW_SBUS_OP_RESET:
+ FM10K_SW_TRACE("sbus %s: RESET dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_RESET;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_WRITE:
+ FM10K_SW_TRACE("sbus %s: WRITE dev=0x%02x reg=0x%02x data=0x%08x",
+ sb->name, req->dev, req->reg, req->data);
+ expected_result = FM10K_SW_SBUS_RESULT_WRITE;
+ fm10k_write_switch_reg(sw, sb->req_reg, req->data);
+ break;
+ case FM10K_SW_SBUS_OP_READ:
+ FM10K_SW_TRACE("sbus %s: READ dev=0x%02x reg=0x%02x",
+ sb->name, req->dev, req->reg);
+ expected_result = FM10K_SW_SBUS_RESULT_READ;
+ req->data = 0;
+ break;
+ default:
+ FM10K_SW_INFO("sbus %s: invalid opcode 0x%02x",
+ sb->name, req->op);
+ error = -1;
+ goto done;
+ }
+
+ /* Clear the execute bit */
+ fm10k_write_switch_reg(sw, sb->cmd_reg, 0);
+
+ data =
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_OP, req->op) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_ADDRESS, req->dev) |
+ FM10K_SW_MAKE_REG_FIELD(SBUS_COMMAND_REGISTER, req->reg) |
+ FM10K_SW_SBUS_COMMAND_EXECUTE;
+ fm10k_write_switch_reg(sw, sb->cmd_reg, data);
+ fm10k_write_flush(sw);
+
+ total_usecs = 0;
+ delay_time = 1;
+ drop_lock = 0;
+ do {
+ if (drop_lock)
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_udelay(delay_time);
+ if (drop_lock)
+ FM10K_SW_SWITCH_LOCK(sw);
+ total_usecs += delay_time;
+ if (total_usecs >= FM10K_SW_SBUS_COMMAND_WAIT_LOCK_US_MAX) {
+ drop_lock = 1;
+ delay_time <<= 1;
+ }
+
+ data = fm10k_read_switch_reg(sw, sb->cmd_reg);
+ if (!(data & FM10K_SW_SBUS_COMMAND_BUSY)) {
+ if (FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE)
+ != expected_result) {
+ FM10K_SW_INFO("sbus %s: expected "
+ "result code %u, got %u", sb->name,
+ expected_result,
+ FM10K_SW_REG_FIELD(data,
+ SBUS_COMMAND_RESULT_CODE));
+ error = -1;
+ goto done;
+ }
+ if (req->op == FM10K_SW_SBUS_OP_READ) {
+ req->data =
+ fm10k_read_switch_reg(sw, sb->resp_reg);
+ FM10K_SW_TRACE("sbus %s: READ data=0x%02x",
+ sb->name, req->data);
+ }
+ goto done;
+ }
+
+ } while (total_usecs < FM10K_SW_SBUS_COMMAND_TIMEOUT_US);
+
+ error = -1;
+ FM10K_SW_INFO("sbus %s: command timed out after %u us "
+ "(op=0x%02x dev=0x%02x reg=0x%02x data=0x%08x)", sb->name,
+ total_usecs, req->op, req->dev, req->reg, req->data);
+
+done:
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ FM10K_SW_SBUS_UNLOCK(sb);
+ return error;
+}
+
+static int
+fm10k_sbus_reset_all(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_BROADCAST;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: broadcast reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+static int
+fm10k_sbus_sbm_reset(struct fm10k_sbus *sb)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_RESET;
+ req.dev = FM10K_SW_SBUS_ADDR_SPICO;
+ req.reg = 0;
+ req.data = 0;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ FM10K_SW_TRACE("sbus %s: SBM reset %s (%d)", sb->name,
+ error ? "failed" : "succeeded", error);
+
+ return error;
+}
+
+int
+fm10k_sbus_read(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t *data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_READ;
+ req.dev = dev;
+ req.reg = reg;
+
+ error = fm10k_sbus_exec(sb, &req);
+ *data = error ? 0 : req.data;
+
+ return error;
+}
+
+int
+fm10k_sbus_write(struct fm10k_sbus *sb, uint8_t dev, uint8_t reg, uint32_t data)
+{
+ struct fm10k_sbus_req req;
+ int error;
+
+ req.op = FM10K_SW_SBUS_OP_WRITE;
+ req.dev = dev;
+ req.reg = reg;
+ req.data = data;
+
+ error = fm10k_sbus_exec(sb, &req);
+
+ return error;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sbus.h b/drivers/net/fm10k/switch/fm10k_sbus.h
new file mode 100644
index 0000000..e67967f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sbus.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SBUS_H_
+#define _FM10K_SW_SBUS_H_
+
+#include <pthread.h>
+
+
+struct fm10k_sbus {
+ struct fm10k_switch *sw;
+ const char *name;
+ pthread_mutex_t lock;
+ unsigned int cfg_reg;
+ unsigned int cmd_reg;
+ unsigned int req_reg;
+ unsigned int resp_reg;
+};
+
+#define FM10K_SW_SBUS_LOCK(sb_) pthread_mutex_lock(&((sb_)->lock))
+#define FM10K_SW_SBUS_UNLOCK(sb_) pthread_mutex_unlock(&((sb_)->lock))
+
+struct fm10k_sbus_req {
+ uint8_t op;
+ uint8_t dev;
+ uint8_t reg;
+ uint32_t data;
+};
+
+
+struct fm10k_sbus *fm10k_sbus_attach(struct fm10k_switch *sw,
+ const char *name, unsigned int cfg_reg);
+void fm10k_sbus_detach(struct fm10k_sbus *sb);
+int fm10k_sbus_read(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t *data);
+int fm10k_sbus_write(struct fm10k_sbus *sb,
+ uint8_t dev, uint8_t reg, uint32_t data);
+
+#endif /* _FM10K_SW_SBUS_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
new file mode 100644
index 0000000..2423dbf
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -0,0 +1,335 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SWITCH_H_
+#define _FM10K_SW_SWITCH_H_
+
+#include <rte_spinlock.h>
+#include <pthread.h>
+#include <sys/time.h>
+#include <semaphore.h>
+
+#include "../fm10k.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+
+
+/*
+ * EPL
+ */
+#define FM10K_SW_EXT_PORTS_MAX 4
+
+#define FM10K_SW_EPLS_MAX 9
+#define FM10K_SW_EPLS_SUPPORTED 2
+#define FM10K_SW_EPL_LANES 4
+#define FM10K_SW_EPLS_PEP_MAX 2
+
+/*
+ * PEP
+ */
+#define FM10K_SW_PEP_GROUP_MAX 10
+#define FM10K_SW_LOGICAL_PORTS_MAX 48
+
+#define FM10K_SW_PEPS_MAX 9
+#define FM10K_SW_PEPS_SUPPORTED 4
+#define FM10K_SW_PEP_PORTS_MAX 4
+
+/*
+ * GLORT
+ */
+#define FM10K_SW_GROUP_MAX 10
+#define FM10K_SW_VFS_MAX 64
+#define FM10K_SW_PFS_GLORT_START 0x1000
+#define FM10K_SW_VF_GLORT_START 0x2000
+#define FM10K_SW_MULTI_GLORT_START 0x3000
+#define FM10K_SW_DPORT_MASK(lport) (1ULL << (lport))
+
+
+/*
+ * CONFIG
+ */
+#define FM10K_SW_CONFIG_MAX 1000
+#define FM10K_SW_FFU_RULE_MAX 256
+#define FM10K_SW_FFU_RULE_ITEM_MAX 10
+
+/*
+ * FFU COUNT
+ */
+#define FM10K_SW_FFU_CNT_BANK 3
+#define FM10K_SW_POLICER_LAST 19
+#define FM10K_SW_FFU_CNT_START (FM10K_SW_POLICER_LAST + 1)
+#define FM10K_SW_FFU_CNT_MAX 100
+
+#define FM10K_SW_PEP_QUEUES_MAX 256
+#define FM10K_SW_VF_QUEUES_MAX (FM10K_SW_PEP_QUEUES_MAX - 1)
+
+#define FM10K_SW_FTAG_SIZE 8
+#define FM10K_SW_PACKET_SIZE_MIN 17
+#define FM10K_SW_PACKET_SIZE_MAX (15 * 1024)
+#define FM10K_SW_MTU_MAX \
+ (FM10K_SW_PACKET_SIZE_MAX - ETHER_HDR_LEN - ETHER_VLAN_ENCAP_LEN)
+#define FM10K_SW_SEG_SIZE_MAX (16 * 1024)
+#define FM10K_SW_TSO_SIZE_MAX (256 * 1024 - 1)
+
+#define FM10K_SW_MEM_POOL_SEG_SIZE 192
+#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
+#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+
+
+
+#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
+#define FM10K_SW_CARD(vname_, dname_) \
+ FM10K_SW_CARD_ID(FM10K_SW_VENDOR_ID_##vname_, \
+ FM10K_SW_DEV_ID_##dname_)
+
+/*
+ * All IDs that may appear in the vendor ID or subsystem vendor ID.
+ */
+#define FM10K_SW_VENDOR_ID_INTEL 0x8086
+#define FM10K_SW_VENDOR_ID_SILICOM 0x1374
+#define FM10K_SW_VENDOR_ID_SILICOM_RB 0x1B2E
+
+
+/*
+ * All IDs that may appear in the device ID or subsystem device ID.
+ */
+#define FM10K_SW_DEV_ID_FM10K 0x15a4
+
+/* Silicom cards */
+#define FM10K_SW_DEV_ID_PE310G4DBIR_T 0x01B0
+#define FM10K_SW_DEV_ID_PE310G4DBIR_SRD 0x01B1
+#define FM10K_SW_DEV_ID_PE310G4DBIR_LRD 0x01B2
+#define FM10K_SW_DEV_ID_PE310G4DBIR_ER 0x01B3
+#define FM10K_SW_DEV_ID_PE310G4DBIR_DA 0x01B4
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS41 0x01B8
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QS43 0x01B9
+#define FM10K_SW_DEV_ID_PE340G2DBIR_QL4 0x01BA
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4 0x01C0
+#define FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4_REV_2 0x01C4
+#define FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4 0x01C1
+#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
+#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
+/*
+ * SWITCH
+ */
+
+struct fm10k_device_info {
+ uint16_t subvendor;
+ uint16_t subdevice;
+ const char *desc;
+ uint8_t num_ext_ports;
+ uint8_t ext_port_speed;
+ uint8_t num_epls;
+ uint8_t num_peps;
+};
+
+
+struct fm10k_i2c;
+struct fm10k_sbus;
+struct fm10k_hw;
+struct fm10k_ext_ports;
+struct fm10k_dpdk_cfg;
+
+struct fm10k_sw_port_map {
+ uint32_t glort;
+ uint16_t logical_port; /* logical port number */
+ uint16_t physical_port; /* */
+};
+
+struct fm10k_switch {
+ uint32_t *hw_addr;
+ uint32_t *sw_addr;
+ struct fm10k_hw *master_hw;
+ struct fm10k_device_info *info;
+ pthread_mutex_t lock;
+ struct fm10k_i2c *i2c;
+ struct fm10k_sbus *epl_sbus;
+ struct fm10k_ext_ports *ext_ports;
+ sem_t intr_tq;
+ pthread_t intr_task;
+ pthread_t led_task;
+ pthread_t stats_task;
+ uint32_t detaching;
+ uint32_t glort_cam_ram_idx;
+ uint32_t glort_dest_table_idx;
+ uint32_t mcast_dest_table_idx;
+ uint32_t mcast_len_table_idx;
+ uint32_t mcast_vlan_table_idx;
+ uint32_t epl_serdes_code_version_build_id;
+ uint16_t pep_mask; /* mask of non-master peps */
+ uint8_t pepno;
+ uint8_t serdes_loopback;
+ uint8_t epla_no;
+ uint8_t eplb_no;
+ uint8_t mac_addr[6];
+ struct fm10k_dpdk_cfg *dpdk_cfg;
+ struct fm10k_sw_port_map *pep_map;
+ struct fm10k_sw_port_map *epl_map;
+ int inited;
+};
+
+#define FM10K_SW_SWITCH_LOCK(sw_) pthread_mutex_lock(&((sw_)->lock))
+#define FM10K_SW_SWITCH_UNLOCK(sw_) pthread_mutex_unlock(&((sw_)->lock))
+
+#define FM10K_SW_HOWMANY(x, y, y1) (((x) + (y) - 1) / (y1))
+
+
+static inline uint64_t
+fm10k_uptime_us(void)
+{
+ struct timeval tv;
+
+ gettimeofday(&tv, NULL);
+ return ((uint64_t)tv.tv_sec * 1000000 + tv.tv_usec);
+}
+
+
+static inline uint32_t
+fm10k_read_switch_reg(struct fm10k_switch *sw, uint32_t reg)
+{
+ return ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+}
+
+static inline uint64_t
+fm10k_read_switch_reg64(struct fm10k_switch *sw, uint32_t reg)
+{
+ uint64_t temp, result;
+
+ result = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg];
+ temp = ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1];
+ result |= temp << 32;
+
+ return result;
+}
+
+static inline void
+fm10k_read_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *results, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ results[i] =
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i];
+}
+
+
+static inline void
+fm10k_write_switch_reg(struct fm10k_switch *sw, uint32_t reg, uint32_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val;
+}
+
+static inline void
+fm10k_write_switch_reg64(struct fm10k_switch *sw, uint32_t reg, uint64_t val)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] = val & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] = val >> 32;
+}
+
+static inline void
+fm10k_write_switch_reg128(struct fm10k_switch *sw,
+ uint32_t reg, uint64_t val_hi, uint64_t val_lo)
+{
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg] =
+ val_lo & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 1] =
+ val_lo >> 32;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 2] =
+ val_hi & 0xffffffff;
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + 3] =
+ val_hi >> 32;
+}
+
+static inline void
+fm10k_write_switch_array(struct fm10k_switch *sw,
+ uint32_t reg, uint32_t *data, uint32_t count)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++)
+ ((volatile uint32_t *)sw->master_hw->sw_addr)[reg + i] =
+ data[i];
+}
+
+static inline uint32_t
+fm10k_write_flush(struct fm10k_switch *sw)
+{
+ return ((volatile uint32_t *)sw->master_hw->hw_addr)[FM10K_SW_CTRL];
+}
+
+static inline void
+fm10k_gpio_output_set(struct fm10k_switch *sw, int gpio_pin, int value)
+{
+ uint32_t data;
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_CFG);
+ data |= 1 << gpio_pin;
+ data &= ~(1 << (gpio_pin + 16));
+
+ /* set gpio output */
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_CFG, data);
+
+ /*
+ * Wait 1 msec (PCA spec specifies reset pulse width = 4 ns and
+ * and reset time = 100 ns)
+ */
+ usec_delay(1000);
+ /* set reset */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_GPIO_DATA);
+ if (value == 0)
+ data &= ~(1 << gpio_pin);
+ else
+ data |= 1 << gpio_pin;
+
+ fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
+}
+
+#define fm10k_udelay usec_delay
+typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
+
+unsigned int fm10k_switch_eplidx_to_eplno
+ (struct fm10k_switch *sw, unsigned int eplidx);
+void fm10k_switch_intr(struct fm10k_hw *hw);
+
+struct fm10k_switch *fm10k_switch_get(void);
+struct fm10k_device_info *fm10k_get_device_info(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func);
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue);
+int fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw,
+ struct fm10k_hw *hw_list[]);
+int fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw);
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+void *fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw);
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw);
+void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
+
+uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
+uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
+uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
+uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
+uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
+uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
+uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new);
+
+int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
+
+void fm10k_switch_flowset_switchto(const char *name);
+void fm10k_switch_show_port(void);
+void fm10k_switch_show_ffu(void);
+void fm10k_switch_show_bank(void);
+
+void fm10k_flow_list_init(void *flow_list);
+const struct rte_flow_ops *fm10k_flow_ops_get(void);
+
+#endif
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 2/5] net/fm10k: add epl serdes and port control functions
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
` (2 subsequent siblings)
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add epl serdes include loading spico,
controlling pcsl, dma, dfe, ical. Add spico code.
Add state machine for epl lane and port, it creates
a pthread to handle the state changing event.
Add external port management, which will use
state machine to handle the event from lane and port.
The lane state will change between DOWN, WAIT_PLL_LOCK,
WAIT_SIGNAL_OK, WAIT_DFE_ICAL, WAIT_DFE_PCAL, UP.
The port state will change between DOWN, WAIT_LANE_UP, UP.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 4 +
drivers/net/fm10k/switch/fm10k_ext_port.c | 839 ++++++++
drivers/net/fm10k/switch/fm10k_ext_port.h | 136 ++
drivers/net/fm10k/switch/fm10k_serdes.c | 1936 +++++++++++++++++
drivers/net/fm10k/switch/fm10k_serdes.h | 32 +
drivers/net/fm10k/switch/fm10k_sm.c | 190 ++
drivers/net/fm10k/switch/fm10k_sm.h | 80 +
drivers/net/fm10k/switch/fm10k_spico_code.c | 2966 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_spico_code.h | 21 +
9 files changed, 6204 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ext_port.h
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.c
create mode 100644 drivers/net/fm10k/switch/fm10k_serdes.h
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.c
create mode 100644 drivers/net/fm10k/switch/fm10k_sm.h
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.c
create mode 100644 drivers/net/fm10k/switch/fm10k_spico_code.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 49bb554..6f5ae60 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -81,6 +81,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c
ifeq ($(CONFIG_RTE_FM10K_SWITCH_MANAGEMENT),y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_i2c.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sbus.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.c b/drivers/net/fm10k/switch/fm10k_ext_port.c
new file mode 100644
index 0000000..2c91a93
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.c
@@ -0,0 +1,839 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_malloc.h>
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_i2c.h"
+#include "fm10k_regs.h"
+#include "fm10k_serdes.h"
+#include "fm10k_sm.h"
+#include "fm10k_switch.h"
+
+
+#define LANE_EVENT(e_) FM10K_SW_EXT_PORT_LANE_EVENT_##e_
+#define LANE_TIMER(t_) FM10K_SW_EXT_PORT_LANE_TIMER_##t_
+#define LANE_TIMEOUT(t_) FM10K_SW_EXT_PORT_LANE_##t_##_TIMEOUT_MS
+#define LANE_POLL(t_) FM10K_SW_EXT_PORT_LANE_##t_##_POLL_MS
+#define LANE_STATE(s_) FM10K_SW_EXT_PORT_LANE_STATE_##s_
+#define LANE_FLAG(f_) FM10K_SW_EXT_PORT_LANE_FLAG_##f_
+#define LANE_FLAGS(f_) FM10K_SW_EXT_PORT_LANE_FLAGS_##f_
+#define LANE_FLAG_SET(l_, f_) ((l_)->flags & LANE_FLAG(f_))
+#define LANE_FLAGS_SET(l_, f_, f1_) \
+ (((l_)->flags & LANE_FLAGS(f_)) == LANE_FLAGS(f1_))
+
+#define PORT_EVENT(e_) FM10K_SW_EXT_PORT_EVENT_##e_
+#define PORT_STATE(s_) FM10K_SW_EXT_PORT_STATE_##s_
+
+static void fm10k_ext_ports_epl_intr
+ (struct fm10k_ext_ports *ports, unsigned int eplno);
+static void fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip);
+static void fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_bring_down(struct fm10k_ext_port *port);
+static void fm10k_ext_port_lane_process_event
+ (struct fm10k_sm *sm, uint16_t event);
+static void fm10k_ext_port_lane_process_timer
+ (struct fm10k_sm *sm, uint16_t timer);
+static void fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane);
+static void fm10k_ext_port_phy_enable(struct fm10k_ext_port *port);
+static void fm10k_ext_port_phy_disable(struct fm10k_ext_port *port);
+
+struct fm10k_ext_ports *
+fm10k_ext_ports_attach(struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int is_quad, lane_speed;
+ unsigned int i, j;
+ unsigned int eplidx, laneno;
+
+ FM10K_SW_TRACE("attaching external ports\n");
+
+ ports = (struct fm10k_ext_ports *)rte_zmalloc
+ ("fm10k_ext_ports", sizeof(struct fm10k_ext_ports), 0);
+ if (ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports\n");
+ goto fail;
+ }
+
+ ports->ports = (struct fm10k_ext_port *)rte_zmalloc
+ ("fm10k_ext_port_list",
+ sizeof(struct fm10k_ext_port) * cfg->num_ext_ports, 0);
+ if (ports->ports == NULL) {
+ FM10K_SW_ERR("failed to allocate external ports list\n");
+ goto fail;
+ }
+ ports->sw = sw;
+ ports->num_ports = cfg->num_ext_ports;
+ ports->ports_per_epl = cfg->num_ext_ports / cfg->num_epls;
+ ports->epl_mask = 0;
+
+ switch (cfg->ext_port_speed) {
+ default:
+ case 10:
+ is_quad = 0;
+ lane_speed = 10;
+ break;
+ case 25:
+ is_quad = 0;
+ lane_speed = 25;
+ break;
+ case 40:
+ is_quad = 1;
+ lane_speed = 10;
+ break;
+ case 100:
+ is_quad = 1;
+ lane_speed = 25;
+ break;
+ }
+
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ port->ports = ports;
+ fm10k_ext_port_eplidx_lane(ports, i, &eplidx, &laneno);
+ port->portno = i;
+ port->eplno = eplidx ? sw->eplb_no : sw->epla_no;
+ snprintf(port->name, sizeof(port->name), "EPL[%u][%u]",
+ port->eplno, port->first_lane);
+ port->first_lane = laneno;
+ port->is_quad = is_quad;
+ port->num_lanes = port->is_quad ? 4 : 1;
+ port->lane_speed = lane_speed;
+ port->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_NUM_EVENTS, 0,
+ fm10k_ext_port_process_event, NULL,
+ (void *)port);
+ if (port->sm == NULL)
+ goto fail;
+ port->sm->portno = port->portno;
+ port->sm->laneno = 0;
+ port->an_im = FM10K_SW_AN_IM_ALL;
+ port->link_im = FM10K_SW_LINK_IM_ALL;
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ lane->port = port;
+ lane->abs_laneno = port->eplno * FM10K_SW_EPL_LANES + j;
+ lane->rel_laneno = j;
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+ lane->sm = fm10k_sm_attach
+ (FM10K_SW_EXT_PORT_LANE_NUM_EVENTS,
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS,
+ fm10k_ext_port_lane_process_event,
+ fm10k_ext_port_lane_process_timer,
+ (void *)lane);
+ if (lane->sm == NULL)
+ goto fail;
+ lane->sm->portno = lane->port->portno;
+ lane->sm->laneno = lane->rel_laneno;
+ }
+
+ ports->epl_mask |= (1 << port->eplno);
+ }
+
+ return (ports);
+fail:
+ if (ports)
+ fm10k_ext_ports_detach(ports);
+ return NULL;
+}
+
+void
+fm10k_ext_ports_detach(struct fm10k_ext_ports *ports)
+{
+ struct fm10k_ext_port *port;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i, j;
+
+ FM10K_SW_TRACE("detaching external ports\n");
+
+ if (ports->ports) {
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+
+ if (port->sm)
+ fm10k_sm_detach(port->sm);
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ lane = &port->lanes[j];
+ if (lane->sm)
+ fm10k_sm_detach(lane->sm);
+ }
+
+ fm10k_ext_port_phy_disable(port);
+ }
+ rte_free(ports->ports);
+ }
+
+ rte_free(ports);
+}
+
+unsigned int
+fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports, uint64_t gid)
+{
+ unsigned int i;
+ uint16_t epl_mask;
+
+ /*
+ * Process EPL interrupts for all EPLs that are in use and have
+ * their bit set in global interrupt detect.
+ */
+ epl_mask = ports->epl_mask &
+ FM10K_SW_REG_FIELD64(gid, GLOBAL_INTERRUPT_DETECT_EPL);
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++)
+ if (epl_mask & (1 << i))
+ fm10k_ext_ports_epl_intr(ports, i);
+
+ return ports->epl_mask;
+}
+
+static void
+fm10k_ext_ports_epl_intr(struct fm10k_ext_ports *ports, unsigned int eplno)
+{
+ struct fm10k_switch *sw = ports->sw;
+ struct fm10k_ext_port *port;
+ uint64_t gid;
+ unsigned int i, j;
+ uint32_t epl_ip;
+
+ /*
+ * The global interrupt detect architecture makes it somewhat
+ * challenging to ensure events are not lost and spurious interrupts
+ * are not generated.
+ *
+ * The EPLs have an internal interrupt hierarchy whose reporting is
+ * rooted at EPL_IP. Whenever an EPL transitions from having no
+ * unmasked interrupts pending to at least one unmasked interrupt
+ * pending or from having at least one unmasked interrupt pending to
+ * no unmasked interrupts pending, it sends a vector to the
+ * interrupt controller over a serial bus indicating the
+ * at-least-one/none pending state, and the controller updates the
+ * bit for that EPL in the global interrupt detect register
+ * accordingly.
+ *
+ * Because of the update-vector-in-flight potential, there is a race
+ * between re-enabling the EPL PCIe interrupt mask after servicing
+ * the EPL interrupts and the all-clear vector from the EPL reaching
+ * the interrupt controller - it's possible for the software to
+ * clear all the EPL-internal interrupt sources during servicing and
+ * then re-enable the EPL interrupt in the PCIe mask before the
+ * all-clear vector reaches the interrupt controller and clears the
+ * EPL pending bit in the global interrupt detect register. This
+ * would result in a spurious EPL interrupt, potentially for every
+ * necessary EPL interrupt.
+ *
+ * The approach taken here to avoid spurious interrupts is to first
+ * mask off all EPL-internal sources, then poll the EPL bits in the
+ * global interrupt detect register until they clear (indicating the
+ * update vector from the EPL has reached the interrupt controller).
+ * Then software interrupt processing proceeds, and the EPL-internal
+ * masks are restored to their desired state at the end. This
+ * ensures that when the EPL PCIe interrupt mask is re-enabled, the
+ * EPL pending bits in the global interrupt detect register will
+ * only be set if there are new, unprocessed EPL events of interest.
+ *
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ /* Grab epl_ip before it is cleared by the masks below */
+ epl_ip = fm10k_read_switch_reg(sw, FM10K_SW_EPL_IP(eplno));
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_AN_IM(eplno, port->first_lane),
+ FM10K_SW_AN_IM_ALL);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IM(eplno, port->first_lane),
+ FM10K_SW_LINK_IM_ALL);
+
+ for (j = port->first_lane;
+ j < port->first_lane + port->num_lanes; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(eplno, j),
+ FM10K_SW_SERDES_IM_ALL);
+ }
+ }
+
+ /*
+ * Wait for the all-clear vector to propagate over the internal
+ * interrupt serial bus to the global interrupt detect.
+ */
+ do {
+ gid = fm10k_read_switch_reg64
+ (sw, FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ } while (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_EPL(eplno));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ /*
+ * Proceed with interrupt processing per-external-port.
+ */
+ for (i = 0; i < ports->num_ports; i++) {
+ port = &ports->ports[i];
+ if (port->eplno != eplno)
+ continue;
+ fm10k_ext_port_intr(port, epl_ip);
+ }
+}
+
+static void
+fm10k_ext_port_intr(struct fm10k_ext_port *port, uint32_t epl_ip)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ uint32_t serdes_ip, data = 0;
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (epl_ip & FM10K_SW_EPL_IP_SERDES_INTERRUPT(i)) {
+ FM10K_SW_SWITCH_LOCK(sw);
+ serdes_ip = fm10k_read_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i));
+ serdes_ip &= ~lane->im;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, i), serdes_ip);
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK)
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_STATUS(port->eplno,
+ i));
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_SIGNAL_OK) {
+ fm10k_sm_send_event(lane->sm,
+ (data &
+ FM10K_SW_LANE_SERDES_STATUS_RX_SIGNAL_OK) ?
+ LANE_EVENT(SIGNAL_OK) : LANE_EVENT(SIGNAL_NOK));
+ }
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_RX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RX_PLL_LOCK));
+
+ if (serdes_ip & FM10K_SW_SERDES_IP_TX_RDY)
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(TX_PLL_LOCK));
+ } else {
+ fm10k_sm_send_event(lane->sm,
+ LANE_EVENT(RESTORE_IM));
+ }
+ }
+}
+
+void
+fm10k_ext_port_up(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_UP));
+}
+
+void
+fm10k_ext_port_down(struct fm10k_ext_port *port)
+{
+ fm10k_sm_send_event(port->sm, PORT_EVENT(BRING_DOWN));
+}
+
+unsigned int
+fm10k_ext_port_isup(struct fm10k_ext_port *port)
+{
+ /* no locked needed as we are just inspecting
+ * the current value of an int
+ */
+ unsigned int port_state = port->sm->state;
+
+ return (port_state == FM10K_SW_EXT_PORT_STATE_UP);
+}
+
+static void
+fm10k_ext_port_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port *port = sm->ctx;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+ unsigned int lanes_up;
+
+ switch (sm->state) {
+ case PORT_STATE(DOWN):
+ if (event == PORT_EVENT(BRING_UP)) {
+ fm10k_ext_port_phy_enable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event
+ (lane->sm, LANE_EVENT(BRING_UP));
+ }
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ }
+ break;
+
+ case PORT_STATE(WAITING_FOR_LANES):
+ if (event == PORT_EVENT(LANE_UP)) {
+ lanes_up = 0;
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ if (lane->sm->state != LANE_STATE(UP))
+ break;
+ lanes_up++;
+ }
+ if (lanes_up == port->num_lanes) {
+ sm->state = PORT_STATE(UP);
+ FM10K_SW_INFO
+ ("Port %d is LINK UP", port->portno);
+ }
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ }
+ break;
+
+ case PORT_STATE(UP):
+ if (event == PORT_EVENT(LANE_DOWN)) {
+ sm->state = PORT_STATE(WAITING_FOR_LANES);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ } else if (event == PORT_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_bring_down(port);
+ FM10K_SW_INFO("Port %d is LINK DOWN", port->portno);
+ }
+ }
+}
+
+static void
+fm10k_ext_port_bring_down(struct fm10k_ext_port *port)
+{
+ struct fm10k_sm *sm = port->sm;
+ struct fm10k_ext_port_lane *lane;
+ unsigned int i;
+
+ sm->state = PORT_STATE(DOWN);
+
+ fm10k_ext_port_phy_disable(port);
+
+ for (i = port->first_lane;
+ i < port->first_lane + port->num_lanes; i++) {
+ lane = &port->lanes[i];
+ fm10k_sm_send_event(lane->sm, LANE_EVENT(BRING_DOWN));
+ }
+}
+
+static void
+fm10k_ext_port_lane_process_event(struct fm10k_sm *sm, uint16_t event)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ struct fm10k_ext_port *port = lane->port;
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ int error;
+
+ switch (sm->state) {
+ case LANE_STATE(DOWN):
+ if (event == LANE_EVENT(BRING_UP)) {
+ lane->flags &= ~LANE_FLAGS(PLLS_LOCKED);
+
+ /*
+ * Clear all pending interrupts.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IP(port->eplno, lane->rel_laneno),
+ FM10K_SW_SERDES_IP_ALL);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->im &= ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+ }
+ break;
+
+ case LANE_STATE(WAIT_PLL_LOCK):
+ if (event == LANE_EVENT(RX_PLL_LOCK) ||
+ event == LANE_EVENT(TX_PLL_LOCK)) {
+ if (event == LANE_EVENT(RX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(RX_LOCKED);
+ if (event == LANE_EVENT(TX_PLL_LOCK))
+ lane->flags |= LANE_FLAG(TX_LOCKED);
+ if (LANE_FLAGS_SET(lane, PLLS_LOCKED, PLLS_LOCKED)) {
+ error = fm10k_epl_serdes_continue_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_SIGNAL_OK);
+ /* unmask RX signal OK interrupt */
+ lane->im |=
+ (FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ lane->im &=
+ ~FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ }
+ }
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_SIGNAL_OK):
+ if (event == LANE_EVENT(SIGNAL_OK)) {
+ sm->state = LANE_STATE(WAIT_DFE_ICAL);
+
+ fm10k_epl_serdes_configure_for_dfe_tuning(lane);
+ fm10k_epl_serdes_start_dfe_ical(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ sm->state = LANE_STATE(WAIT_DFE_PCAL);
+
+ fm10k_epl_serdes_start_dfe_pcal(lane);
+
+ lane->dfe_poll_time_ms = 0;
+ fm10k_sm_timer_start(sm, LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (event == LANE_EVENT(DFE_TUNING_COMPLETE)) {
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+ sm->state = LANE_STATE(UP);
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_UP));
+ } else if ((event == LANE_EVENT(DFE_TUNING_FAILED)) ||
+ (event == LANE_EVENT(DFE_TUNING_TIMED_OUT)) ||
+ (event == LANE_EVENT(SIGNAL_NOK))) {
+ fm10k_ext_port_restart_lane(lane);
+ } else if (event == LANE_EVENT(BRING_DOWN)) {
+ fm10k_ext_port_disable_lane(lane);
+ }
+ break;
+
+ case LANE_STATE(UP):
+ if (event == LANE_EVENT(SIGNAL_NOK))
+ fm10k_ext_port_restart_lane(lane);
+ else if (event == LANE_EVENT(BRING_DOWN))
+ fm10k_ext_port_disable_lane(lane);
+ break;
+ }
+
+ /* Always restore the interrupt mask */
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SERDES_IM(port->eplno,
+ lane->rel_laneno),
+ lane->im);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_ext_port_lane_process_timer(struct fm10k_sm *sm, uint16_t timer)
+{
+ struct fm10k_ext_port_lane *lane = sm->ctx;
+ uint32_t status;
+
+ switch (sm->state) {
+ case LANE_STATE(WAIT_DFE_ICAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_ICAL);
+ fm10k_epl_serdes_dfe_ical_status(lane, &status);
+
+ if (status == FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_ICAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_ICAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else if (status == FM10K_SW_SERDES_DFE_ICAL_CONVERGED)
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_FAILED));
+ }
+ break;
+
+ case LANE_STATE(WAIT_DFE_PCAL):
+ if (timer == LANE_TIMER(DFE_TUNING)) {
+ lane->dfe_poll_time_ms += LANE_POLL(DFE_PCAL);
+
+ fm10k_epl_serdes_dfe_pcal_status(lane, &status);
+ if (status == FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ if (lane->dfe_poll_time_ms <
+ LANE_TIMEOUT(DFE_PCAL))
+ fm10k_sm_timer_start(sm,
+ LANE_TIMER(DFE_TUNING),
+ LANE_POLL(DFE_PCAL));
+ else
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_TIMED_OUT));
+ } else {
+ fm10k_sm_send_event(sm,
+ LANE_EVENT(DFE_TUNING_COMPLETE));
+ }
+ }
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_restart_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+ struct fm10k_ext_port *port = lane->port;
+ int error;
+
+ error = fm10k_epl_serdes_start_bringup(lane);
+ if (error == 0) {
+ sm->state = LANE_STATE(WAIT_PLL_LOCK);
+
+ /*
+ * Enable PLL lock interrupts.
+ */
+ lane->flags &= ~FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED;
+ lane->im |= FM10K_SW_SERDES_IM_RX_SIGNAL_OK;
+ lane->im = ~(FM10K_SW_SERDES_IM_RX_RDY |
+ FM10K_SW_SERDES_IM_TX_RDY);
+ }
+
+ fm10k_sm_send_event(port->sm, PORT_EVENT(LANE_DOWN));
+}
+
+static void
+fm10k_ext_port_disable_lane(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_sm *sm = lane->sm;
+
+ fm10k_sm_timer_cancel(sm, LANE_TIMER(DFE_TUNING));
+ fm10k_epl_serdes_disable(lane);
+
+ /*
+ * Disable all interrupts.
+ */
+ lane->im = FM10K_SW_SERDES_IM_ALL;
+
+ sm->state = LANE_STATE(DOWN);
+}
+
+void
+fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane)
+{
+ FM10K_SW_ASSERT(ext_port_idx < ports->num_ports,
+ ("ext_port_idx out of range"));
+
+ *eplidx = ext_port_idx / ports->ports_per_epl;
+ *lane = ext_port_idx % ports->ports_per_epl;
+}
+
+
+static void
+fm10k_ext_port_phy_enable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the
+ * PCA9505 that the QSFP control lines are connected
+ * to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get
+ * at the PCA9505 that the QSFP control lines
+ * are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x08);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x08);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to low to enable high
+ * power mode and set ResetL to high to take it out
+ * of reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x10);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x10);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to enable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+static void
+fm10k_ext_port_phy_disable(struct fm10k_ext_port *port)
+{
+ struct fm10k_switch *sw = port->ports->sw;
+ struct fm10k_device_info *cfg = sw->info;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Set up the first PCA9545 mux so we can get at the PCA9505
+ * that the QSFP control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* Configure the I/O pins on the PCA9505 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x18, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x08, 0x01);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x19, 0xf6);
+ fm10k_i2c_write16(sw->i2c, 0x20, 0x09, 0x01);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_LOCK(sw->i2c);
+ /* Configure the I/O pins on the PCA9538 that are
+ * connected to LpMode and ResetL on the QSFP as
+ * outputs, and set LpMode to high to disable high
+ * power mode and set ResetL to low to put it in
+ * reset.
+ */
+ if (port->portno == 0) {
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x60, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x60);
+ } else {
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x03, 0xcf);
+ fm10k_i2c_write16(sw->i2c, 0x61, 0x01, 0x20);
+ FM10K_SW_ERR("%s config PCA9538 %#x set LpMode",
+ __func__, 0x61);
+ }
+ FM10K_SW_I2C_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to disable phy for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)\n",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ext_port.h b/drivers/net/fm10k/switch/fm10k_ext_port.h
new file mode 100644
index 0000000..28c5e51
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ext_port.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_EXT_PORT_H_
+#define _FM10K_SW_EXT_PORT_H_
+
+#include "fm10k_regs.h"
+#include "fm10k_sm.h"
+#include "fm10k_sbus.h"
+#include "fm10k_switch.h"
+
+
+struct fm10k_sbus;
+struct fm10k_switch;
+
+#define FM10K_SW_TRACE_ENABLE 0
+
+enum fm10k_ext_port_lane_state {
+ FM10K_SW_EXT_PORT_LANE_STATE_DOWN,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_ICAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_WAIT_DFE_PCAL,
+ FM10K_SW_EXT_PORT_LANE_STATE_UP
+};
+
+enum fm10k_ext_port_lane_event {
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_TX_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_SIGNAL_NOK,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_COMPLETE,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_FAILED,
+ FM10K_SW_EXT_PORT_LANE_EVENT_DFE_TUNING_TIMED_OUT,
+ FM10K_SW_EXT_PORT_LANE_EVENT_RESTORE_IM,
+ FM10K_SW_EXT_PORT_LANE_EVENT_BRING_DOWN,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_EVENTS
+};
+
+enum fm10k_ext_port_lane_timer {
+ FM10K_SW_EXT_PORT_LANE_TIMER_PLL_LOCK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_SIGNAL_OK,
+ FM10K_SW_EXT_PORT_LANE_TIMER_DFE_TUNING,
+
+ FM10K_SW_EXT_PORT_LANE_NUM_TIMERS
+};
+
+#define FM10K_SW_EXT_PORT_LANE_PLL_LOCK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_SIGNAL_OK_TIMEOUT_MS 2000
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_POLL_MS 200
+#define FM10K_SW_EXT_PORT_LANE_DFE_ICAL_TIMEOUT_MS 3000
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_POLL_MS 100
+#define FM10K_SW_EXT_PORT_LANE_DFE_PCAL_TIMEOUT_MS 2000
+
+
+enum fm10k_ext_port_state {
+ FM10K_SW_EXT_PORT_STATE_DOWN,
+ FM10K_SW_EXT_PORT_STATE_WAITING_FOR_LANES,
+ FM10K_SW_EXT_PORT_STATE_UP
+};
+
+enum fm10k_ext_port_event {
+ FM10K_SW_EXT_PORT_EVENT_BRING_UP,
+ FM10K_SW_EXT_PORT_EVENT_BRING_DOWN,
+ FM10K_SW_EXT_PORT_EVENT_LANE_UP,
+ FM10K_SW_EXT_PORT_EVENT_LANE_DOWN,
+
+ FM10K_SW_EXT_PORT_NUM_EVENTS
+};
+
+struct fm10k_ext_port;
+
+struct fm10k_ext_port_lane {
+ struct fm10k_ext_port *port;
+ struct fm10k_sm *sm;
+ uint32_t im;
+ uint16_t dfe_poll_time_ms;
+ uint8_t abs_laneno;
+ uint8_t rel_laneno;
+ uint8_t flags;
+};
+
+#define FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED 0x01
+#define FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED 0x02
+
+#define FM10K_SW_EXT_PORT_LANE_FLAGS_PLLS_LOCKED \
+ (FM10K_SW_EXT_PORT_LANE_FLAG_RX_LOCKED | \
+ FM10K_SW_EXT_PORT_LANE_FLAG_TX_LOCKED)
+
+struct fm10k_ext_ports;
+
+struct fm10k_ext_port {
+ struct fm10k_ext_ports *ports;
+ struct fm10k_sm *sm;
+ struct fm10k_ext_port_lane lanes[FM10K_SW_EPL_LANES];
+ char name[10];
+ uint8_t portno;
+ uint8_t eplno;
+ uint8_t lport; /* filled in by switch */
+ uint8_t first_lane; /* range is [0..3] */
+ uint8_t num_lanes;
+ uint8_t is_quad;
+ uint8_t lane_speed;
+ uint8_t last_led_flags;
+ uint32_t an_im;
+ uint32_t link_im;
+};
+
+#define FM10K_SW_EXT_PORT_LED_FLAG_UP 0x01
+#define FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE 0x02
+
+struct fm10k_ext_ports {
+ struct fm10k_switch *sw;
+ struct fm10k_ext_port *ports;
+ uint16_t epl_mask;
+ uint8_t num_ports;
+ uint8_t ports_per_epl;
+};
+
+#define FM10K_SW_EXT_PORTS_EPL_USED(p_, n_) ((p_)->epl_mask & (1 << (n_)))
+
+struct fm10k_ext_ports *fm10k_ext_ports_attach(struct fm10k_switch *sw);
+void fm10k_ext_ports_detach(struct fm10k_ext_ports *ports);
+unsigned int fm10k_ext_ports_epl_intrs(struct fm10k_ext_ports *ports,
+ uint64_t gid);
+void fm10k_ext_port_up(struct fm10k_ext_port *port);
+void fm10k_ext_port_down(struct fm10k_ext_port *port);
+unsigned int fm10k_ext_port_isup(struct fm10k_ext_port *port);
+void fm10k_ext_port_eplidx_lane(struct fm10k_ext_ports *ports,
+ unsigned int ext_port_idx, unsigned int *eplidx,
+ unsigned int *lane);
+
+#endif /* _FM10K_SW_EXT_PORT_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.c b/drivers/net/fm10k/switch/fm10k_serdes.c
new file mode 100644
index 0000000..01b76a3
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.c
@@ -0,0 +1,1936 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "fm10k_debug.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_regs.h"
+#include "fm10k_sbus.h"
+#include "fm10k_serdes.h"
+#include "fm10k_spico_code.h"
+#include "fm10k_switch.h"
+
+
+#define FM10K_SW_SERDES_BIST_TIMEOUT_MS 5000
+#define FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS 5000
+#define FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS 3000
+#define FM10K_SW_SERDES_INT02_TIMEOUT_MS 500
+
+#define FM10K_SW_SERDES_CONFIG_DELAY_US 20
+#define FM10K_SW_SERDES_RESET_DELAY_US 22
+#define FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US 200
+
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_25G 0xa5
+#define FM10K_SW_SERDES_DIVIDER_ETHMODE_10G 0x42
+
+#define FM10K_SW_SERDES_WIDTH_40 3 /* 25G */
+#define FM10K_SW_SERDES_WIDTH_20 1 /* 10G */
+
+#define FM10K_SW_SERDES_EQ_SEL_PRECUR 0
+#define FM10K_SW_SERDES_EQ_SEL_ATTEN 1
+#define FM10K_SW_SERDES_EQ_SEL_POSTCUR 2
+
+#define FM10K_SW_SERDES_DFE_DEFAULT_HF 0x00
+#define FM10K_SW_SERDES_DFE_DEFAULT_LF 0x0c
+#define FM10K_SW_SERDES_DFE_DEFAULT_DC 0x38
+#define FM10K_SW_SERDES_DFE_DEFAULT_BW 0x0f
+
+#define FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES 10
+#define FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES 500
+
+#define FM10K_SERDES_SPICO_REG_0X00 0x00
+#define FM10K_SERDES_SPICO_REG_0X01 0x01
+#define FM10K_SERDES_SPICO_REG_0X02 0x02
+#define FM10K_SERDES_SPICO_REG_0X03 0x03
+#define FM10K_SERDES_SPICO_REG_0X04 0x04
+#define FM10K_SERDES_SPICO_REG_0X05 0x05
+#define FM10K_SERDES_SPICO_REG_0X07 0x07
+#define FM10K_SERDES_SPICO_REG_0X08 0x08
+#define FM10K_SERDES_SPICO_REG_0X0A 0x0A
+#define FM10K_SERDES_SPICO_REG_0X0B 0x0b
+#define FM10K_SERDES_SPICO_REG_0X14 0x14
+#define FM10K_SERDES_SPICO_REG_0X16 0x16
+
+
+/*
+ * These values correspond to IES FM_PORT_LINK_OPTIMIZATION_BALANCE
+ * settings.
+ */
+#define FM10K_SW_LINK_OPT_PARAM_A10G 0x4800
+#define FM10K_SW_LINK_OPT_PARAM_B10G 0xA000
+#define FM10K_SW_LINK_OPT_PARAM_A25G 0x10001
+#define FM10K_SW_LINK_OPT_PARAM_B25G 0x10001
+
+#define FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD 10
+
+
+static int fm10k_spico_ram_bist(struct fm10k_sbus *sb);
+static int fm10k_load_epl_spico_code(struct fm10k_switch *sw);
+static int fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id);
+static int fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb);
+static int fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id);
+static int fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words);
+static int fm10k_swap_image_check_crc(struct fm10k_sbus *sb,
+ unsigned int crc_code);
+static int fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words);
+static int fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id);
+static int fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id);
+static int fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result);
+static int fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider);
+static int fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static void fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode);
+static int fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes);
+static int fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold);
+static int fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq);
+static int fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms);
+static int fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data);
+static int fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data,
+ uint32_t mask, uint32_t *result);
+static int fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result);
+static int fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param);
+static void fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane);
+static int fm10k_epl_serdes_configure_near_loopback
+ (struct fm10k_ext_port_lane *lane);
+static int fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param);
+static int fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status);
+static int fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged);
+
+
+int
+fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: initializing EPL serdes", sb->name);
+
+ error = fm10k_spico_ram_bist(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_load_epl_spico_code(sw);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_spico_ram_bist(struct fm10k_sbus *sb)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: starting SPICO RAM BIST", sb->name);
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x03);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x05);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_BIST_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, &data);
+ if (error)
+ goto done;
+
+ if (data & 0x18) {
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ if ((data & 0x18) != 0x08) {
+ FM10K_SW_INFO("sbus %s: SPICO RAM "
+ "BIST failed with bad status (0x%08x)",
+ sb->name, data);
+ error = -1;
+ goto done;
+ }
+ goto done;
+ }
+ }
+ fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SPICO RAM BIST timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_load_epl_spico_code(struct fm10k_switch *sw)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ const uint16_t *sbm_code_image;
+ uint32_t sbm_code_size;
+ uint32_t sbm_code_version_build_id;
+ const uint16_t *serdes_code_image;
+ uint32_t serdes_code_size;
+ uint32_t serdes_code_version_build_id;
+ const uint16_t *swap_code_image;
+ uint32_t swap_code_size;
+ unsigned int swap_crc_code;
+ int error = 0;
+
+ FM10K_SW_TRACE("sbus %s: loading SPICO code", sb->name);
+
+ sbm_code_image = fm10000_sbus_master_code_prd;
+ sbm_code_size = fm10000_sbus_master_code_size_prd;
+ sbm_code_version_build_id =
+ fm10000_sbus_master_code_version_build_id_prd;
+
+ FM10K_SW_TRACE("sbus %s: sbm code version=%4.4x build=%4.4x",
+ sb->name, sbm_code_version_build_id >> 16,
+ sbm_code_version_build_id & 0xffff);
+
+ serdes_code_image = fm10000_serdes_spico_code_prd2;
+ serdes_code_size = fm10000_serdes_spico_code_size_prd2;
+ serdes_code_version_build_id =
+ fm10000_serdes_spico_code_version_build_id_prd2;
+ sw->epl_serdes_code_version_build_id = serdes_code_version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: serdes code version=%4.4x build=%4.4x",
+ sb->name, serdes_code_version_build_id >> 16,
+ serdes_code_version_build_id & 0xffff);
+
+ swap_code_image = fm10000_serdes_swap_code_prd2;
+ swap_code_size = fm10000_serdes_swap_code_size_prd2;
+
+ error = fm10k_sbm_spico_upload_image(sb, sbm_code_image,
+ sbm_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_check_crc_version_build_id(sb,
+ sbm_code_version_build_id);
+ if (error)
+ goto done;
+
+ if (swap_code_size > 0) {
+ if ((sbm_code_version_build_id & 0x00008000) == 0) {
+ error = fm10k_serdes_swap_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x1a;
+ } else {
+ error = fm10k_serdes_swap_alt_upload_image(sb,
+ swap_code_image, swap_code_size);
+ swap_crc_code = 0x04;
+ }
+ if (error)
+ goto done;
+
+ error = fm10k_swap_image_check_crc(sb, swap_crc_code);
+ if (error)
+ goto done;
+ }
+
+ error = fm10k_serdes_spico_upload_image(sb,
+ FM10K_SW_SBUS_ADDR_BROADCAST,
+ serdes_code_image, serdes_code_size);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_check_crc_version_build_id(sw,
+ serdes_code_version_build_id);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+
+static int
+fm10k_sbm_spico_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM SPICO image "
+ "(%u words)", sb->name, num_words);
+
+ data = (1 << 6) | (1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 7);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ data = 0x80000000 | ((uint32_t)image[i] << 16) | i;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM image upload time %u ms",
+ sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ data = (1 << 6);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X16, data);
+ if (error)
+ goto done;
+
+ data = (1 << 6) | (1 << 8);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_check_crc_version_build_id(struct fm10k_sbus *sb,
+ uint32_t expected_version_build_id)
+{
+ int error;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SBM SPICO code CRC "
+ "and version", sb->name);
+
+ error = fm10k_sbm_spico_do_crc(sb);
+ if (error)
+ goto done;
+
+ error = fm10k_sbm_get_version_build_id(sb, &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO code version "
+ "compare failed (expected 0x%08x, got 0x%08x)",
+ sb->name, expected_version_build_id, version_build_id);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM SPICO image "
+ "version=0x%4.4x build_id=0x%4.4x", sb->name,
+ version_build_id >> 16,
+ version_build_id & 0xffff);
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_do_crc(struct fm10k_sbus *sb)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM SPICO CRC "
+ "check", sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x02, 0, &crc);
+ if (error)
+ goto done;
+
+ crc = crc >> 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM SPICO CRC check failed "
+ "(CRC interrupt returned 0xffff)", sb->name);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM SPICO CRC check "
+ "passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM SPICO CRC "
+ "check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_get_version_build_id(struct fm10k_sbus *sb,
+ uint32_t *version_build_id)
+{
+ int error;
+ uint32_t result;
+
+ FM10K_SW_TRACE("sbus %s: getting SBM code version",
+ sb->name);
+
+ error = fm10k_sbm_spico_int(sb, 0x00, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id = result & 0xffff0000;
+ error = fm10k_sbm_spico_int(sb, 0x01, 0, &result);
+ if (error)
+ goto done;
+
+ *version_build_id |= (result >> 16) & 0xffff;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ error = fm10k_sbm_spico_int(sb, 0x1c, 0, &addr);
+ if (error)
+ goto done;
+
+ addr >>= 16;
+ if (addr == 0xffff) {
+ FM10K_SW_INFO("sbus %s: invalid build - cannot "
+ "upload SBM swap image", sb->name);
+ error = -1;
+ goto done;
+ }
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, addr);
+ if (error)
+ goto done;
+ error = fm10k_sbus_write(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X03, 0x80000000 | addr);
+ if (error)
+ goto done;
+
+ for (i = 0; i < num_words - 2; i += 3) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0xc0000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10) |
+ ((uint32_t)image[i + 2] << 20));
+ if (error)
+ goto done;
+ }
+
+ if (num_words - i == 2) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x80000000 | image[i] |
+ ((uint32_t)image[i + 1] << 10));
+ if (error)
+ goto done;
+ } else if (num_words - i == 1) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X14,
+ 0x40000000 | image[i]);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image upload time "
+ "%u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~(1 << 9);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_swap_alt_upload_image(struct fm10k_sbus *sb,
+ const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t addr;
+ uint32_t reg01, reg05;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: alt-uploading SBM swap image "
+ "(%u words)", sb->name, num_words);
+
+ reg05 = 1;
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, ®01);
+ if (error)
+ goto done;
+
+ reg01 |= (1 << 10) | (1 << 11);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ addr = 0x0400;
+ FM10K_SW_TRACE("sbus %s: SBM swap image initial load "
+ "address 0x%04x (%u)", sb->name, addr, addr);
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i++) {
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X04,
+ 0x8000 | ((uint32_t)image[i + 1] << 16) |
+ (addr + i));
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: SBM swap image alt-upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ reg01 &= ~((1 << 10) | (1 << 11));
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X01, reg01);
+ if (error)
+ goto done;
+
+ reg05 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X05, reg05);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+static int
+fm10k_swap_image_check_crc(struct fm10k_sbus *sb, unsigned int crc_code)
+{
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: performing SBM swap image CRC "
+ "check", sb->name);
+
+ error =
+ fm10k_sbm_spico_int(sb, crc_code, 0, &crc);
+ if (error)
+ goto done;
+
+ crc >>= 16;
+ if (crc == 0xffff) {
+ FM10K_SW_INFO("sbus %s: SBM swap image CRC check "
+ "failed (CRC interrupt 0x%02x returned 0xffff)",
+ sb->name, crc_code);
+ error = -1;
+ goto done;
+ } else if (crc == 0x0001)
+ FM10K_SW_TRACE("sbus %s: SBM swap image CRC "
+ "check passed ", sb->name);
+ else
+ FM10K_SW_INFO("sbus %s: unexpected SBM swap image "
+ "CRC check result 0x%04x", sb->name, crc);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_serdes_spico_upload_image(struct fm10k_sbus *sb,
+ uint8_t dev, const uint16_t *image, unsigned int num_words)
+{
+ uint64_t start_time_us, elapsed_time_us;
+ unsigned int i;
+ uint32_t data;
+ uint32_t reg07;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: uploading serdes SPICO image "
+ "to bus address 0x%02x (%u words)", sb->name, dev, num_words);
+
+ reg07 = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X03, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ data = (1 << 30);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, data);
+ if (error)
+ goto done;
+
+ data = (1 << 4) | (1 << 5);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, data);
+ if (error)
+ goto done;
+
+ start_time_us = fm10k_uptime_us();
+ for (i = 0; i < num_words; i += 3) {
+ data = 0xc0000000 | image[i];
+ if (i + 1 < num_words) {
+ data |= (uint32_t)image[i + 1] << 10;
+
+ if (i + 2 < num_words)
+ data |= (uint32_t)image[i + 2] << 20;
+ }
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X0A, data);
+ if (error)
+ goto done;
+ }
+ elapsed_time_us = fm10k_uptime_us() - start_time_us;
+ FM10K_SW_TRACE("sbus %s: serdes SPICO image upload "
+ "time %u ms", sb->name, (uint32_t)(elapsed_time_us / 1000));
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X00, 0);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev,
+ FM10K_SERDES_SPICO_REG_0X01, 0x20000000);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X0B, data);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ reg07 |= (1 << 1);
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X07, reg07);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, dev, FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_check_crc_version_build_id(struct fm10k_switch *sw,
+ uint32_t expected_version_build_id)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ unsigned int first = 0;
+ unsigned int last = FM10K_SW_EPLS_MAX * FM10K_SW_EPL_LANES - 1;
+ unsigned int i;
+ int error;
+ uint32_t data;
+ uint32_t version_build_id;
+
+ FM10K_SW_TRACE("sbus %s: checking SPICO code CRC and "
+ "version for serdes %u through %u", sb->name, first, last);
+
+ for (i = first; i <= last; i++) {
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(i), 0xff, &data);
+ if (error || data != 0x01) {
+ FM10K_SW_TRACE("sbus %s: skipping "
+ "serdes %u (error=%d, data=0x%08x)", sb->name, i,
+ error, data);
+ continue;
+ }
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, i);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_get_version_build_id(sw, i,
+ &version_build_id);
+ if (error)
+ goto done;
+
+ if (version_build_id != expected_version_build_id) {
+ FM10K_SW_INFO("sbus %s: SERDES SPICO code "
+ "version compare failed (expected 0x%08x, got "
+ "0x%08x)", sb->name, expected_version_build_id,
+ version_build_id);
+ error = -1;
+ goto done;
+ }
+ }
+ FM10K_SW_TRACE("sbus %s: serdes %u through %u are OK",
+ sb->name, first, last);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_do_crc(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t crc;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: performing SPICO "
+ "CRC check", sb->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3c, 0, &crc);
+ if (error)
+ goto done;
+
+ if (crc != 0) {
+ FM10K_SW_INFO("sbus %s: serdes %u: SPICO CRC check "
+ "failed (CRC interrupt returned 0x%08x)",
+ sb->name, serdes, crc);
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_reset(struct fm10k_switch *sw, unsigned int serdes)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ int error;
+ uint32_t data;
+ uint8_t addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: resetting SPICO",
+ sb->name, serdes);
+
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X00, 0x00);
+ if (error)
+ goto done;
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+
+ data = (1 << 0) | (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 18) | (1 << 19);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X0B, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 4);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ data = (1 << 1);
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X07, data);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_write(sb, addr, FM10K_SERDES_SPICO_REG_0X08, 0);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_version_build_id(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *version_build_id)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: getting code "
+ "version", sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x00, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id = data << 16;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x3f, 0, &data);
+ if (error)
+ goto done;
+
+ *version_build_id |= data & 0xffff;
+done:
+ return (error);
+}
+
+static int
+fm10k_sbm_spico_int(struct fm10k_sbus *sb,
+ uint8_t int_num, uint32_t param, uint32_t *result)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: SBM interrupt 0x%02x "
+ "(param=0x%08x)", sb->name, int_num, param);
+
+ /* int write */
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X02,
+ (param << 16) | int_num);
+ if (error)
+ goto done;
+
+ error = fm10k_sbus_read(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, &data);
+ if (error)
+ goto done;
+
+ data |= (1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ data &= ~(1 << 0);
+ error = fm10k_sbus_write(sb, FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X07, data);
+ if (error)
+ goto done;
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SBM_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb,
+ FM10K_SW_SBUS_ADDR_SPICO,
+ FM10K_SERDES_SPICO_REG_0X08, &data);
+ if (error)
+ goto done;
+
+ if ((data & 0x8000) || ((data & 0x3ff) == 0)) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ FM10K_SW_TRACE("sbus %s: SBM "
+ "interrupt 0x%02x (param=0x%08x, result=0x%08x)",
+ sb->name, int_num, param, data);
+ *result = data;
+ goto done;
+ }
+ }
+ error = -1;
+ FM10K_SW_INFO("sbus %s: SBM interrupt timed out after %u "
+ "ms", sb->name, (uint32_t)elapsed_time_ms);
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int(struct fm10k_switch *sw,
+ unsigned int serdes, uint16_t int_num,
+ uint32_t param, uint32_t *result)
+{
+ struct fm10k_sbus *sb = sw->epl_sbus;
+ uint64_t start_time_us, elapsed_time_ms;
+ unsigned int sbus_addr;
+ int error;
+ uint32_t data, intr;
+
+ sbus_addr = FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes);
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: (sbus addr 0x%02x) "
+ "interrupt 0x%04x (param=0x%08x)", sb->name, serdes, sbus_addr,
+ int_num, param);
+
+ /* int write */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ error = fm10k_sbus_write(sb, sbus_addr,
+ FM10K_SERDES_SPICO_REG_0X03,
+ ((uint32_t)int_num << 16) | param);
+ if (error)
+ goto done;
+ }
+
+ /* int read */
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error = fm10k_sbus_read(sb, sbus_addr, 0x04, &data);
+ if (error)
+ goto done;
+
+ if (data & ((1 << 16) | (1 << 17))) {
+ if (elapsed_time_ms > 5)
+ fm10k_udelay(5);
+ } else {
+ break;
+ }
+ }
+ if (elapsed_time_ms >= FM10K_SW_SERDES_INTERRUPT_TIMEOUT_MS) {
+ error = fm10k_sbus_read(sb, sbus_addr, 0x03, &intr);
+ FM10K_SW_INFO("sbus %s: serdes %u: interrupt timed out "
+ "(2) after %u ms (intr=0x%04x param=0x%04x error=%u "
+ "reg[4]=0x%08x)", sb->name, serdes,
+ (uint32_t)elapsed_time_ms, intr >> 16, intr & 0xffff, error,
+ data);
+ error = -1;
+ goto done;
+ } else {
+ FM10K_SW_TRACE("sbus %s: serdes %u: interrupt "
+ "0x%04x (param=0x%08x, result=0x%08x)", sb->name, serdes,
+ int_num, param, data);
+ if (result)
+ *result = data;
+ }
+
+done:
+ return (error);
+}
+
+
+int
+fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int divider, width_mode;
+ int error = 0;
+ uint32_t data;
+
+ /*
+ * Bit rate and width
+ */
+ if (speed == 25) {
+ /* 25G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_25G;
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ divider = FM10K_SW_SERDES_DIVIDER_ETHMODE_10G;
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ /*
+ * state = CONFIGURED
+ */
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring",
+ sw->epl_sbus->name, serdes);
+
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_do_crc(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_bit_rate(sw, serdes, divider);
+ if (error)
+ goto done;
+
+ fm10k_epl_serdes_set_pcsl_width_mode(sw, serdes,
+ width_mode);
+
+ /*
+ * Data select
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX data select",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_spico_int02_retry(sw, serdes, 0x1ff,
+ FM10K_SW_SERDES_INT02_TIMEOUT_MS);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_reg_write(sw, serdes, 0x21, 0x0c00);
+ if (error)
+ goto done;
+
+ /*
+ * Configure TX equalization
+ *
+ * Use defaults - copper and optical are the same.
+ *
+ * DEFAULTS| precursor | cursor | postcursor |
+ * --------+-----------+--------+------------+
+ * copper | 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ * optical| 0 | 0 | 15 |
+ * --------+-----------+--------+------------+
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursors",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_ATTEN, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_PRECUR, 0);
+ if (error)
+ goto done;
+ error = fm10k_epl_serdes_set_tx_eq(sw, serdes,
+ FM10K_SW_SERDES_EQ_SEL_POSTCUR, 15);
+ if (error)
+ goto done;
+
+ /*
+ * Configure 'options', which means:
+ * - PLL calibration mode
+ * - Phase slip
+ * - RX termination
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting PLL calibration mode",
+ sw->epl_sbus->name, serdes);
+ /* enable PLL calibration */
+ data = (1 << 0) | (1 << 1);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x11, data);
+ if (error)
+ goto done;
+ /* There appears to be no action to take for phase slip */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring RX termination",
+ sw->epl_sbus->name, serdes);
+ /* RX termination appears to default to AVDD */
+ data = (1 << 0);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x2b, data);
+ if (error)
+ goto done;
+
+ /*
+ * Set TX and RX lane polarity.
+ * Assuming no inversion required.
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring TX/RX lane polarity",
+ sw->epl_sbus->name, serdes);
+ data = 0x0300;
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x13, data, NULL);
+ if (error)
+ goto done;
+
+ /*
+ * Force local fault
+ */
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /*
+ * Enable TX/RX
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX/RX",
+ sw->epl_sbus->name, serdes);
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /*
+ * Clear and enable interrupts
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring interrupts",
+ sw->epl_sbus->name, serdes);
+ error = fm10k_sbus_write(sw->epl_sbus,
+ FM10K_SW_SBUS_ADDR_EPL_SERDES(serdes),
+ FM10K_SERDES_SPICO_REG_0X08, 0);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ unsigned int width_mode;
+ int error = 0;
+ int near_loopback = sw->serdes_loopback;
+
+ if (speed == 25) {
+ /* 25G */
+ width_mode = FM10K_SW_SERDES_WIDTH_40;
+ } else {
+ /* 10G */
+ width_mode = FM10K_SW_SERDES_WIDTH_20;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_pcsl_bit_slip(sw, serdes);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_set_width_mode(sw, serdes, width_mode);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: enabling TX output",
+ sw->epl_sbus->name, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x07);
+ if (error)
+ goto done;
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error = fm10k_epl_serdes_init_signal_ok(sw, serdes, 0);
+ if (error)
+ goto done;
+
+ if (near_loopback) {
+ /* enable near loopback */
+ error = fm10k_epl_serdes_configure_near_loopback(lane);
+ if (error)
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error = 0;
+
+ fm10k_epl_serdes_set_signal_detect(lane,
+ FM10K_SW_LANE_OVERRIDE_FORCE_BAD);
+
+ /* is this delay necessary? in some places the IES SDK does this */
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_reset(sw, serdes);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_bit_rate(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int divider)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting bit rate",
+ sw->epl_sbus->name, serdes);
+
+ data = divider & 0x7ff;
+ data |= (1 << 12);
+ data |= (1 << 15);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x05, data);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_set_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting width mode to %u",
+ sw->epl_sbus->name, serdes, width_mode);
+
+ data = FM10K_SW_SERDES_WIDTH_20 | (FM10K_SW_SERDES_WIDTH_20 << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+ data = width_mode | (width_mode << 4);
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x14, data);
+ if (error)
+ goto done;
+
+done:
+ fm10k_udelay(FM10K_SW_SERDES_RESET_DELAY_US);
+ return error;
+}
+
+static int
+fm10k_epl_serdes_set_tx_eq(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int which, unsigned int tx_eq)
+{
+ int error;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting TX cursor %u to %u",
+ sw->epl_sbus->name, serdes, which, tx_eq);
+
+ data = (tx_eq & 0xff) | (which << 14);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x15, data);
+ if (error)
+ goto done;
+
+done:
+ return error;
+}
+
+void
+fm10k_epl_serdes_set_signal_detect(struct fm10k_ext_port_lane *lane,
+ unsigned int override)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int eplno = lane->port->eplno;
+ uint32_t data;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal detect to %s",
+ sw->epl_sbus->name, lane->abs_laneno,
+ (override == 0) ? "NORMAL" :
+ (override == 1) ? "FORCE_GOOD" :
+ (override == 2) ? "FORCE_BAD" : "<unknown>");
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno));
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LANE_SIGNAL_DETECT_CFG_SD_OVERRIDE,
+ override, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(eplno, lane->rel_laneno), data);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static void
+fm10k_epl_serdes_set_pcsl_width_mode(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int width_mode)
+{
+ unsigned int eplno, lane;
+ uint32_t pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl width mode",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ if (width_mode == FM10K_SW_SERDES_WIDTH_40) {
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ } else {
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_GB_NARROW;
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_TX_GB_NARROW;
+ }
+ pcsl_cfg &= ~FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+}
+
+static int
+fm10k_epl_serdes_set_pcsl_bit_slip(struct fm10k_switch *sw,
+ unsigned int serdes)
+{
+ unsigned int eplno, lane;
+ int error;
+ uint32_t int_return, pcsl_cfg;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting pcsl bit slip",
+ sw->epl_sbus->name, serdes);
+
+ fm10k_epl_serdes_eplno_lane(sw, serdes, &eplno, &lane);
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x0c, (1 << 7), &int_return);
+ if (error)
+ goto done;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ pcsl_cfg = fm10k_read_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane));
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_ENABLE;
+ if (int_return & 0x1)
+ pcsl_cfg |= FM10K_SW_PCSL_CFG_RX_BIT_SLIP_INITIAL;
+ fm10k_write_switch_reg(sw, FM10K_SW_PCSL_CFG(eplno, lane), pcsl_cfg);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0c, (1 << 8));
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_init_signal_ok(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int threshold)
+{
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: setting signal OK threshold to %u",
+ sw->epl_sbus->name, serdes, threshold);
+
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x20, 0x20, NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_read_modify_write(sw, serdes, 0x080,
+ (threshold & 0xf) << 2, 0x3c, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_int02_retry(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t data, unsigned int timeout_ms)
+{
+ uint64_t start_time_us, elapsed_time_ms;
+ uint32_t result;
+ int error;
+
+ start_time_us = fm10k_uptime_us();
+ elapsed_time_ms = 0;
+ while (elapsed_time_ms < timeout_ms) {
+ elapsed_time_ms = (fm10k_uptime_us() - start_time_us) / 1000;
+ error =
+ fm10k_epl_serdes_spico_int(sw, serdes, 0x02, data, &result);
+ if (result == 0x02) {
+ error = 0;
+ break;
+ }
+
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ }
+
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_reg_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x8000 | (addr & 0xff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t *result)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), result);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x1a, 0x00, result);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr, uint32_t data)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x18,
+ 0x4000 | (addr & 0x3fff), NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x19, data, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_dma_esb_read_modify_write(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int addr,
+ uint32_t data, uint32_t mask, uint32_t *result)
+{
+ int error;
+ uint32_t read_data;
+
+ error = fm10k_epl_serdes_dma_esb_read(sw, serdes, addr, &read_data);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_dma_esb_write(sw, serdes, addr,
+ (data & mask) | (read_data & ~mask));
+ if (error)
+ goto done;
+ if (result)
+ *result = (data & mask) | (read_data & ~mask);
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_spico_wr_only_int(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int int_num, uint32_t param)
+{
+ int error;
+ uint32_t result;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, int_num, param, &result);
+ if (error)
+ goto done;
+ if (result != int_num) {
+ error = -1;
+ goto done;
+ }
+done:
+ return (error);
+}
+
+static void
+fm10k_epl_serdes_eplno_lane(struct fm10k_switch *sw,
+ unsigned int serdes, unsigned int *eplno, unsigned int *lane)
+{
+ if (sw) {
+ *eplno = serdes / FM10K_SW_EPL_LANES;
+ *lane = serdes % FM10K_SW_EPL_LANES;
+ }
+}
+
+static int
+fm10k_epl_serdes_configure_near_loopback(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ uint32_t data;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring near loopback",
+ sw->epl_sbus->name, serdes);
+
+ /* assuming no lane polarity has been set */
+ /* set near loopback mode */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ data = (1 << 8) | (1 << 0);
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x08, data);
+ if (error)
+ goto done;
+
+ /* disable tx output, leave tx/ex enabled */
+ fm10k_udelay(FM10K_SW_SERDES_CONFIG_DELAY_US);
+ error =
+ fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x01, 0x03);
+ if (error)
+ goto done;
+
+ /* set signal detect override to normal */
+ fm10k_epl_serdes_set_signal_detect(lane, FM10K_SW_LANE_OVERRIDE_NORMAL);
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: configuring serdes for dfe tuning",
+ sw->epl_sbus->name, serdes);
+
+ /*
+ * Configure the serdes to run DFE tuning
+ */
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (0 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_HF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (1 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_LF & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (2 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_DC & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26,
+ ((2 << 12) | (3 << 8) | (FM10K_SW_SERDES_DFE_DEFAULT_BW & 0xff)),
+ NULL);
+ if (error)
+ goto done;
+
+ /* allow early link up */
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x26, 0x5b01, NULL);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning initial calibration (ical)
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe ical",
+ sw->epl_sbus->name, serdes);
+
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_A25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_A10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x01);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_ical_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status, converged;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ if ((dfe_status & 0x11) == 0x11) {
+ *status = FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS;
+ } else {
+ error = fm10k_epl_serdes_get_ical_result(sw,
+ serdes, &converged);
+ if (error)
+ goto done;
+ if (converged)
+ *status = FM10K_SW_SERDES_DFE_ICAL_CONVERGED;
+ else
+ *status = FM10K_SW_SERDES_DFE_ICAL_FAILED;
+ }
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ unsigned int speed = lane->port->lane_speed;
+ int error;
+ uint32_t param;
+
+ /*
+ * Start DFE tuning periodic calibration (pcal), just one
+ */
+ FM10K_SW_TRACE("sbus %s: serdes %u: starting dfe pcal",
+ sw->epl_sbus->name, serdes);
+ if (speed == 25)
+ param = FM10K_SW_LINK_OPT_PARAM_B25G;
+ else
+ param = FM10K_SW_LINK_OPT_PARAM_B10G;
+
+ if ((sw->epl_serdes_code_version_build_id >> 16) > 0x1055)
+ error = fm10k_epl_serdes_config_dfe_param(sw, serdes, param);
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x0a, 0x02);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_dfe_pcal_status(struct fm10k_ext_port_lane *lane,
+ uint32_t *status)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ uint32_t dfe_status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+ if (dfe_status & 0x02)
+ *status = FM10K_SW_SERDES_DFE_PCAL_COMPLETE;
+ else
+ *status = FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS;
+
+done:
+ return (error);
+}
+
+int
+fm10k_epl_serdes_stop_dfe_tuning(struct fm10k_ext_port_lane *lane,
+ unsigned int force_reset)
+{
+ struct fm10k_switch *sw = lane->port->ports->sw;
+ unsigned int serdes = lane->abs_laneno;
+ int error;
+ unsigned int ical_stopped, pcal_stopped;
+ unsigned int i;
+ uint32_t dfe_status, status;
+
+ error = fm10k_epl_serdes_get_dfe_status(sw, serdes, &dfe_status);
+ if (error)
+ goto done;
+
+ ical_stopped = 0;
+ pcal_stopped = 0;
+
+ if (!force_reset) {
+ if (dfe_status & 0x3) {
+ /* stop all DFE tuning */
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes,
+ 0x0a, 0);
+ if (error)
+ goto done;
+
+ for (i = 0;
+ i < FM10K_SW_SERDES_ICAL_STOP_MAX_CYCLES;
+ i++) {
+ error = fm10k_epl_serdes_dfe_ical_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS) {
+ ical_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+
+ if (ical_stopped)
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES;
+ i++) {
+ error =
+ fm10k_epl_serdes_dfe_pcal_status
+ (lane, &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 0;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+
+ if (ical_stopped && pcal_stopped && (dfe_status & 0x40)) {
+ error = fm10k_epl_serdes_start_dfe_pcal(lane);
+ if (error)
+ goto done;
+ pcal_stopped = 0;
+ for (i = 0;
+ i < FM10K_SW_SERDES_PCAL_STOP_MAX_CYCLES; i++) {
+ error = fm10k_epl_serdes_dfe_pcal_status(lane,
+ &status);
+ if (error)
+ goto done;
+ if (status !=
+ FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS) {
+ pcal_stopped = 1;
+ break;
+ }
+ fm10k_udelay
+ (FM10K_SW_SERDES_DFE_STOP_CYCLE_DELAY_US);
+ }
+ }
+ }
+
+ if (!ical_stopped || !pcal_stopped) {
+ error = -1;
+ goto done;
+ }
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_config_dfe_param(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t param)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x18, 0x07);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param & 0xffff);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_wr_only_int(sw, serdes, 0x19,
+ param >> 16);
+ if (error)
+ goto done;
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_dfe_status(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *status)
+{
+ int error;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126, (0x0b << 8),
+ status);
+ if (error)
+ goto done;
+
+ FM10K_SW_TRACE("sbus %s: serdes %u: dfe status: "
+ "coarse_active=%u fine_active=%u adaptive=%u",
+ sw->epl_sbus->name, serdes, *status & (1 << 0),
+ *status & (1 << 1), *status & (1 << 6));
+
+done:
+ return (error);
+}
+
+static int
+fm10k_epl_serdes_get_ical_result(struct fm10k_switch *sw,
+ unsigned int serdes, uint32_t *converged)
+{
+ uint32_t val1;
+ uint32_t val2;
+ uint32_t diff;
+ int error;
+
+ *converged = 0;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12), &val1);
+ if (error)
+ goto done;
+
+ error = fm10k_epl_serdes_spico_int(sw, serdes, 0x126,
+ (4 << 12) | (1 << 8), &val2);
+ if (error)
+ goto done;
+
+ /* sign extension */
+ if (val1 & 0x8000)
+ val1 |= 0xffff0000;
+ if (val2 & 0x8000)
+ val2 |= 0xffff0000;
+ diff = abs(val2 - val1);
+
+ if (diff >= FM10K_SW_SERDES_DFE_DATA_LEVEL0_THRESHOLD)
+ *converged = 1;
+
+done:
+ return (error);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_serdes.h b/drivers/net/fm10k/switch/fm10k_serdes.h
new file mode 100644
index 0000000..f13ad3a
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_serdes.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SERDES_H_
+#define _FM10K_SW_SERDES_H_
+
+
+#define FM10K_SW_SERDES_DFE_ICAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_ICAL_CONVERGED 1
+#define FM10K_SW_SERDES_DFE_ICAL_FAILED 2
+#define FM10K_SW_SERDES_DFE_PCAL_IN_PROGRESS 0
+#define FM10K_SW_SERDES_DFE_PCAL_COMPLETE 1
+
+
+int fm10k_epl_serdes_reset_and_load_all(struct fm10k_switch *sw);
+int fm10k_epl_serdes_start_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_continue_bringup(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_disable(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_configure_for_dfe_tuning(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_start_dfe_ical(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_ical_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_start_dfe_pcal(struct fm10k_ext_port_lane *lane);
+int fm10k_epl_serdes_dfe_pcal_status
+ (struct fm10k_ext_port_lane *lane, uint32_t *status);
+int fm10k_epl_serdes_stop_dfe_tuning
+ (struct fm10k_ext_port_lane *lane, unsigned int force_reset);
+void fm10k_epl_serdes_set_signal_detect
+ (struct fm10k_ext_port_lane *lane, unsigned int override);
+
+#endif /* _FM10K_SW_SERDES_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_sm.c b/drivers/net/fm10k/switch/fm10k_sm.c
new file mode 100644
index 0000000..26fe7dd
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.c
@@ -0,0 +1,190 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <rte_malloc.h>
+
+#include "../base/fm10k_osdep.h"
+#include "fm10k_debug.h"
+#include "fm10k_sm.h"
+
+static void *fm10k_sm_handle_event(void *ptr);
+static void *fm10k_sm_handle_timer(void *ptr);
+static void fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap);
+static uint16_t fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap);
+static void fm10k_sm_event_heap_push
+ (struct fm10k_sm_event_heap *heap, uint16_t id);
+
+
+struct fm10k_sm *
+fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx)
+{
+ struct fm10k_sm *sm;
+ struct fm10k_sm_timer *timer;
+ unsigned int i;
+
+ sm = (struct fm10k_sm *)rte_zmalloc("fm10k_sm",
+ sizeof(struct fm10k_sm), 0);
+ if (sm == NULL)
+ goto fail;
+
+ sm->is_running = true;
+ sm->num_events = num_events;
+ fm10k_sm_event_heap_init(&sm->events);
+ pthread_create(&sm->event_task, NULL, fm10k_sm_handle_event, sm);
+ sm->num_timers = num_timers;
+ if (sm->num_timers > 0) {
+ sm->timers = (struct fm10k_sm_timer *)rte_zmalloc
+ ("fm10k_sm_timers",
+ sizeof(struct fm10k_sm_timer) * sm->num_timers, 0);
+ if (sm->timers == NULL)
+ goto fail;
+ }
+
+ sm->event_handler = event_handler;
+ sm->timer_handler = timer_handler;
+ sm->ctx = ctx;
+
+ for (i = 0; i < sm->num_timers; i++) {
+ timer = &sm->timers[i];
+ sem_init(&timer->tq, 0, 0);
+ timer->sm = sm;
+ pthread_create(&timer->t, NULL, fm10k_sm_handle_timer, timer);
+ timer->timer_id = i;
+ }
+
+ return sm;
+
+fail:
+ if (sm)
+ fm10k_sm_detach(sm);
+
+ return NULL;
+}
+
+void
+fm10k_sm_detach(struct fm10k_sm *sm)
+{
+ int i;
+
+ sm->is_running = false;
+ pthread_join(sm->event_task, NULL);
+ for (i = 0; i < sm->num_timers; i++)
+ pthread_join(sm->timers[i].t, NULL);
+ rte_free(sm);
+}
+
+void
+fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id)
+{
+ fm10k_sm_event_heap_push(&sm->events, event_id);
+}
+
+void
+fm10k_sm_timer_start(struct fm10k_sm *sm, uint16_t timer_id,
+ unsigned int timeout_ms)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+
+ gettimeofday(&timer->start, 0);
+ timer->timeout_ms = timeout_ms;
+ timer->running = 1;
+ sem_post(&timer->tq);
+}
+
+/*
+ * It is possible that an expiration events from this timer will
+ * occur after this is called.
+ */
+void
+fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id)
+{
+ struct fm10k_sm_timer *timer = &sm->timers[timer_id];
+ timer->running = 0;
+}
+
+static void *
+fm10k_sm_handle_event(void *ctx)
+{
+ struct fm10k_sm *sm = ctx;
+ uint16_t event_id;
+
+ while (sm->is_running) {
+ event_id = fm10k_sm_event_heap_pop(&sm->events);
+ sm->event_handler(sm, event_id);
+ }
+ return NULL;
+}
+
+static void *
+fm10k_sm_handle_timer(void *ctx)
+{
+ struct fm10k_sm_timer *timer = ctx;
+ struct fm10k_sm *sm = timer->sm;
+ struct timeval now;
+ struct timeval sleep;
+ uint64_t cost, timeout;
+
+ while (sm->is_running) {
+ sem_wait(&timer->tq);
+ if (timer->running) {
+ gettimeofday(&now, 0);
+ cost = (now.tv_sec - timer->start.tv_sec) * 1000000;
+ cost += (now.tv_usec - timer->start.tv_usec);
+ if (cost < timer->timeout_ms * 1000) {
+ timeout = timer->timeout_ms * 1000;
+ timeout -= cost;
+ sleep.tv_sec = timeout / 1000000;
+ sleep.tv_usec = timeout % 1000000;
+ select(0, NULL, NULL, NULL, &sleep);
+ }
+ if (timer->running)
+ sm->timer_handler(sm, timer->timer_id);
+ }
+ }
+
+ return NULL;
+}
+
+
+static void
+fm10k_sm_event_heap_init(struct fm10k_sm_event_heap *heap)
+{
+ heap->count = 0;
+ pthread_mutex_init(&heap->lock, NULL);
+ sem_init(&heap->s, 0, 0);
+}
+
+static uint16_t
+fm10k_sm_event_heap_pop(struct fm10k_sm_event_heap *heap)
+{
+ int i;
+ uint16_t id;
+
+ sem_wait(&heap->s);
+ pthread_mutex_lock(&heap->lock);
+ id = heap->heap[0];
+ for (i = 1; i < heap->count; i++)
+ heap->heap[i - 1] = heap->heap[i];
+ heap->count--;
+ pthread_mutex_unlock(&heap->lock);
+ return id;
+}
+
+static void
+fm10k_sm_event_heap_push(struct fm10k_sm_event_heap *heap, uint16_t id)
+{
+ pthread_mutex_lock(&heap->lock);
+ if (heap->count >= FM10K_SM_ID_HEAP_MAX) {
+ pthread_mutex_unlock(&heap->lock);
+ return;
+ }
+ heap->heap[heap->count] = id;
+ heap->count++;
+ pthread_mutex_unlock(&heap->lock);
+ sem_post(&heap->s);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_sm.h b/drivers/net/fm10k/switch/fm10k_sm.h
new file mode 100644
index 0000000..75098d4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_sm.h
@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_SM_H_
+#define _FM10K_SW_SM_H_
+
+#include <stdint.h>
+#include <signal.h>
+#include <sys/time.h>
+#include <pthread.h>
+#include <semaphore.h>
+#include <stdbool.h>
+
+
+/*
+ * This framework provides state machine implementation support for use in
+ * the rest of the driver.
+ *
+ * The model is that for a given state machine, there are N>0 events
+ * identified by the integers [0..N-1] and there are M>=0 timers identified
+ * by the integers [0..M-1].
+ *
+ * The timers are one-shot timers.
+ *
+ * Since all state machine processing occurs in the context of the
+ * task queue, if that task queue was created with only a single thread, then
+ * no locking is required in the provided event and timer handlers to
+ * prevent overlapping execution.
+ *
+ */
+
+struct fm10k_sm;
+
+struct fm10k_sm_timer {
+ struct fm10k_sm *sm;
+ pthread_t t;
+ sem_t tq;
+ struct timeval start;
+ uint16_t timeout_ms;
+ uint16_t timer_id;
+ uint8_t running;
+};
+
+typedef void (*fm10k_sm_process_event_t)(struct fm10k_sm *sm, uint16_t event);
+typedef void (*fm10k_sm_process_timer_t)(struct fm10k_sm *sm, uint16_t timer);
+
+#define FM10K_SM_ID_HEAP_MAX 128
+struct fm10k_sm_event_heap {
+ pthread_mutex_t lock;
+ sem_t s;
+ uint16_t count;
+ uint16_t heap[FM10K_SM_ID_HEAP_MAX];
+};
+
+struct fm10k_sm {
+ bool is_running;
+ pthread_t event_task;
+ struct fm10k_sm_timer *timers;
+ struct fm10k_sm_event_heap events;
+ fm10k_sm_process_event_t event_handler;
+ fm10k_sm_process_timer_t timer_handler;
+ void *ctx;
+ unsigned int state;
+ uint16_t num_events;
+ uint16_t num_timers;
+ uint8_t portno;
+ uint8_t laneno;
+};
+
+struct fm10k_sm *fm10k_sm_attach(uint16_t num_events,
+ uint16_t num_timers, fm10k_sm_process_event_t event_handler,
+ fm10k_sm_process_timer_t timer_handler, void *ctx);
+void fm10k_sm_detach(struct fm10k_sm *sm);
+void fm10k_sm_send_event(struct fm10k_sm *sm, uint16_t event_id);
+void fm10k_sm_timer_start(struct fm10k_sm *sm,
+ uint16_t timer_id, unsigned int timeout_ms);
+void fm10k_sm_timer_cancel(struct fm10k_sm *sm, uint16_t timer_id);
+
+#endif /* _FM10K_SW_SM_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.c b/drivers/net/fm10k/switch/fm10k_spico_code.c
new file mode 100644
index 0000000..cfab632
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.c
@@ -0,0 +1,2966 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include "fm10k_spico_code.h"
+
+
+/* Production versions */
+
+
+/* SBus master code (production version)
+ * source file: release_2015_01_14/sbus_master.0x1013_0001.rom
+ */
+
+const uint32_t fm10000_sbus_master_code_version_build_id_prd = 0x10130001;
+const uint32_t fm10000_sbus_master_code_size_prd = 3338;
+
+const uint16_t fm10000_sbus_master_code_prd[] = {
+ 0x132, 0x002, 0x006, 0x000, 0x390, 0x002, 0x080, 0x247,
+ 0x071, 0x247, 0x01f, 0x247, 0x020, 0x247, 0x02e, 0x247,
+ 0x02f, 0x247, 0x03f, 0x247, 0x040, 0x247, 0x042, 0x247,
+ 0x043, 0x247, 0x03d, 0x0c0, 0x02d, 0x247, 0x031, 0x247,
+ 0x032, 0x081, 0x18b, 0x247, 0x039, 0x248, 0x03a, 0x181,
+ 0x247, 0x034, 0x248, 0x036, 0x209, 0x247, 0x033, 0x248,
+ 0x035, 0x305, 0x034, 0x247, 0x037, 0x248, 0x038, 0x190,
+ 0x247, 0x03b, 0x248, 0x03c, 0x3d4, 0x1ff, 0x00c, 0x3d5,
+ 0x100, 0x00c, 0x3d4, 0x2ff, 0x00c, 0x3c7, 0x0fe, 0x018,
+ 0x0c2, 0x019, 0x044, 0x233, 0x000, 0x152, 0x01b, 0x347,
+ 0x01b, 0x030, 0x085, 0x187, 0x247, 0x03e, 0x3c7, 0x100,
+ 0x041, 0x080, 0x247, 0x065, 0x3c4, 0x3fc, 0x010, 0x3d4,
+ 0x22f, 0x00e, 0x347, 0x033, 0x008, 0x247, 0x008, 0x081,
+ 0x189, 0x227, 0x2c7, 0x044, 0x220, 0x2c7, 0x045, 0x220,
+ 0x2c7, 0x046, 0x220, 0x2c7, 0x047, 0x220, 0x2c7, 0x048,
+ 0x220, 0x2c7, 0x049, 0x220, 0x2c7, 0x04a, 0x220, 0x2c7,
+ 0x04b, 0x3a7, 0x05b, 0x1a7, 0x3a5, 0x08d, 0x2c7, 0x04c,
+ 0x0c0, 0x04d, 0x044, 0x07d, 0x001, 0x0df, 0x02c, 0x000,
+ 0x009, 0x322, 0x02c, 0x009, 0x005, 0x347, 0x02d, 0x02c,
+ 0x3d4, 0x3df, 0x00e, 0x307, 0x02c, 0x384, 0x01f, 0x229,
+ 0x220, 0x3b0, 0x0aa, 0x287, 0x199, 0x390, 0x000, 0x041,
+ 0x040, 0x045, 0x020, 0x12c, 0x000, 0x020, 0x163, 0x000,
+ 0x020, 0x175, 0x000, 0x020, 0x27e, 0x000, 0x020, 0x362,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x1de, 0x000, 0x020, 0x330, 0x001, 0x020, 0x270, 0x001,
+ 0x020, 0x2af, 0x001, 0x020, 0x2a1, 0x001, 0x020, 0x338,
+ 0x001, 0x020, 0x285, 0x001, 0x020, 0x2b9, 0x001, 0x020,
+ 0x2a8, 0x001, 0x020, 0x001, 0x002, 0x020, 0x105, 0x001,
+ 0x020, 0x19e, 0x001, 0x020, 0x3e6, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000,
+ 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a,
+ 0x000, 0x020, 0x10a, 0x000, 0x020, 0x10a, 0x000, 0x020,
+ 0x10a, 0x000, 0x044, 0x140, 0x000, 0x054, 0x362, 0x03d,
+ 0x000, 0x001, 0x0b0, 0x364, 0x03d, 0x001, 0x001, 0x0c6,
+ 0x307, 0x02e, 0x044, 0x187, 0x000, 0x055, 0x141, 0x02e,
+ 0x307, 0x030, 0x302, 0x02e, 0x00b, 0x374, 0x0c0, 0x02e,
+ 0x327, 0x02d, 0x000, 0x367, 0x044, 0x140, 0x000, 0x3d4,
+ 0x3df, 0x00e, 0x207, 0x009, 0x00a, 0x04f, 0x04e, 0x055,
+ 0x3d5, 0x200, 0x014, 0x000, 0x35d, 0x054, 0x000, 0x3f8,
+ 0x055, 0x081, 0x364, 0x010, 0x001, 0x001, 0x00d, 0x364,
+ 0x011, 0x002, 0x009, 0x009, 0x3c4, 0x3fe, 0x010, 0x3d4,
+ 0x3bf, 0x00e, 0x080, 0x101, 0x364, 0x010, 0x002, 0x001,
+ 0x00a, 0x364, 0x011, 0x001, 0x001, 0x006, 0x3c4, 0x3fd,
+ 0x010, 0x111, 0x045, 0x347, 0x026, 0x018, 0x347, 0x027,
+ 0x019, 0x347, 0x028, 0x01b, 0x347, 0x029, 0x01c, 0x044,
+ 0x222, 0x000, 0x0a1, 0x000, 0x31e, 0x347, 0x026, 0x018,
+ 0x347, 0x027, 0x019, 0x044, 0x233, 0x000, 0x347, 0x01b,
+ 0x028, 0x347, 0x01c, 0x029, 0x0a2, 0x000, 0x30c, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x000, 0x001, 0x016, 0x362,
+ 0x01b, 0x001, 0x001, 0x013, 0x362, 0x01b, 0x002, 0x001,
+ 0x00d, 0x362, 0x01b, 0x003, 0x001, 0x013, 0x362, 0x01b,
+ 0x004, 0x001, 0x013, 0x04e, 0x045, 0x364, 0x03d, 0x002,
+ 0x001, 0x005, 0x044, 0x01a, 0x001, 0x000, 0x3f7, 0x348,
+ 0x01b, 0x01a, 0x000, 0x3f2, 0x348, 0x01b, 0x01a, 0x000,
+ 0x3ed, 0x101, 0x247, 0x018, 0x3d7, 0x0ff, 0x019, 0x000,
+ 0x074, 0x3c7, 0x100, 0x014, 0x364, 0x004, 0x100, 0x009,
+ 0x00f, 0x044, 0x08d, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x055, 0x0c1,
+ 0x03d, 0x0bf, 0x000, 0x2b7, 0x000, 0x2bc, 0x3c7, 0x3ff,
+ 0x028, 0x087, 0x0b6, 0x0c1, 0x04e, 0x044, 0x37c, 0x000,
+ 0x347, 0x067, 0x029, 0x2c7, 0x02a, 0x322, 0x04e, 0x009,
+ 0x00c, 0x3c2, 0x016, 0x067, 0x009, 0x00c, 0x0c1, 0x028,
+ 0x0a7, 0x000, 0x298, 0x347, 0x033, 0x028, 0x000, 0x3fa,
+ 0x347, 0x034, 0x028, 0x000, 0x3f5, 0x111, 0x009, 0x3ff,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x040,
+ 0x103, 0x051, 0x042, 0x2a7, 0x001, 0x00c, 0x040, 0x280,
+ 0x051, 0x1a7, 0x101, 0x050, 0x225, 0x042, 0x280, 0x045,
+ 0x287, 0x045, 0x364, 0x005, 0x008, 0x009, 0x3fd, 0x347,
+ 0x01b, 0x00a, 0x347, 0x01c, 0x00b, 0x3c7, 0x041, 0x00c,
+ 0x020, 0x260, 0x000, 0x364, 0x005, 0x008, 0x009, 0x3fd,
+ 0x3d5, 0x100, 0x00c, 0x3c7, 0x042, 0x00c, 0x044, 0x260,
+ 0x000, 0x364, 0x005, 0x010, 0x001, 0x3fd, 0x307, 0x005,
+ 0x384, 0x007, 0x262, 0x004, 0x009, 0x009, 0x347, 0x002,
+ 0x01b, 0x347, 0x003, 0x01c, 0x045, 0x262, 0x006, 0x009,
+ 0x3ef, 0x141, 0x020, 0x0c0, 0x01b, 0x0c0, 0x01c, 0x045,
+ 0x347, 0x019, 0x00d, 0x1c7, 0x00d, 0x345, 0x018, 0x00d,
+ 0x3d5, 0x200, 0x00c, 0x364, 0x005, 0x008, 0x001, 0x3fd,
+ 0x3d4, 0x1ff, 0x00c, 0x045, 0x364, 0x005, 0x008, 0x009,
+ 0x3fd, 0x0c0, 0x00c, 0x020, 0x260, 0x000, 0x054, 0x347,
+ 0x032, 0x031, 0x307, 0x031, 0x044, 0x2ad, 0x000, 0x141,
+ 0x031, 0x307, 0x030, 0x302, 0x031, 0x00b, 0x004, 0x0c0,
+ 0x031, 0x2a7, 0x001, 0x00a, 0x347, 0x031, 0x032, 0x347,
+ 0x031, 0x070, 0x000, 0x00b, 0x307, 0x032, 0x302, 0x031,
+ 0x009, 0x3e2, 0x3c7, 0x3ff, 0x070, 0x055, 0x3d4, 0x2ff,
+ 0x00e, 0x0a3, 0x020, 0x091, 0x000, 0x044, 0x1b9, 0x000,
+ 0x362, 0x01b, 0x001, 0x001, 0x009, 0x362, 0x01b, 0x016,
+ 0x001, 0x004, 0x000, 0x00b, 0x0cf, 0x019, 0x044, 0x233,
+ 0x000, 0x307, 0x01b, 0x003, 0x004, 0x0a0, 0x045, 0x304,
+ 0x035, 0x227, 0x1bc, 0x2e2, 0x003, 0x001, 0x093, 0x2e2,
+ 0x002, 0x001, 0x046, 0x2e2, 0x001, 0x001, 0x063, 0x227,
+ 0x001, 0x03d, 0x2e4, 0x001, 0x009, 0x010, 0x2e4, 0x002,
+ 0x009, 0x019, 0x2e4, 0x004, 0x009, 0x029, 0x2e4, 0x008,
+ 0x009, 0x029, 0x000, 0x02b, 0x041, 0x0a2, 0x1a7, 0x3b5,
+ 0x03a, 0x044, 0x1b1, 0x001, 0x043, 0x3a4, 0x3fe, 0x000,
+ 0x3e1, 0x041, 0x327, 0x018, 0x041, 0x044, 0x08d, 0x001,
+ 0x247, 0x01e, 0x043, 0x2c7, 0x018, 0x044, 0x0fb, 0x001,
+ 0x043, 0x3a4, 0x3fd, 0x000, 0x3cd, 0x3a4, 0x3fb, 0x000,
+ 0x3c9, 0x3a4, 0x3f7, 0x000, 0x3c5, 0x0a1, 0x045, 0x227,
+ 0x3a4, 0x0ff, 0x197, 0x384, 0x01f, 0x001, 0x010, 0x262,
+ 0x001, 0x001, 0x00e, 0x262, 0x002, 0x001, 0x00f, 0x262,
+ 0x003, 0x001, 0x00d, 0x000, 0x3ea, 0x000, 0x3e8, 0x044,
+ 0x079, 0x002, 0x000, 0x3e3, 0x000, 0x3e1, 0x000, 0x3df,
+ 0x0bf, 0x1aa, 0x2a8, 0x224, 0x19a, 0x384, 0x003, 0x001,
+ 0x00c, 0x262, 0x001, 0x001, 0x017, 0x262, 0x002, 0x001,
+ 0x015, 0x000, 0x015, 0x0c5, 0x019, 0x247, 0x01b, 0x0a2,
+ 0x1a7, 0x3b5, 0x03a, 0x2c7, 0x01c, 0x044, 0x222, 0x000,
+ 0x000, 0x3bd, 0x000, 0x3bb, 0x000, 0x3b9, 0x000, 0x3b7,
+ 0x000, 0x3b5, 0x054, 0x307, 0x066, 0x191, 0x384, 0x07f,
+ 0x3c7, 0x0fe, 0x018, 0x0c2, 0x019, 0x044, 0x233, 0x000,
+ 0x347, 0x01b, 0x064, 0x0a7, 0x044, 0x0bf, 0x003, 0x055,
+ 0x0a4, 0x020, 0x091, 0x000, 0x186, 0x285, 0x181, 0x0a7,
+ 0x1ab, 0x324, 0x004, 0x285, 0x055, 0x364, 0x010, 0x002,
+ 0x009, 0x026, 0x364, 0x011, 0x001, 0x009, 0x029, 0x247,
+ 0x064, 0x044, 0x0cd, 0x003, 0x3c4, 0x2ff, 0x065, 0x04f,
+ 0x04e, 0x055, 0x364, 0x065, 0x100, 0x001, 0x3fa, 0x327,
+ 0x065, 0x3a4, 0x007, 0x364, 0x065, 0x040, 0x001, 0x007,
+ 0x287, 0x302, 0x04e, 0x009, 0x3ec, 0x045, 0x364, 0x011,
+ 0x001, 0x001, 0x005, 0x3c4, 0x3fd, 0x010, 0x054, 0x04e,
+ 0x000, 0x3cc, 0x309, 0x023, 0x003, 0x024, 0x347, 0x023,
+ 0x04f, 0x00b, 0x01f, 0x3d5, 0x010, 0x00e, 0x08f, 0x187,
+ 0x395, 0x0a0, 0x247, 0x017, 0x080, 0x247, 0x016, 0x0c0,
+ 0x015, 0x3c5, 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4,
+ 0x3fe, 0x014, 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e,
+ 0x347, 0x04f, 0x009, 0x020, 0x20b, 0x002, 0x307, 0x04f,
+ 0x00b, 0x028, 0x3c5, 0x002, 0x03d, 0x3d5, 0x010, 0x00e,
+ 0x387, 0x0b2, 0x187, 0x385, 0x080, 0x247, 0x017, 0x08e,
+ 0x187, 0x385, 0x0e6, 0x247, 0x016, 0x0c0, 0x015, 0x3c5,
+ 0x001, 0x014, 0x3c5, 0x004, 0x014, 0x3c4, 0x3fe, 0x014,
+ 0x3c5, 0x002, 0x014, 0x3d4, 0x3ef, 0x00e, 0x000, 0x006,
+ 0x082, 0x208, 0x244, 0x03d, 0x0df, 0x02c, 0x0b3, 0x020,
+ 0x091, 0x000, 0x307, 0x018, 0x3a7, 0x051, 0x044, 0x061,
+ 0x001, 0x2c7, 0x061, 0x04c, 0x324, 0x061, 0x001, 0x020,
+ 0x040, 0x3b7, 0x126, 0x08b, 0x187, 0x044, 0x1c4, 0x001,
+ 0x009, 0x005, 0x3c7, 0x022, 0x01b, 0x042, 0x364, 0x01b,
+ 0x022, 0x009, 0x027, 0x04c, 0x348, 0x061, 0x061, 0x324,
+ 0x061, 0x04d, 0x151, 0x050, 0x000, 0x01c, 0x327, 0x04f,
+ 0x3a4, 0x0ff, 0x322, 0x050, 0x00f, 0x014, 0x040, 0x0aa,
+ 0x084, 0x044, 0x1c4, 0x001, 0x042, 0x362, 0x01b, 0x00a,
+ 0x009, 0x008, 0x04c, 0x325, 0x061, 0x04d, 0x141, 0x050,
+ 0x045, 0x384, 0x0ff, 0x040, 0x193, 0x220, 0x042, 0x384,
+ 0x00f, 0x041, 0x0a1, 0x111, 0x003, 0x005, 0x2a9, 0x000,
+ 0x3fc, 0x042, 0x045, 0x0a0, 0x0d0, 0x061, 0x04d, 0x101,
+ 0x151, 0x061, 0x009, 0x3fc, 0x045, 0x0c0, 0x04f, 0x0c0,
+ 0x050, 0x387, 0x051, 0x044, 0x073, 0x001, 0x044, 0x209,
+ 0x000, 0x051, 0x101, 0x051, 0x045, 0x307, 0x03f, 0x001,
+ 0x006, 0x247, 0x018, 0x000, 0x01f, 0x0a0, 0x287, 0x044,
+ 0x1b9, 0x000, 0x362, 0x01b, 0x012, 0x001, 0x012, 0x362,
+ 0x01b, 0x011, 0x009, 0x005, 0x347, 0x018, 0x040, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3eb, 0x0a0, 0x045, 0x347,
+ 0x018, 0x03f, 0x0c0, 0x019, 0x3c7, 0x070, 0x01b, 0x3c5,
+ 0x001, 0x01b, 0x044, 0x222, 0x000, 0x3d4, 0x3fe, 0x01b,
+ 0x044, 0x222, 0x000, 0x307, 0x03e, 0x044, 0x205, 0x000,
+ 0x3c5, 0x002, 0x01b, 0x044, 0x222, 0x000, 0x307, 0x03e,
+ 0x194, 0x044, 0x205, 0x000, 0x0c1, 0x019, 0x044, 0x233,
+ 0x000, 0x327, 0x01b, 0x0c0, 0x019, 0x3c5, 0x001, 0x01b,
+ 0x044, 0x222, 0x000, 0x3b5, 0x200, 0x045, 0x2c7, 0x01e,
+ 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001, 0x007, 0x041,
+ 0x044, 0x0fb, 0x001, 0x043, 0x121, 0x307, 0x030, 0x282,
+ 0x00b, 0x3f1, 0x045, 0x3b7, 0x03b, 0x307, 0x01e, 0x000,
+ 0x0b2, 0x091, 0x020, 0x362, 0x002, 0x054, 0x327, 0x06f,
+ 0x009, 0x009, 0x044, 0x08d, 0x001, 0x001, 0x080, 0x000,
+ 0x00d, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x012, 0x009,
+ 0x076, 0x044, 0x0b2, 0x001, 0x2c7, 0x042, 0x044, 0x1de,
+ 0x001, 0x287, 0x001, 0x06b, 0x2c7, 0x01d, 0x327, 0x040,
+ 0x3a4, 0x0ff, 0x2c7, 0x018, 0x0c9, 0x01a, 0x044, 0x2d7,
+ 0x001, 0x044, 0x206, 0x001, 0x324, 0x035, 0x0c0, 0x01a,
+ 0x087, 0x187, 0x385, 0x06c, 0x282, 0x00b, 0x015, 0x0c1,
+ 0x01a, 0x087, 0x187, 0x385, 0x0bc, 0x282, 0x00b, 0x00c,
+ 0x0c2, 0x01a, 0x088, 0x187, 0x08c, 0x282, 0x00b, 0x004,
+ 0x0c3, 0x01a, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x010, 0x041, 0x307, 0x01a, 0x18b, 0x305, 0x042, 0x385,
+ 0x0e0, 0x3b7, 0x33b, 0x044, 0x1b1, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3e8, 0x307, 0x03e, 0x044,
+ 0x205, 0x000, 0x0a0, 0x287, 0x044, 0x300, 0x001, 0x001,
+ 0x00b, 0x041, 0x307, 0x01d, 0x3b7, 0x03b, 0x044, 0x1b1,
+ 0x001, 0x043, 0x121, 0x307, 0x030, 0x282, 0x00b, 0x3ed,
+ 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387, 0x3ff, 0x3c7,
+ 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296, 0x001, 0x347,
+ 0x023, 0x043, 0x092, 0x020, 0x365, 0x002, 0x054, 0x044,
+ 0x1de, 0x001, 0x001, 0x005, 0x044, 0x239, 0x001, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b1, 0x020, 0x296,
+ 0x001, 0x040, 0x044, 0x1c9, 0x001, 0x001, 0x00c, 0x042,
+ 0x0c5, 0x019, 0x2c7, 0x01c, 0x247, 0x01b, 0x020, 0x222,
+ 0x000, 0x042, 0x080, 0x045, 0x044, 0x1b1, 0x001, 0x001,
+ 0x3fb, 0x041, 0x3a7, 0x0ff, 0x0c6, 0x019, 0x131, 0x001,
+ 0x00c, 0x044, 0x233, 0x000, 0x30a, 0x01c, 0x002, 0x3f8,
+ 0x081, 0x000, 0x003, 0x080, 0x043, 0x045, 0x307, 0x040,
+ 0x384, 0x0ff, 0x009, 0x017, 0x0a0, 0x287, 0x044, 0x1b9,
+ 0x000, 0x362, 0x01b, 0x011, 0x001, 0x00a, 0x121, 0x307,
+ 0x030, 0x282, 0x00b, 0x3f3, 0x0a0, 0x045, 0x347, 0x018,
+ 0x040, 0x0c0, 0x01a, 0x327, 0x040, 0x044, 0x2c3, 0x001,
+ 0x324, 0x034, 0x009, 0x3f2, 0x227, 0x045, 0x247, 0x01c,
+ 0x2c7, 0x01b, 0x280, 0x001, 0x3e9, 0x0c3, 0x019, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222,
+ 0x000, 0x0c2, 0x01b, 0x044, 0x222, 0x000, 0x347, 0x01e,
+ 0x019, 0x3d0, 0x041, 0x019, 0x307, 0x041, 0x044, 0x205,
+ 0x000, 0x044, 0x233, 0x000, 0x327, 0x01b, 0x00b, 0x3f6,
+ 0x0c0, 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x287,
+ 0x045, 0x2c7, 0x01e, 0x0a0, 0x287, 0x044, 0x300, 0x001,
+ 0x001, 0x007, 0x041, 0x044, 0x24e, 0x001, 0x043, 0x121,
+ 0x307, 0x030, 0x282, 0x00b, 0x3f1, 0x045, 0x09f, 0x18a,
+ 0x0a1, 0x1aa, 0x324, 0x01e, 0x001, 0x006, 0x305, 0x01e,
+ 0x000, 0x005, 0x208, 0x304, 0x01e, 0x300, 0x043, 0x00b,
+ 0x005, 0x080, 0x000, 0x008, 0x272, 0x3e8, 0x00b, 0x004,
+ 0x397, 0x3e8, 0x305, 0x033, 0x3b7, 0x13b, 0x000, 0x343,
+ 0x327, 0x06e, 0x0c0, 0x01a, 0x044, 0x2c3, 0x001, 0x0a9,
+ 0x305, 0x033, 0x247, 0x007, 0x344, 0x035, 0x007, 0x3d4,
+ 0x37f, 0x00e, 0x020, 0x091, 0x000, 0x327, 0x06f, 0x0c0,
+ 0x01a, 0x044, 0x2c3, 0x001, 0x324, 0x034, 0x009, 0x005,
+ 0x347, 0x06f, 0x040, 0x0ad, 0x0c1, 0x008, 0x247, 0x009,
+ 0x345, 0x033, 0x008, 0x344, 0x035, 0x008, 0x020, 0x091,
+ 0x000, 0x327, 0x06e, 0x054, 0x055, 0x0ab, 0x000, 0x3d2,
+ 0x327, 0x06f, 0x054, 0x055, 0x0af, 0x000, 0x3e7, 0x327,
+ 0x06e, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001, 0x0aa, 0x000,
+ 0x3c1, 0x327, 0x06f, 0x0c9, 0x01a, 0x044, 0x2c3, 0x001,
+ 0x0ae, 0x000, 0x3d3, 0x044, 0x2f8, 0x001, 0x362, 0x01b,
+ 0x011, 0x009, 0x00a, 0x044, 0x2d7, 0x001, 0x044, 0x206,
+ 0x001, 0x055, 0x045, 0x307, 0x034, 0x000, 0x3fc, 0x080,
+ 0x1bb, 0x2e2, 0x008, 0x00b, 0x00d, 0x0a1, 0x0c0, 0x01e,
+ 0x340, 0x01a, 0x01e, 0x001, 0x014, 0x1a8, 0x000, 0x011,
+ 0x320, 0x01a, 0x2c7, 0x01e, 0x2c7, 0x01a, 0x0a1, 0x151,
+ 0x01a, 0x003, 0x006, 0x2a9, 0x20b, 0x000, 0x3fa, 0x045,
+ 0x054, 0x287, 0x384, 0x0ff, 0x111, 0x020, 0x1b9, 0x000,
+ 0x044, 0x1b9, 0x000, 0x362, 0x01b, 0x016, 0x001, 0x007,
+ 0x362, 0x01b, 0x001, 0x009, 0x023, 0x3d7, 0x0fe, 0x019,
+ 0x044, 0x233, 0x000, 0x387, 0x0bb, 0x187, 0x385, 0x031,
+ 0x302, 0x01b, 0x001, 0x014, 0x387, 0x0b9, 0x187, 0x385,
+ 0x031, 0x302, 0x01b, 0x001, 0x00b, 0x387, 0x0b6, 0x187,
+ 0x385, 0x031, 0x302, 0x01b, 0x009, 0x003, 0x080, 0x045,
+ 0x327, 0x06e, 0x044, 0x340, 0x001, 0x0a8, 0x000, 0x342,
+ 0x327, 0x06f, 0x044, 0x340, 0x001, 0x0ac, 0x000, 0x356,
+ 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x00b, 0x009, 0x38d,
+ 0x0c0, 0x01c, 0x041, 0x1bb, 0x3a4, 0x007, 0x001, 0x047,
+ 0x131, 0x001, 0x02f, 0x131, 0x001, 0x017, 0x083, 0x3a7,
+ 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0, 0x044, 0x3f6,
+ 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x081, 0x044, 0x3f6,
+ 0x001, 0x000, 0x03e, 0x082, 0x3a7, 0x3ff, 0x044, 0x3f6,
+ 0x001, 0x084, 0x0a0, 0x044, 0x3f6, 0x001, 0x083, 0x044,
+ 0x3f6, 0x001, 0x081, 0x044, 0x3f6, 0x001, 0x000, 0x029,
+ 0x081, 0x3a7, 0x3ff, 0x044, 0x3f6, 0x001, 0x084, 0x0a0,
+ 0x044, 0x3f6, 0x001, 0x083, 0x044, 0x3f6, 0x001, 0x082,
+ 0x044, 0x3f6, 0x001, 0x000, 0x014, 0x081, 0x3a7, 0x3ff,
+ 0x044, 0x3f6, 0x001, 0x084, 0x044, 0x3f6, 0x001, 0x083,
+ 0x044, 0x3f6, 0x001, 0x082, 0x044, 0x3f6, 0x001, 0x0c6,
+ 0x019, 0x0c0, 0x01b, 0x044, 0x222, 0x000, 0x0c5, 0x019,
+ 0x387, 0x0ff, 0x3a7, 0x3ff, 0x322, 0x04c, 0x303, 0x04d,
+ 0x2c7, 0x01b, 0x247, 0x01c, 0x044, 0x222, 0x000, 0x0c0,
+ 0x019, 0x0c1, 0x01b, 0x044, 0x222, 0x000, 0x397, 0x3ff,
+ 0x044, 0x205, 0x000, 0x044, 0x233, 0x000, 0x3c4, 0x001,
+ 0x01b, 0x009, 0x3f5, 0x0c6, 0x019, 0x044, 0x233, 0x000,
+ 0x307, 0x01b, 0x043, 0x2a7, 0x003, 0x016, 0x0c0, 0x01a,
+ 0x397, 0x044, 0x04c, 0x322, 0x01b, 0x00b, 0x00b, 0x272,
+ 0x04b, 0x001, 0x007, 0x101, 0x141, 0x01a, 0x000, 0x3f4,
+ 0x307, 0x01a, 0x305, 0x033, 0x055, 0x045, 0x247, 0x019,
+ 0x2c7, 0x01b, 0x020, 0x222, 0x000, 0x090, 0x020, 0x362,
+ 0x002, 0x327, 0x06f, 0x287, 0x384, 0x0ff, 0x262, 0x0df,
+ 0x00b, 0x00a, 0x262, 0x0e7, 0x003, 0x006, 0x247, 0x018,
+ 0x000, 0x00a, 0x044, 0x2f8, 0x001, 0x362, 0x01b, 0x001,
+ 0x009, 0x057, 0x0c7, 0x019, 0x0d1, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0d0, 0x01b, 0x044, 0x222, 0x000,
+ 0x0c0, 0x019, 0x0c0, 0x01b, 0x347, 0x034, 0x01c, 0x044,
+ 0x222, 0x000, 0x0ca, 0x019, 0x081, 0x189, 0x247, 0x012,
+ 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x133, 0x347, 0x013,
+ 0x01b, 0x347, 0x013, 0x01c, 0x044, 0x222, 0x000, 0x132,
+ 0x007, 0x3f6, 0x0c0, 0x019, 0x0c0, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0cb, 0x019, 0x0cc, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c7, 0x019, 0x0c2, 0x01b, 0x0c0, 0x01c,
+ 0x044, 0x222, 0x000, 0x0c8, 0x019, 0x0c0, 0x01b, 0x044,
+ 0x222, 0x000, 0x0c1, 0x008, 0x081, 0x000, 0x007, 0x387,
+ 0x3ff, 0x3c7, 0x0ff, 0x008, 0x055, 0x0b0, 0x020, 0x296,
+ 0x001, 0x0c1, 0x019, 0x0c0, 0x01b, 0x0c1, 0x01c, 0x1cc,
+ 0x01c, 0x044, 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x054,
+ 0x01c, 0x044, 0x222, 0x000, 0x345, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c0, 0x019, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x01d, 0x102, 0x050, 0x282, 0x00f,
+ 0x08f, 0x307, 0x01d, 0x104, 0x1a0, 0x280, 0x051, 0x1a7,
+ 0x101, 0x050, 0x285, 0x300, 0x01d, 0x051, 0x1a7, 0x2c7,
+ 0x01d, 0x101, 0x051, 0x2c5, 0x01d, 0x101, 0x051, 0x1a7,
+ 0x2c7, 0x01e, 0x101, 0x051, 0x2c5, 0x01e, 0x347, 0x034,
+ 0x01c, 0x227, 0x044, 0x222, 0x000, 0x287, 0x044, 0x11b,
+ 0x002, 0x044, 0x11b, 0x002, 0x347, 0x01d, 0x01b, 0x347,
+ 0x034, 0x01c, 0x044, 0x222, 0x000, 0x327, 0x01e, 0x001,
+ 0x057, 0x0ca, 0x019, 0x101, 0x052, 0x01b, 0x101, 0x051,
+ 0x041, 0x1a9, 0x2c5, 0x01b, 0x043, 0x1b5, 0x2c7, 0x01c,
+ 0x101, 0x051, 0x1a3, 0x325, 0x037, 0x2c5, 0x01c, 0x044,
+ 0x222, 0x000, 0x153, 0x01e, 0x007, 0x3e7, 0x0c0, 0x019,
+ 0x0c0, 0x01c, 0x044, 0x222, 0x000, 0x0ca, 0x01b, 0x0c5,
+ 0x019, 0x090, 0x187, 0x395, 0x03a, 0x247, 0x01c, 0x044,
+ 0x222, 0x000, 0x0c2, 0x019, 0x3c7, 0x020, 0x01b, 0x0c0,
+ 0x01c, 0x044, 0x222, 0x000, 0x347, 0x033, 0x01c, 0x044,
+ 0x222, 0x000, 0x045, 0x101, 0x051, 0x1a7, 0x2c7, 0x01b,
+ 0x101, 0x051, 0x2c5, 0x01b, 0x101, 0x051, 0x325, 0x037,
+ 0x2c7, 0x01c, 0x044, 0x222, 0x000, 0x045, 0x0c5, 0x01b,
+ 0x000, 0x3cf, 0x347, 0x000, 0x022, 0x347, 0x001, 0x023,
+ 0x0c0, 0x008, 0x0c0, 0x009, 0x307, 0x022, 0x384, 0x03f,
+ 0x229, 0x220, 0x3b0, 0x14b, 0x287, 0x199, 0x390, 0x002,
+ 0x041, 0x040, 0x045, 0x020, 0x21a, 0x002, 0x020, 0x223,
+ 0x002, 0x020, 0x22c, 0x002, 0x020, 0x351, 0x002, 0x020,
+ 0x281, 0x002, 0x020, 0x304, 0x002, 0x020, 0x309, 0x002,
+ 0x020, 0x30e, 0x002, 0x020, 0x313, 0x002, 0x020, 0x318,
+ 0x002, 0x020, 0x31d, 0x002, 0x020, 0x322, 0x002, 0x020,
+ 0x327, 0x002, 0x020, 0x32c, 0x002, 0x020, 0x331, 0x002,
+ 0x020, 0x336, 0x002, 0x020, 0x339, 0x002, 0x020, 0x33f,
+ 0x002, 0x020, 0x344, 0x002, 0x020, 0x348, 0x002, 0x020,
+ 0x354, 0x002, 0x020, 0x358, 0x002, 0x020, 0x361, 0x002,
+ 0x020, 0x36a, 0x002, 0x020, 0x36d, 0x002, 0x020, 0x370,
+ 0x002, 0x020, 0x2f1, 0x002, 0x020, 0x373, 0x002, 0x020,
+ 0x384, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x38e, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x3fd, 0x001, 0x020, 0x101,
+ 0x001, 0x020, 0x197, 0x001, 0x020, 0x3ba, 0x000, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215,
+ 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002, 0x020,
+ 0x215, 0x002, 0x020, 0x215, 0x002, 0x020, 0x215, 0x002,
+ 0x020, 0x215, 0x002, 0x0c1, 0x008, 0x345, 0x033, 0x008,
+ 0x344, 0x035, 0x008, 0x04e, 0x047, 0x3c7, 0x3ff, 0x008,
+ 0x000, 0x3f8, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7,
+ 0x009, 0x000, 0x3ea, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001,
+ 0x2c7, 0x009, 0x000, 0x3e1, 0x080, 0x3b7, 0x003, 0x1a9,
+ 0x3b5, 0x105, 0x2c7, 0x023, 0x0c0, 0x028, 0x0c0, 0x029,
+ 0x052, 0x022, 0x044, 0x2d6, 0x002, 0x001, 0x005, 0x101,
+ 0x000, 0x3f8, 0x101, 0x052, 0x022, 0x346, 0x028, 0x022,
+ 0x3c4, 0x0ff, 0x022, 0x362, 0x022, 0x0ef, 0x009, 0x02e,
+ 0x101, 0x052, 0x022, 0x327, 0x028, 0x1b7, 0x2c6, 0x022,
+ 0x362, 0x022, 0x0be, 0x009, 0x021, 0x101, 0x052, 0x022,
+ 0x346, 0x029, 0x022, 0x3c4, 0x0ff, 0x022, 0x362, 0x022,
+ 0x0ad, 0x009, 0x013, 0x101, 0x052, 0x022, 0x327, 0x029,
+ 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0de, 0x009, 0x006,
+ 0x0c1, 0x009, 0x000, 0x391, 0x3c7, 0x3ff, 0x009, 0x000,
+ 0x38c, 0x081, 0x189, 0x247, 0x012, 0x0c0, 0x028, 0x0c0,
+ 0x029, 0x327, 0x013, 0x1a7, 0x325, 0x013, 0x2c7, 0x023,
+ 0x247, 0x012, 0x080, 0x347, 0x013, 0x022, 0x044, 0x2d6,
+ 0x002, 0x001, 0x005, 0x101, 0x000, 0x3f7, 0x347, 0x013,
+ 0x022, 0x346, 0x028, 0x022, 0x3c4, 0x0ff, 0x022, 0x362,
+ 0x022, 0x0ef, 0x009, 0x3d2, 0x347, 0x013, 0x022, 0x327,
+ 0x028, 0x1b7, 0x2c6, 0x022, 0x362, 0x022, 0x0be, 0x009,
+ 0x3c5, 0x347, 0x013, 0x022, 0x346, 0x029, 0x022, 0x3c4,
+ 0x0ff, 0x022, 0x362, 0x022, 0x0ad, 0x009, 0x3b7, 0x347,
+ 0x013, 0x022, 0x327, 0x029, 0x1b7, 0x2c6, 0x022, 0x362,
+ 0x022, 0x0de, 0x009, 0x3aa, 0x000, 0x3a4, 0x340, 0x022,
+ 0x028, 0x3c1, 0x000, 0x029, 0x3c6, 0x0d8, 0x028, 0x349,
+ 0x028, 0x028, 0x34b, 0x029, 0x029, 0x00a, 0x008, 0x3c0,
+ 0x0e5, 0x028, 0x3c1, 0x000, 0x029, 0x327, 0x023, 0x222,
+ 0x045, 0x397, 0x003, 0x189, 0x395, 0x105, 0x105, 0x051,
+ 0x1a7, 0x2c7, 0x023, 0x101, 0x051, 0x2c5, 0x023, 0x111,
+ 0x240, 0x023, 0x000, 0x332, 0x347, 0x023, 0x026, 0x000,
+ 0x304, 0x347, 0x023, 0x027, 0x000, 0x2ff, 0x347, 0x023,
+ 0x028, 0x000, 0x2fa, 0x347, 0x023, 0x029, 0x000, 0x2f5,
+ 0x347, 0x023, 0x02a, 0x000, 0x2f0, 0x347, 0x023, 0x02b,
+ 0x000, 0x2eb, 0x347, 0x028, 0x009, 0x000, 0x2e6, 0x347,
+ 0x029, 0x009, 0x000, 0x2e1, 0x347, 0x02a, 0x009, 0x000,
+ 0x2dc, 0x347, 0x02b, 0x009, 0x000, 0x2d7, 0x347, 0x023,
+ 0x02d, 0x055, 0x347, 0x023, 0x02c, 0x000, 0x2ce, 0x347,
+ 0x023, 0x021, 0x000, 0x2c9, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x327, 0x023, 0x04d, 0x000,
+ 0x2bc, 0x347, 0x023, 0x021, 0x307, 0x021, 0x000, 0x006,
+ 0x307, 0x021, 0x141, 0x021, 0x04c, 0x2c7, 0x009, 0x000,
+ 0x2ac, 0x08c, 0x347, 0x023, 0x06f, 0x055, 0x247, 0x02c,
+ 0x000, 0x2ab, 0x08d, 0x000, 0x3f7, 0x08e, 0x000, 0x3f4,
+ 0x08f, 0x000, 0x3f1, 0x307, 0x022, 0x183, 0x19b, 0x262,
+ 0x009, 0x00b, 0x007, 0x347, 0x037, 0x008, 0x000, 0x292,
+ 0x390, 0x044, 0x000, 0x3ca, 0x397, 0x003, 0x189, 0x395,
+ 0x105, 0x105, 0x247, 0x009, 0x000, 0x27f, 0x000, 0x27d,
+ 0x364, 0x004, 0x002, 0x009, 0x090, 0x364, 0x004, 0x008,
+ 0x009, 0x165, 0x364, 0x004, 0x004, 0x009, 0x08e, 0x0c0,
+ 0x007, 0x3d5, 0x080, 0x00e, 0x347, 0x006, 0x062, 0x0c0,
+ 0x064, 0x307, 0x062, 0x197, 0x384, 0x00f, 0x229, 0x220,
+ 0x3b0, 0x3ca, 0x287, 0x199, 0x390, 0x002, 0x041, 0x040,
+ 0x045, 0x345, 0x033, 0x064, 0x347, 0x064, 0x007, 0x344,
+ 0x035, 0x007, 0x3d4, 0x37f, 0x00e, 0x3c5, 0x020, 0x00e,
+ 0x04e, 0x047, 0x020, 0x3ff, 0x002, 0x020, 0x00a, 0x003,
+ 0x020, 0x011, 0x003, 0x020, 0x01a, 0x003, 0x020, 0x01d,
+ 0x003, 0x020, 0x020, 0x003, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002,
+ 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa,
+ 0x002, 0x020, 0x3fa, 0x002, 0x020, 0x3fa, 0x002, 0x020,
+ 0x3fa, 0x002, 0x345, 0x037, 0x064, 0x000, 0x3bf, 0x3b7,
+ 0x010, 0x1a7, 0x3a5, 0x013, 0x324, 0x038, 0x2c7, 0x064,
+ 0x000, 0x3b1, 0x3b7, 0x000, 0x1a7, 0x3a5, 0x001, 0x000,
+ 0x3f5, 0x088, 0x055, 0x247, 0x02c, 0x347, 0x062, 0x06e,
+ 0x000, 0x3ad, 0x089, 0x000, 0x3f7, 0x08a, 0x000, 0x3f4,
+ 0x08b, 0x000, 0x3f1, 0x055, 0x0c3, 0x02c, 0x3d5, 0x100,
+ 0x00e, 0x000, 0x39c, 0x347, 0x011, 0x066, 0x3c4, 0x3fc,
+ 0x066, 0x364, 0x065, 0x040, 0x009, 0x0a3, 0x044, 0x085,
+ 0x003, 0x001, 0x06f, 0x044, 0x06b, 0x003, 0x001, 0x005,
+ 0x055, 0x000, 0x384, 0x044, 0x074, 0x003, 0x3c4, 0x37f,
+ 0x065, 0x1b5, 0x3a4, 0x038, 0x3c4, 0x3c7, 0x065, 0x2c5,
+ 0x065, 0x1b2, 0x2e2, 0x007, 0x001, 0x047, 0x2e2, 0x006,
+ 0x001, 0x043, 0x2e2, 0x002, 0x001, 0x047, 0x2e2, 0x001,
+ 0x001, 0x031, 0x3d7, 0x07f, 0x064, 0x0a6, 0x044, 0x0bf,
+ 0x003, 0x000, 0x35c, 0x364, 0x010, 0x002, 0x009, 0x005,
+ 0x364, 0x011, 0x001, 0x045, 0x3d5, 0x040, 0x00e, 0x3c5,
+ 0x001, 0x010, 0x3c4, 0x3bf, 0x065, 0x327, 0x066, 0x00b,
+ 0x005, 0x3c5, 0x040, 0x065, 0x045, 0x327, 0x004, 0x307,
+ 0x066, 0x198, 0x384, 0x038, 0x1b8, 0x3a4, 0x038, 0x222,
+ 0x045, 0x3b7, 0x010, 0x1a7, 0x3a5, 0x013, 0x2c7, 0x064,
+ 0x0a7, 0x000, 0x3cd, 0x347, 0x066, 0x064, 0x044, 0x0cd,
+ 0x003, 0x000, 0x324, 0x055, 0x0c4, 0x02c, 0x000, 0x31f,
+ 0x044, 0x074, 0x003, 0x3c5, 0x080, 0x065, 0x3c4, 0x3f8,
+ 0x065, 0x3a4, 0x1fc, 0x081, 0x245, 0x065, 0x1b1, 0x390,
+ 0x066, 0x04d, 0x3c5, 0x100, 0x065, 0x000, 0x308, 0x3c4,
+ 0x07f, 0x064, 0x1c1, 0x064, 0x307, 0x066, 0x198, 0x384,
+ 0x038, 0x285, 0x188, 0x245, 0x064, 0x3c4, 0x003, 0x010,
+ 0x345, 0x064, 0x010, 0x3c5, 0x002, 0x010, 0x045, 0x044,
+ 0x085, 0x003, 0x009, 0x009, 0x307, 0x065, 0x364, 0x065,
+ 0x080, 0x009, 0x00c, 0x044, 0x06b, 0x003, 0x009, 0x35a,
+ 0x044, 0x074, 0x003, 0x000, 0x3b0, 0x044, 0x074, 0x003,
+ 0x384, 0x007, 0x262, 0x007, 0x001, 0x2d1, 0x141, 0x065,
+ 0x101, 0x324, 0x035, 0x000, 0x3bb, 0x3d5, 0x010, 0x00e,
+ 0x055, 0x0d3, 0x02c, 0x000, 0x2c2, 0x04e, 0x062, 0x0b1,
+ 0x0e7, 0x079
+
+}; /* end fm10000_sbus_master_code_prd */
+
+/* Spico SerDes code (production version 2)
+ * source file: serdes.0x2055/serdes.0x2055_0045.rom
+ * KR support
+ */
+const uint32_t fm10000_serdes_spico_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_spico_code_size_prd2 = 12167;
+
+const uint16_t fm10000_serdes_spico_code_prd2[] = {
+ 0x035, 0x006, 0x00a, 0x000, 0x1a7, 0x006, 0x020, 0x055,
+ 0x000, 0x045, 0x3c7, 0x03f, 0x000, 0x0c0, 0x200, 0x307,
+ 0x200, 0x304, 0x02b, 0x009, 0x00b, 0x0a0, 0x397, 0x3bb,
+ 0x04d, 0x111, 0x272, 0x1ff, 0x009, 0x3fc, 0x3c8, 0x000,
+ 0x21e, 0x3ce, 0x3ff, 0x21f, 0x348, 0x21f, 0x21d, 0x34e,
+ 0x21d, 0x221, 0x34e, 0x221, 0x21c, 0x34e, 0x21c, 0x220,
+ 0x3c7, 0x3fd, 0x23d, 0x3c7, 0x020, 0x23e, 0x3c7, 0x077,
+ 0x22d, 0x0c0, 0x202, 0x044, 0x36d, 0x003, 0x044, 0x360,
+ 0x002, 0x3d7, 0x303, 0x030, 0x347, 0x032, 0x236, 0x3b7,
+ 0x213, 0x090, 0x044, 0x1a9, 0x000, 0x3b7, 0x204, 0x387,
+ 0x07c, 0x044, 0x1a9, 0x000, 0x0a4, 0x397, 0x27c, 0x044,
+ 0x1a9, 0x000, 0x3d5, 0x230, 0x232, 0x345, 0x21d, 0x232,
+ 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000, 0x3b7,
+ 0x080, 0x044, 0x192, 0x000, 0x385, 0x00c, 0x384, 0x3cf,
+ 0x044, 0x1bd, 0x000, 0x3c5, 0x040, 0x021, 0x3b7, 0x020,
+ 0x387, 0x021, 0x189, 0x044, 0x1a9, 0x000, 0x0c0, 0x008,
+ 0x0c0, 0x005, 0x0c0, 0x00b, 0x0c0, 0x000, 0x3c5, 0x020,
+ 0x027, 0x055, 0x307, 0x221, 0x304, 0x233, 0x009, 0x012,
+ 0x374, 0x027, 0x200, 0x001, 0x00d, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x009, 0x006, 0x3b8, 0x200, 0x2c4, 0x027,
+ 0x054, 0x364, 0x200, 0x004, 0x029, 0x3de, 0x003, 0x364,
+ 0x33b, 0x010, 0x029, 0x3de, 0x003, 0x054, 0x364, 0x200,
+ 0x040, 0x029, 0x034, 0x008, 0x364, 0x2cf, 0x040, 0x009,
+ 0x01c, 0x307, 0x026, 0x384, 0x003, 0x266, 0x003, 0x009,
+ 0x014, 0x362, 0x26b, 0x0e0, 0x009, 0x00f, 0x364, 0x234,
+ 0x002, 0x001, 0x00a, 0x3c5, 0x060, 0x2cf, 0x0c0, 0x26b,
+ 0x3c5, 0x040, 0x200, 0x054, 0x364, 0x200, 0x100, 0x029,
+ 0x22f, 0x003, 0x054, 0x044, 0x141, 0x001, 0x055, 0x364,
+ 0x23f, 0x003, 0x001, 0x012, 0x044, 0x34c, 0x001, 0x029,
+ 0x0af, 0x006, 0x044, 0x340, 0x001, 0x227, 0x326, 0x249,
+ 0x1ba, 0x029, 0x0af, 0x006, 0x3c4, 0x3df, 0x000, 0x364,
+ 0x33c, 0x020, 0x001, 0x017, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x055, 0x364, 0x200, 0x080, 0x001, 0x00d, 0x3c4,
+ 0x3fd, 0x26f, 0x3c4, 0x3fe, 0x200, 0x366, 0x200, 0x080,
+ 0x001, 0x00f, 0x140, 0x200, 0x009, 0x365, 0x364, 0x23f,
+ 0x003, 0x009, 0x360, 0x04f, 0x04e, 0x000, 0x35c, 0x3c5,
+ 0x030, 0x000, 0x347, 0x244, 0x054, 0x374, 0x000, 0x2c0,
+ 0x009, 0x34c, 0x387, 0x0c1, 0x187, 0x245, 0x054, 0x04f,
+ 0x055, 0x141, 0x26c, 0x347, 0x245, 0x200, 0x347, 0x245,
+ 0x26d, 0x0c0, 0x245, 0x364, 0x026, 0x002, 0x001, 0x336,
+ 0x364, 0x020, 0x005, 0x009, 0x007, 0x0c2, 0x26f, 0x044,
+ 0x1b8, 0x001, 0x055, 0x364, 0x020, 0x005, 0x009, 0x326,
+ 0x362, 0x26b, 0x0e0, 0x009, 0x321, 0x364, 0x234, 0x002,
+ 0x001, 0x31c, 0x3c5, 0x060, 0x2cf, 0x3c5, 0x040, 0x200,
+ 0x000, 0x314, 0x044, 0x290, 0x001, 0x3c5, 0x080, 0x200,
+ 0x347, 0x2cf, 0x26b, 0x3c4, 0x0e0, 0x26b, 0x3c4, 0x3bf,
+ 0x2cf, 0x045, 0x387, 0x080, 0x020, 0x2f8, 0x001, 0x387,
+ 0x02f, 0x111, 0x009, 0x3ff, 0x045, 0x054, 0x111, 0x009,
+ 0x3fe, 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x032, 0x054,
+ 0x045, 0x044, 0x1a0, 0x000, 0x307, 0x033, 0x054, 0x045,
+ 0x055, 0x2c7, 0x030, 0x364, 0x035, 0x001, 0x009, 0x3fd,
+ 0x045, 0x044, 0x1a0, 0x000, 0x305, 0x033, 0x000, 0x012,
+ 0x044, 0x1a0, 0x000, 0x306, 0x033, 0x000, 0x00b, 0x208,
+ 0x044, 0x1a0, 0x000, 0x304, 0x033, 0x055, 0x2c7, 0x030,
+ 0x247, 0x031, 0x345, 0x21d, 0x030, 0x140, 0x035, 0x009,
+ 0x3fe, 0x346, 0x21d, 0x030, 0x054, 0x045, 0x040, 0x140,
+ 0x036, 0x00b, 0x020, 0x044, 0x206, 0x000, 0x344, 0x21f,
+ 0x036, 0x3c4, 0x3ef, 0x036, 0x09f, 0x044, 0x189, 0x000,
+ 0x000, 0x010, 0x040, 0x140, 0x036, 0x003, 0x00c, 0x044,
+ 0x206, 0x000, 0x345, 0x21d, 0x036, 0x3c4, 0x3ef, 0x036,
+ 0x054, 0x327, 0x22b, 0x3a4, 0x003, 0x133, 0x009, 0x00e,
+ 0x0a1, 0x1aa, 0x2a8, 0x2c4, 0x021, 0x08f, 0x044, 0x189,
+ 0x000, 0x2a8, 0x2c5, 0x021, 0x042, 0x045, 0x055, 0x088,
+ 0x3c5, 0x010, 0x036, 0x000, 0x37e, 0x3c4, 0x3fe, 0x026,
+ 0x327, 0x22c, 0x3a4, 0x007, 0x2e6, 0x001, 0x001, 0x01b,
+ 0x044, 0x1e2, 0x000, 0x364, 0x234, 0x001, 0x009, 0x013,
+ 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x3b6, 0x003,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x066, 0x004,
+ 0x054, 0x045, 0x364, 0x233, 0x002, 0x009, 0x00d, 0x347,
+ 0x271, 0x270, 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x044,
+ 0x02b, 0x008, 0x374, 0x232, 0x200, 0x001, 0x0a5, 0x3d7,
+ 0x200, 0x27b, 0x044, 0x29c, 0x001, 0x044, 0x2f6, 0x002,
+ 0x009, 0x005, 0x044, 0x028, 0x008, 0x3b7, 0x221, 0x364,
+ 0x22b, 0x003, 0x009, 0x007, 0x083, 0x18b, 0x044, 0x1a9,
+ 0x000, 0x398, 0x200, 0x244, 0x232, 0x3b7, 0x221, 0x091,
+ 0x189, 0x385, 0x128, 0x044, 0x1a9, 0x000, 0x083, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x220, 0x044, 0x192, 0x000, 0x197,
+ 0x264, 0x007, 0x009, 0x005, 0x091, 0x000, 0x003, 0x081,
+ 0x3b7, 0x221, 0x189, 0x044, 0x1b7, 0x000, 0x364, 0x246,
+ 0x010, 0x009, 0x002, 0x3b7, 0x221, 0x081, 0x18d, 0x044,
+ 0x1a9, 0x000, 0x081, 0x189, 0x044, 0x1a9, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x364, 0x22b, 0x003, 0x009,
+ 0x009, 0x3b7, 0x221, 0x081, 0x18b, 0x044, 0x1b7, 0x000,
+ 0x364, 0x02b, 0x020, 0x009, 0x020, 0x0a3, 0x364, 0x22b,
+ 0x003, 0x001, 0x006, 0x044, 0x025, 0x008, 0x0a0, 0x307,
+ 0x251, 0x197, 0x384, 0x00f, 0x220, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x327, 0x251, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x044, 0x1ce, 0x000, 0x364, 0x232,
+ 0x100, 0x001, 0x005, 0x044, 0x0a9, 0x001, 0x3c4, 0x3f0,
+ 0x22d, 0x3c5, 0x001, 0x026, 0x364, 0x026, 0x002, 0x001,
+ 0x1c5, 0x044, 0x283, 0x001, 0x3b7, 0x300, 0x387, 0x020,
+ 0x000, 0x2c1, 0x307, 0x036, 0x00b, 0x3ea, 0x044, 0x1ce,
+ 0x000, 0x000, 0x3e5, 0x387, 0x3ff, 0x327, 0x22c, 0x3a4,
+ 0x007, 0x131, 0x001, 0x01a, 0x3d5, 0x200, 0x232, 0x000,
+ 0x006, 0x080, 0x131, 0x001, 0x011, 0x3b7, 0x205, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x222, 0x044, 0x1bd, 0x000, 0x3b7,
+ 0x251, 0x044, 0x1bd, 0x000, 0x3b7, 0x214, 0x100, 0x009,
+ 0x00a, 0x044, 0x1bd, 0x000, 0x3c4, 0x37b, 0x021, 0x000,
+ 0x29e, 0x3c5, 0x084, 0x021, 0x000, 0x299, 0x364, 0x233,
+ 0x002, 0x009, 0x00d, 0x347, 0x272, 0x270, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x02e, 0x008, 0x307, 0x22d,
+ 0x384, 0x070, 0x386, 0x010, 0x001, 0x04a, 0x364, 0x02b,
+ 0x020, 0x009, 0x045, 0x3b7, 0x071, 0x387, 0x128, 0x044,
+ 0x1a9, 0x000, 0x387, 0x368, 0x044, 0x1b8, 0x000, 0x387,
+ 0x2d7, 0x044, 0x1b8, 0x000, 0x0a1, 0x307, 0x22b, 0x384,
+ 0x030, 0x001, 0x003, 0x0ac, 0x307, 0x256, 0x384, 0x01f,
+ 0x220, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x080, 0x327,
+ 0x22b, 0x3a4, 0x030, 0x3a2, 0x030, 0x009, 0x00e, 0x0a1,
+ 0x1ab, 0x2a8, 0x2c4, 0x024, 0x08f, 0x044, 0x189, 0x000,
+ 0x2a8, 0x2c5, 0x024, 0x327, 0x256, 0x1b7, 0x3a4, 0x07f,
+ 0x220, 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x044, 0x2f6,
+ 0x002, 0x009, 0x007, 0x0c0, 0x27b, 0x044, 0x29c, 0x001,
+ 0x044, 0x0e6, 0x001, 0x3c4, 0x30f, 0x22d, 0x044, 0x2f6,
+ 0x002, 0x009, 0x005, 0x044, 0x031, 0x008, 0x345, 0x21d,
+ 0x01c, 0x307, 0x21d, 0x304, 0x233, 0x001, 0x007, 0x3a7,
+ 0x071, 0x044, 0x1a9, 0x000, 0x3c5, 0x002, 0x026, 0x044,
+ 0x141, 0x001, 0x020, 0x283, 0x001, 0x055, 0x3c4, 0x3ef,
+ 0x027, 0x3c4, 0x3fd, 0x026, 0x364, 0x26f, 0x002, 0x001,
+ 0x008, 0x3c4, 0x3fd, 0x26f, 0x044, 0x290, 0x001, 0x344,
+ 0x21f, 0x01c, 0x3c4, 0x3bf, 0x200, 0x3c4, 0x39f, 0x2cf,
+ 0x0c0, 0x2d8, 0x0c0, 0x26b, 0x364, 0x234, 0x002, 0x009,
+ 0x012, 0x364, 0x33b, 0x007, 0x001, 0x00d, 0x044, 0x066,
+ 0x004, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x3b6,
+ 0x003, 0x054, 0x0a0, 0x081, 0x189, 0x020, 0x1a9, 0x000,
+ 0x044, 0x3fd, 0x000, 0x387, 0x3ff, 0x044, 0x02b, 0x001,
+ 0x3a7, 0x022, 0x020, 0x1bd, 0x000, 0x0a0, 0x081, 0x189,
+ 0x020, 0x1a9, 0x000, 0x080, 0x044, 0x02b, 0x001, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x364, 0x024,
+ 0x002, 0x001, 0x006, 0x399, 0x305, 0x000, 0x004, 0x397,
+ 0x200, 0x044, 0x322, 0x001, 0x3a7, 0x022, 0x044, 0x1bd,
+ 0x000, 0x364, 0x232, 0x002, 0x001, 0x005, 0x044, 0x382,
+ 0x001, 0x000, 0x0c9, 0x3a7, 0x061, 0x044, 0x1bd, 0x000,
+ 0x0a5, 0x044, 0x1bd, 0x000, 0x3a7, 0x072, 0x044, 0x1bd,
+ 0x000, 0x3a7, 0x0f1, 0x044, 0x1bd, 0x000, 0x385, 0x004,
+ 0x0b1, 0x020, 0x1bd, 0x000, 0x364, 0x22c, 0x002, 0x009,
+ 0x2ac, 0x364, 0x234, 0x001, 0x001, 0x058, 0x020, 0x232,
+ 0x000, 0x3c4, 0x3fe, 0x027, 0x364, 0x234, 0x002, 0x001,
+ 0x04d, 0x000, 0x2cd, 0x3b7, 0x211, 0x044, 0x199, 0x000,
+ 0x384, 0x00f, 0x140, 0x239, 0x00b, 0x004, 0x385, 0x010,
+ 0x140, 0x237, 0x00b, 0x004, 0x385, 0x020, 0x044, 0x1bd,
+ 0x000, 0x307, 0x239, 0x00b, 0x004, 0x208, 0x101, 0x384,
+ 0x01f, 0x183, 0x327, 0x237, 0x00b, 0x004, 0x2a8, 0x121,
+ 0x3a4, 0x00f, 0x285, 0x184, 0x3c4, 0x01f, 0x238, 0x362,
+ 0x238, 0x017, 0x00b, 0x005, 0x0a1, 0x1ad, 0x285, 0x305,
+ 0x238, 0x32a, 0x238, 0x3a4, 0x008, 0x285, 0x3b7, 0x210,
+ 0x044, 0x1bd, 0x000, 0x347, 0x237, 0x23a, 0x347, 0x238,
+ 0x23c, 0x347, 0x239, 0x23b, 0x045, 0x345, 0x21c, 0x021,
+ 0x045, 0x308, 0x21c, 0x244, 0x021, 0x0c0, 0x240, 0x307,
+ 0x022, 0x384, 0x001, 0x245, 0x240, 0x142, 0x240, 0x364,
+ 0x240, 0x100, 0x009, 0x3eb, 0x0a1, 0x044, 0x0d9, 0x001,
+ 0x307, 0x240, 0x327, 0x022, 0x286, 0x384, 0x001, 0x001,
+ 0x3ee, 0x0a2, 0x307, 0x22b, 0x384, 0x003, 0x001, 0x008,
+ 0x0aa, 0x262, 0x003, 0x001, 0x003, 0x0a5, 0x289, 0x240,
+ 0x240, 0x387, 0x084, 0x020, 0x2f8, 0x001, 0x094, 0x020,
+ 0x2f2, 0x001, 0x090, 0x020, 0x2f2, 0x001, 0x0a0, 0x082,
+ 0x18b, 0x304, 0x020, 0x009, 0x3b9, 0x081, 0x189, 0x020,
+ 0x1b7, 0x000, 0x0a1, 0x1a9, 0x364, 0x234, 0x002, 0x001,
+ 0x018, 0x364, 0x22c, 0x020, 0x001, 0x016, 0x364, 0x22c,
+ 0x010, 0x001, 0x009, 0x364, 0x232, 0x080, 0x001, 0x00c,
+ 0x000, 0x007, 0x364, 0x232, 0x040, 0x001, 0x005, 0x2c5,
+ 0x024, 0x045, 0x309, 0x024, 0x003, 0x3fb, 0x2a8, 0x2c4,
+ 0x024, 0x045, 0x3b7, 0x213, 0x364, 0x234, 0x004, 0x009,
+ 0x017, 0x307, 0x233, 0x182, 0x00a, 0x007, 0x3c5, 0x040,
+ 0x021, 0x000, 0x005, 0x3c4, 0x3bf, 0x021, 0x082, 0x044,
+ 0x1b7, 0x000, 0x088, 0x020, 0x1a9, 0x000, 0x082, 0x044,
+ 0x1a9, 0x000, 0x3c4, 0x3bf, 0x021, 0x088, 0x020, 0x1b7,
+ 0x000, 0x055, 0x364, 0x026, 0x002, 0x001, 0x031, 0x364,
+ 0x024, 0x002, 0x009, 0x007, 0x364, 0x025, 0x004, 0x009,
+ 0x027, 0x309, 0x01c, 0x003, 0x004, 0x00a, 0x021, 0x364,
+ 0x2cf, 0x011, 0x009, 0x01c, 0x364, 0x33b, 0x007, 0x001,
+ 0x007, 0x364, 0x33b, 0x002, 0x001, 0x012, 0x3c5, 0x010,
+ 0x027, 0x364, 0x234, 0x040, 0x001, 0x005, 0x3c4, 0x37f,
+ 0x024, 0x3c5, 0x008, 0x025, 0x000, 0x010, 0x364, 0x234,
+ 0x040, 0x001, 0x005, 0x3c5, 0x080, 0x024, 0x3c4, 0x3ef,
+ 0x027, 0x3c5, 0x010, 0x026, 0x054, 0x045, 0x364, 0x026,
+ 0x001, 0x001, 0x008, 0x3c5, 0x004, 0x232, 0x044, 0x20d,
+ 0x000, 0x09f, 0x18a, 0x208, 0x244, 0x243, 0x244, 0x250,
+ 0x327, 0x243, 0x326, 0x250, 0x001, 0x308, 0x347, 0x243,
+ 0x250, 0x087, 0x187, 0x208, 0x244, 0x020, 0x364, 0x026,
+ 0x002, 0x001, 0x2fb, 0x3c5, 0x008, 0x232, 0x020, 0x3b5,
+ 0x000, 0x364, 0x233, 0x008, 0x001, 0x004, 0x307, 0x220,
+ 0x3c5, 0x001, 0x00c, 0x247, 0x01b, 0x083, 0x18d, 0x244,
+ 0x01c, 0x044, 0x1c9, 0x001, 0x0a3, 0x1a8, 0x2c5, 0x01f,
+ 0x045, 0x3c4, 0x0ff, 0x01f, 0x085, 0x044, 0x189, 0x000,
+ 0x3d5, 0x100, 0x01f, 0x3b7, 0x300, 0x397, 0x200, 0x044,
+ 0x1a9, 0x000, 0x3c7, 0x07f, 0x252, 0x055, 0x151, 0x252,
+ 0x001, 0x006, 0x140, 0x01e, 0x00b, 0x3fa, 0x397, 0x200,
+ 0x044, 0x1b7, 0x000, 0x3c4, 0x2ff, 0x01f, 0x045, 0x327,
+ 0x22c, 0x307, 0x22d, 0x2e4, 0x004, 0x009, 0x00d, 0x286,
+ 0x384, 0x007, 0x001, 0x00f, 0x2e4, 0x007, 0x001, 0x032,
+ 0x000, 0x039, 0x384, 0x007, 0x3c4, 0x3f8, 0x22c, 0x245,
+ 0x22c, 0x327, 0x22c, 0x307, 0x22d, 0x2e4, 0x040, 0x009,
+ 0x016, 0x286, 0x264, 0x070, 0x001, 0x018, 0x3a4, 0x070,
+ 0x001, 0x029, 0x1aa, 0x002, 0x02e, 0x044, 0x3b5, 0x000,
+ 0x044, 0x3fd, 0x000, 0x000, 0x009, 0x384, 0x070, 0x3c4,
+ 0x38f, 0x22c, 0x245, 0x22c, 0x347, 0x22c, 0x22d, 0x045,
+ 0x227, 0x044, 0x301, 0x000, 0x044, 0x232, 0x000, 0x000,
+ 0x3d2, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x000,
+ 0x3ca, 0x044, 0x003, 0x001, 0x044, 0x326, 0x000, 0x000,
+ 0x3e5, 0x044, 0x0f2, 0x001, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x000, 0x3da, 0x3a7, 0x080, 0x384, 0x001,
+ 0x001, 0x011, 0x082, 0x044, 0x1a9, 0x000, 0x081, 0x044,
+ 0x1b7, 0x000, 0x3c5, 0x004, 0x024, 0x3c5, 0x008, 0x025,
+ 0x045, 0x366, 0x22e, 0x020, 0x001, 0x010, 0x082, 0x044,
+ 0x1b7, 0x000, 0x081, 0x044, 0x1a9, 0x000, 0x3c4, 0x3fb,
+ 0x024, 0x3c4, 0x3f7, 0x025, 0x045, 0x308, 0x248, 0x19e,
+ 0x020, 0x254, 0x001, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x020, 0x1b6, 0x001,
+ 0x3c5, 0x001, 0x00c, 0x0c0, 0x01b, 0x044, 0x1c9, 0x001,
+ 0x3c4, 0x3fe, 0x00c, 0x045, 0x0a1, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x247, 0x24f, 0x397, 0x3e0, 0x044, 0x1a9,
+ 0x000, 0x0a0, 0x325, 0x27b, 0x387, 0x140, 0x044, 0x1a9,
+ 0x000, 0x044, 0x187, 0x000, 0x084, 0x044, 0x1a9, 0x000,
+ 0x044, 0x187, 0x000, 0x387, 0x040, 0x044, 0x1b7, 0x000,
+ 0x387, 0x106, 0x044, 0x189, 0x000, 0x387, 0x100, 0x044,
+ 0x1b7, 0x000, 0x387, 0x099, 0x044, 0x189, 0x000, 0x0a1,
+ 0x325, 0x27b, 0x307, 0x24f, 0x044, 0x1bd, 0x000, 0x0a0,
+ 0x325, 0x27b, 0x084, 0x044, 0x1b7, 0x000, 0x387, 0x099,
+ 0x044, 0x2f6, 0x002, 0x009, 0x004, 0x399, 0x3ff, 0x020,
+ 0x189, 0x000, 0x041, 0x327, 0x236, 0x3a4, 0x00c, 0x134,
+ 0x043, 0x045, 0x055, 0x3d7, 0x071, 0x030, 0x000, 0x006,
+ 0x055, 0x3d7, 0x221, 0x030, 0x347, 0x034, 0x258, 0x0c0,
+ 0x034, 0x140, 0x035, 0x009, 0x3fe, 0x347, 0x032, 0x031,
+ 0x345, 0x21d, 0x030, 0x246, 0x031, 0x374, 0x030, 0x200,
+ 0x001, 0x007, 0x246, 0x031, 0x044, 0x18c, 0x000, 0x131,
+ 0x009, 0x3f3, 0x346, 0x21d, 0x030, 0x347, 0x258, 0x034,
+ 0x054, 0x045, 0x364, 0x233, 0x040, 0x001, 0x006, 0x3b7,
+ 0x200, 0x2a8, 0x284, 0x045, 0x227, 0x055, 0x040, 0x397,
+ 0x203, 0x300, 0x202, 0x04d, 0x141, 0x202, 0x362, 0x202,
+ 0x018, 0x009, 0x004, 0x0c0, 0x202, 0x042, 0x054, 0x045,
+ 0x055, 0x3c7, 0x0f2, 0x030, 0x364, 0x035, 0x001, 0x009,
+ 0x3fd, 0x307, 0x032, 0x045, 0x0a1, 0x1aa, 0x324, 0x233,
+ 0x045, 0x327, 0x222, 0x3a4, 0x07f, 0x001, 0x02c, 0x055,
+ 0x307, 0x034, 0x0c0, 0x034, 0x040, 0x140, 0x035, 0x009,
+ 0x3fe, 0x3c7, 0x071, 0x030, 0x387, 0x0d7, 0x347, 0x032,
+ 0x031, 0x345, 0x21d, 0x030, 0x141, 0x23d, 0x009, 0x007,
+ 0x153, 0x23d, 0x346, 0x21c, 0x031, 0x246, 0x031, 0x131,
+ 0x009, 0x3f4, 0x346, 0x21d, 0x030, 0x042, 0x247, 0x034,
+ 0x054, 0x045, 0x083, 0x18d, 0x345, 0x21d, 0x233, 0x364,
+ 0x2cf, 0x004, 0x001, 0x005, 0x245, 0x305, 0x045, 0x3a7,
+ 0x071, 0x020, 0x1a9, 0x000, 0x083, 0x18d, 0x364, 0x2cf,
+ 0x004, 0x001, 0x006, 0x208, 0x244, 0x305, 0x045, 0x328,
+ 0x21d, 0x2c4, 0x233, 0x3a7, 0x071, 0x020, 0x1b7, 0x000,
+ 0x120, 0x001, 0x027, 0x041, 0x2f4, 0x200, 0x001, 0x00a,
+ 0x3a7, 0x073, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x009,
+ 0x3a7, 0x073, 0x387, 0x3fd, 0x044, 0x1b8, 0x000, 0x043,
+ 0x2e4, 0x180, 0x001, 0x016, 0x2e4, 0x080, 0x009, 0x017,
+ 0x3c5, 0x002, 0x232, 0x044, 0x382, 0x001, 0x000, 0x012,
+ 0x3c4, 0x3fd, 0x232, 0x044, 0x394, 0x001, 0x000, 0x00a,
+ 0x044, 0x353, 0x001, 0x000, 0x005, 0x081, 0x043, 0x040,
+ 0x020, 0x181, 0x006, 0x3c4, 0x3fe, 0x03a, 0x247, 0x25e,
+ 0x0c5, 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263,
+ 0x3c4, 0x3bf, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03a,
+ 0x3c5, 0x040, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042,
+ 0x100, 0x00b, 0x008, 0x247, 0x03c, 0x042, 0x247, 0x03b,
+ 0x042, 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03a,
+ 0x041, 0x307, 0x25d, 0x040, 0x152, 0x03a, 0x345, 0x263,
+ 0x03a, 0x045, 0x3c4, 0x3fe, 0x03d, 0x247, 0x25f, 0x0c5,
+ 0x263, 0x264, 0x100, 0x001, 0x004, 0x148, 0x263, 0x3c4,
+ 0x37f, 0x00c, 0x384, 0x003, 0x187, 0x247, 0x03d, 0x3c5,
+ 0x080, 0x00c, 0x042, 0x247, 0x25d, 0x043, 0x042, 0x100,
+ 0x00b, 0x008, 0x247, 0x03f, 0x042, 0x247, 0x03e, 0x042,
+ 0x247, 0x039, 0x042, 0x247, 0x038, 0x142, 0x03d, 0x041,
+ 0x307, 0x25d, 0x040, 0x152, 0x03d, 0x345, 0x263, 0x03d,
+ 0x045, 0x3c4, 0x3fe, 0x040, 0x0c1, 0x041, 0x3c4, 0x3cf,
+ 0x00c, 0x151, 0x041, 0x0c0, 0x263, 0x247, 0x262, 0x0c0,
+ 0x040, 0x384, 0x003, 0x227, 0x1a0, 0x285, 0x384, 0x3fd,
+ 0x187, 0x245, 0x040, 0x245, 0x041, 0x245, 0x263, 0x3c5,
+ 0x030, 0x00c, 0x3c7, 0x3ff, 0x039, 0x142, 0x040, 0x152,
+ 0x040, 0x144, 0x040, 0x364, 0x262, 0x100, 0x001, 0x004,
+ 0x148, 0x040, 0x042, 0x043, 0x247, 0x261, 0x2c7, 0x260,
+ 0x044, 0x0de, 0x002, 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x04a,
+ 0x307, 0x262, 0x384, 0x030, 0x187, 0x245, 0x04a, 0x0a1,
+ 0x307, 0x262, 0x384, 0x030, 0x193, 0x262, 0x000, 0x001,
+ 0x008, 0x262, 0x001, 0x001, 0x007, 0x000, 0x006, 0x0a0,
+ 0x000, 0x003, 0x0a2, 0x1ad, 0x2c5, 0x263, 0x0a3, 0x307,
+ 0x262, 0x384, 0x00c, 0x191, 0x262, 0x001, 0x001, 0x008,
+ 0x262, 0x002, 0x001, 0x007, 0x000, 0x006, 0x0a1, 0x000,
+ 0x003, 0x0a2, 0x1a5, 0x2c5, 0x263, 0x374, 0x262, 0x200,
+ 0x001, 0x00e, 0x307, 0x261, 0x327, 0x260, 0x041, 0x040,
+ 0x347, 0x263, 0x041, 0x141, 0x040, 0x045, 0x347, 0x263,
+ 0x041, 0x141, 0x040, 0x020, 0x089, 0x000, 0x042, 0x043,
+ 0x247, 0x25c, 0x042, 0x264, 0x3ff, 0x00b, 0x008, 0x247,
+ 0x043, 0x042, 0x247, 0x042, 0x042, 0x247, 0x039, 0x042,
+ 0x247, 0x038, 0x142, 0x040, 0x152, 0x040, 0x307, 0x25c,
+ 0x041, 0x040, 0x045, 0x055, 0x140, 0x259, 0x001, 0x009,
+ 0x322, 0x259, 0x001, 0x007, 0x0a1, 0x054, 0x045, 0x2c7,
+ 0x259, 0x264, 0x008, 0x009, 0x06e, 0x0c0, 0x012, 0x043,
+ 0x2c7, 0x25a, 0x043, 0x2c7, 0x25b, 0x227, 0x274, 0x200,
+ 0x009, 0x022, 0x3a4, 0x006, 0x2c7, 0x00d, 0x227, 0x3c4,
+ 0x3fd, 0x00c, 0x3c4, 0x3fc, 0x018, 0x384, 0x001, 0x245,
+ 0x018, 0x142, 0x00c, 0x042, 0x247, 0x014, 0x042, 0x247,
+ 0x013, 0x120, 0x00b, 0x008, 0x042, 0x247, 0x011, 0x042,
+ 0x247, 0x010, 0x3c7, 0x050, 0x26e, 0x142, 0x012, 0x141,
+ 0x00d, 0x080, 0x2e4, 0x100, 0x001, 0x003, 0x108, 0x151,
+ 0x26e, 0x001, 0x007, 0x364, 0x012, 0x040, 0x009, 0x3f9,
+ 0x152, 0x012, 0x151, 0x00d, 0x2e4, 0x080, 0x001, 0x003,
+ 0x104, 0x245, 0x012, 0x307, 0x25b, 0x040, 0x307, 0x25a,
+ 0x040, 0x2a9, 0x003, 0x017, 0x151, 0x26e, 0x003, 0x007,
+ 0x364, 0x012, 0x080, 0x009, 0x3f9, 0x364, 0x026, 0x002,
+ 0x001, 0x007, 0x364, 0x025, 0x004, 0x009, 0x007, 0x141,
+ 0x012, 0x0a0, 0x054, 0x045, 0x364, 0x2cf, 0x007, 0x001,
+ 0x3f8, 0x3c4, 0x3fc, 0x2cf, 0x3d5, 0x200, 0x2cf, 0x020,
+ 0x33a, 0x002, 0x247, 0x017, 0x287, 0x00b, 0x006, 0x3a7,
+ 0x3ff, 0x000, 0x003, 0x1bb, 0x384, 0x0ff, 0x264, 0x080,
+ 0x001, 0x004, 0x385, 0x300, 0x380, 0x011, 0x00b, 0x018,
+ 0x041, 0x0a0, 0x131, 0x380, 0x014, 0x003, 0x3fd, 0x041,
+ 0x0bf, 0x1a6, 0x2a8, 0x2c4, 0x018, 0x186, 0x245, 0x018,
+ 0x083, 0x18d, 0x245, 0x017, 0x042, 0x043, 0x103, 0x121,
+ 0x18a, 0x1a7, 0x285, 0x247, 0x01a, 0x387, 0x3ff, 0x247,
+ 0x019, 0x18b, 0x245, 0x018, 0x307, 0x017, 0x19d, 0x262,
+ 0x003, 0x001, 0x00e, 0x262, 0x001, 0x001, 0x013, 0x262,
+ 0x002, 0x009, 0x037, 0x3a7, 0x0aa, 0x000, 0x00d, 0x0c1,
+ 0x019, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x000, 0x039,
+ 0x3a7, 0x055, 0x041, 0x3a7, 0x071, 0x044, 0x192, 0x000,
+ 0x00b, 0x010, 0x040, 0x3a7, 0x0f2, 0x044, 0x192, 0x000,
+ 0x198, 0x043, 0x3a4, 0x001, 0x009, 0x004, 0x386, 0x001,
+ 0x043, 0x264, 0x001, 0x001, 0x004, 0x3a6, 0x0ff, 0x287,
+ 0x187, 0x285, 0x247, 0x019, 0x18b, 0x103, 0x244, 0x018,
+ 0x364, 0x24d, 0x030, 0x009, 0x00c, 0x08f, 0x18b, 0x208,
+ 0x244, 0x018, 0x397, 0x1fe, 0x182, 0x244, 0x019, 0x327,
+ 0x017, 0x1ba, 0x3a4, 0x007, 0x001, 0x019, 0x2e2, 0x003,
+ 0x001, 0x015, 0x041, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x043, 0x384, 0x008, 0x001, 0x00a, 0x2e2, 0x003, 0x003,
+ 0x004, 0x3a6, 0x002, 0x3a6, 0x001, 0x387, 0x002, 0x189,
+ 0x395, 0x23f, 0x280, 0x050, 0x245, 0x01a, 0x045, 0x0ff,
+ 0x00c, 0x030, 0x03c, 0x00f, 0x0f0, 0x004, 0x020, 0x307,
+ 0x26f, 0x384, 0x003, 0x229, 0x220, 0x3b0, 0x04b, 0x287,
+ 0x199, 0x390, 0x008, 0x041, 0x040, 0x045, 0x055, 0x329,
+ 0x01f, 0x00b, 0x08b, 0x364, 0x234, 0x080, 0x001, 0x00e,
+ 0x364, 0x024, 0x002, 0x009, 0x009, 0x364, 0x025, 0x004,
+ 0x001, 0x004, 0x000, 0x012, 0x327, 0x01d, 0x009, 0x00a,
+ 0x081, 0x18d, 0x245, 0x020, 0x327, 0x01d, 0x246, 0x020,
+ 0x3a2, 0x020, 0x003, 0x02b, 0x309, 0x01c, 0x003, 0x022,
+ 0x3c4, 0x3ef, 0x027, 0x3c5, 0x010, 0x026, 0x0a0, 0x387,
+ 0x3df, 0x044, 0x1b8, 0x000, 0x081, 0x189, 0x044, 0x1a9,
+ 0x000, 0x140, 0x01c, 0x00b, 0x00d, 0x0a1, 0x044, 0x192,
+ 0x000, 0x247, 0x27f, 0x395, 0x3ff, 0x044, 0x1bd, 0x000,
+ 0x344, 0x21f, 0x01c, 0x000, 0x02a, 0x307, 0x01c, 0x003,
+ 0x026, 0x345, 0x21d, 0x01c, 0x209, 0x003, 0x020, 0x307,
+ 0x020, 0x181, 0x003, 0x01b, 0x364, 0x025, 0x004, 0x009,
+ 0x005, 0x3c5, 0x008, 0x025, 0x0a0, 0x081, 0x189, 0x044,
+ 0x1b7, 0x000, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1,
+ 0x307, 0x27f, 0x044, 0x1bd, 0x000, 0x055, 0x364, 0x234,
+ 0x002, 0x001, 0x00f, 0x307, 0x020, 0x384, 0x070, 0x266,
+ 0x010, 0x001, 0x007, 0x0c2, 0x26f, 0x044, 0x1b6, 0x001,
+ 0x054, 0x020, 0x0a3, 0x006, 0x346, 0x21d, 0x232, 0x02b,
+ 0x181, 0x006, 0x364, 0x233, 0x001, 0x021, 0x08d, 0x006,
+ 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4, 0x000, 0x327, 0x21d,
+ 0x324, 0x020, 0x045, 0x307, 0x28d, 0x384, 0x07f, 0x22a,
+ 0x286, 0x040, 0x0b0, 0x044, 0x192, 0x000, 0x384, 0x380,
+ 0x043, 0x285, 0x0b0, 0x020, 0x1bd, 0x000, 0x0a2, 0x141,
+ 0x041, 0x141, 0x2f1, 0x397, 0x200, 0x151, 0x041, 0x044,
+ 0x0fb, 0x002, 0x055, 0x364, 0x200, 0x080, 0x009, 0x00e,
+ 0x364, 0x234, 0x002, 0x001, 0x009, 0x054, 0x364, 0x012,
+ 0x040, 0x001, 0x3f1, 0x045, 0x364, 0x2cf, 0x007, 0x009,
+ 0x3fc, 0x044, 0x33f, 0x002, 0x3c4, 0x080, 0x2cf, 0x3c4,
+ 0x2bf, 0x200, 0x080, 0x046, 0x020, 0x0d3, 0x000, 0x0c0,
+ 0x013, 0x0c0, 0x014, 0x0c0, 0x259, 0x045, 0x088, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x364, 0x259, 0x002, 0x009, 0x007,
+ 0x042, 0x042, 0x020, 0x0d3, 0x000, 0x387, 0x085, 0x187,
+ 0x385, 0x080, 0x247, 0x041, 0x3c5, 0x020, 0x00c, 0x045,
+ 0x387, 0x07f, 0x247, 0x2a1, 0x247, 0x2a2, 0x247, 0x2a3,
+ 0x247, 0x2a4, 0x247, 0x2a7, 0x247, 0x2a8, 0x247, 0x2a5,
+ 0x247, 0x2a6, 0x044, 0x150, 0x003, 0x0c3, 0x2e0, 0x364,
+ 0x02b, 0x010, 0x009, 0x011, 0x044, 0x38c, 0x002, 0x044,
+ 0x39f, 0x002, 0x347, 0x290, 0x2c6, 0x347, 0x28f, 0x2c5,
+ 0x347, 0x28e, 0x2c4, 0x045, 0x0c0, 0x2ce, 0x0c8, 0x2d5,
+ 0x0cf, 0x2ca, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x0c5, 0x2cc,
+ 0x0cf, 0x2d1, 0x0c0, 0x2d2, 0x0dd, 0x2cd, 0x045, 0x0c0,
+ 0x2d8, 0x044, 0x3b3, 0x002, 0x3c7, 0x038, 0x290, 0x0cc,
+ 0x28f, 0x0c0, 0x28e, 0x0cd, 0x291, 0x044, 0x034, 0x003,
+ 0x020, 0x06e, 0x003, 0x364, 0x02b, 0x010, 0x009, 0x013,
+ 0x0ca, 0x293, 0x0a0, 0x397, 0x294, 0x04d, 0x101, 0x272,
+ 0x2a0, 0x009, 0x3fc, 0x044, 0x06e, 0x003, 0x044, 0x096,
+ 0x003, 0x090, 0x3a7, 0x022, 0x044, 0x1a9, 0x000, 0x347,
+ 0x2aa, 0x2a1, 0x347, 0x2ab, 0x2a2, 0x347, 0x2ac, 0x2a3,
+ 0x347, 0x2ad, 0x2a4, 0x020, 0x3f0, 0x002, 0x044, 0x034,
+ 0x003, 0x020, 0x096, 0x003, 0x3a7, 0x020, 0x084, 0x020,
+ 0x1b7, 0x000, 0x3a7, 0x020, 0x084, 0x020, 0x1a9, 0x000,
+ 0x307, 0x2a1, 0x187, 0x327, 0x2a3, 0x00b, 0x004, 0x141,
+ 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x024, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a2, 0x187, 0x327, 0x2a4, 0x00b,
+ 0x004, 0x141, 0x2e5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7,
+ 0x026, 0x044, 0x1bd, 0x000, 0x307, 0x2a7, 0x187, 0x327,
+ 0x2a5, 0x3a4, 0x0ff, 0x285, 0x208, 0x3a7, 0x025, 0x044,
+ 0x1bd, 0x000, 0x307, 0x2a8, 0x187, 0x327, 0x2a6, 0x3a4,
+ 0x0ff, 0x285, 0x208, 0x3a7, 0x027, 0x020, 0x1bd, 0x000,
+ 0x0c0, 0x28f, 0x000, 0x00d, 0x140, 0x28f, 0x003, 0x3fa,
+ 0x362, 0x28f, 0x00f, 0x00b, 0x004, 0x0cf, 0x28f, 0x140,
+ 0x28e, 0x003, 0x00b, 0x362, 0x28e, 0x00f, 0x00b, 0x008,
+ 0x0cf, 0x28e, 0x000, 0x004, 0x0c0, 0x28e, 0x307, 0x28f,
+ 0x183, 0x305, 0x28e, 0x22a, 0x3a4, 0x3f7, 0x286, 0x327,
+ 0x290, 0x1a7, 0x285, 0x3a7, 0x02b, 0x044, 0x1bd, 0x000,
+ 0x044, 0x2f6, 0x002, 0x009, 0x005, 0x0c2, 0x2fa, 0x045,
+ 0x045, 0x347, 0x2ca, 0x293, 0x000, 0x011, 0x307, 0x2ca,
+ 0x302, 0x293, 0x003, 0x3f7, 0x327, 0x2c9, 0x322, 0x293,
+ 0x003, 0x005, 0x347, 0x2c9, 0x293, 0x307, 0x291, 0x044,
+ 0x106, 0x003, 0x0a1, 0x364, 0x236, 0x100, 0x009, 0x003,
+ 0x0a5, 0x1a3, 0x285, 0x183, 0x305, 0x292, 0x183, 0x305,
+ 0x293, 0x3a7, 0x02c, 0x020, 0x1bd, 0x000, 0x307, 0x296,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x295, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x294, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x028, 0x044,
+ 0x1bd, 0x000, 0x307, 0x299, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x298, 0x044, 0x106, 0x003, 0x040, 0x307, 0x297,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x029, 0x044, 0x1bd, 0x000, 0x307, 0x29f,
+ 0x044, 0x106, 0x003, 0x040, 0x307, 0x29e, 0x044, 0x106,
+ 0x003, 0x040, 0x307, 0x29d, 0x044, 0x106, 0x003, 0x184,
+ 0x043, 0x285, 0x184, 0x043, 0x285, 0x3a7, 0x031, 0x044,
+ 0x1bd, 0x000, 0x307, 0x29c, 0x044, 0x106, 0x003, 0x040,
+ 0x307, 0x29b, 0x044, 0x106, 0x003, 0x040, 0x307, 0x29a,
+ 0x044, 0x106, 0x003, 0x184, 0x043, 0x285, 0x184, 0x043,
+ 0x285, 0x3a7, 0x02a, 0x020, 0x1bd, 0x000, 0x100, 0x003,
+ 0x005, 0x22a, 0x286, 0x045, 0x228, 0x121, 0x28a, 0x286,
+ 0x385, 0x010, 0x045, 0x083, 0x182, 0x0a1, 0x182, 0x385,
+ 0x001, 0x181, 0x285, 0x181, 0x385, 0x002, 0x181, 0x285,
+ 0x3a7, 0x030, 0x020, 0x1bd, 0x000, 0x044, 0x15b, 0x003,
+ 0x382, 0x081, 0x240, 0x2a5, 0x240, 0x2a6, 0x044, 0x134,
+ 0x003, 0x020, 0x3f0, 0x002, 0x364, 0x2a5, 0x300, 0x001,
+ 0x007, 0x003, 0x010, 0x3c7, 0x0ff, 0x2a5, 0x364, 0x2a6,
+ 0x300, 0x001, 0x007, 0x003, 0x00a, 0x3c7, 0x0ff, 0x2a6,
+ 0x045, 0x0c0, 0x2a5, 0x000, 0x3f3, 0x0c0, 0x2a6, 0x045,
+ 0x397, 0x2a1, 0x04c, 0x109, 0x04d, 0x118, 0x272, 0x2a9,
+ 0x009, 0x3fa, 0x045, 0x347, 0x2ae, 0x2a5, 0x347, 0x2af,
+ 0x2a6, 0x045, 0x342, 0x2fe, 0x00f, 0x003, 0x03f, 0x045,
+ 0x302, 0x2fe, 0x003, 0x019, 0x042, 0x042, 0x045, 0x327,
+ 0x00e, 0x309, 0x00f, 0x002, 0x005, 0x2a8, 0x208, 0x121,
+ 0x20e, 0x040, 0x041, 0x322, 0x2fd, 0x001, 0x3eb, 0x303,
+ 0x2fe, 0x00b, 0x3eb, 0x042, 0x247, 0x2fd, 0x042, 0x247,
+ 0x2fe, 0x000, 0x02e, 0x0c0, 0x2ff, 0x044, 0x30e, 0x002,
+ 0x364, 0x017, 0x004, 0x009, 0x3dc, 0x340, 0x046, 0x047,
+ 0x001, 0x021, 0x342, 0x2fd, 0x00e, 0x001, 0x3c5, 0x343,
+ 0x2fe, 0x00f, 0x00b, 0x017, 0x347, 0x00e, 0x2fd, 0x347,
+ 0x00f, 0x2fe, 0x009, 0x00d, 0x140, 0x2fd, 0x003, 0x009,
+ 0x342, 0x300, 0x2fd, 0x00b, 0x004, 0x0c0, 0x2fd, 0x0c1,
+ 0x2ff, 0x045, 0x30e, 0x21d, 0x102, 0x000, 0x004, 0x30e,
+ 0x21d, 0x327, 0x2c7, 0x041, 0x327, 0x2c8, 0x041, 0x0a2,
+ 0x044, 0x0fb, 0x002, 0x045, 0x366, 0x200, 0x080, 0x009,
+ 0x019, 0x364, 0x026, 0x002, 0x001, 0x014, 0x054, 0x140,
+ 0x2fa, 0x00b, 0x3f3, 0x3c7, 0x3ff, 0x2fa, 0x360, 0x2fa,
+ 0x00a, 0x003, 0x007, 0x054, 0x140, 0x01c, 0x00b, 0x3f8,
+ 0x045, 0x0c0, 0x301, 0x3ce, 0x3ff, 0x2fe, 0x040, 0x044,
+ 0x3f0, 0x002, 0x044, 0x18b, 0x003, 0x042, 0x04c, 0x140,
+ 0x301, 0x001, 0x010, 0x346, 0x301, 0x00f, 0x003, 0x029,
+ 0x320, 0x303, 0x2e4, 0x100, 0x009, 0x02a, 0x04d, 0x000,
+ 0x3e7, 0x0c1, 0x303, 0x347, 0x00f, 0x301, 0x00b, 0x004,
+ 0x152, 0x303, 0x009, 0x004, 0x0c1, 0x301, 0x040, 0x3a7,
+ 0x060, 0x044, 0x192, 0x000, 0x264, 0x008, 0x001, 0x007,
+ 0x348, 0x303, 0x303, 0x141, 0x303, 0x000, 0x3d0, 0x140,
+ 0x2ff, 0x009, 0x005, 0x322, 0x303, 0x04d, 0x045, 0x364,
+ 0x2cf, 0x080, 0x009, 0x019, 0x044, 0x346, 0x002, 0x0a0,
+ 0x397, 0x294, 0x04d, 0x101, 0x272, 0x2a0, 0x009, 0x3fc,
+ 0x044, 0x096, 0x003, 0x0c0, 0x293, 0x044, 0x06e, 0x003,
+ 0x044, 0x2b1, 0x003, 0x140, 0x2d3, 0x009, 0x002, 0x3c4,
+ 0x2fd, 0x00d, 0x044, 0x33f, 0x002, 0x3c4, 0x2ff, 0x200,
+ 0x020, 0x089, 0x000, 0x0cd, 0x2ce, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x307, 0x2e3, 0x140, 0x304, 0x009,
+ 0x004, 0x208, 0x101, 0x240, 0x2a6, 0x240, 0x2a5, 0x044,
+ 0x134, 0x003, 0x044, 0x1ba, 0x003, 0x0c0, 0x309, 0x387,
+ 0x071, 0x140, 0x304, 0x001, 0x003, 0x108, 0x364, 0x309,
+ 0x001, 0x001, 0x004, 0x380, 0x040, 0x187, 0x106, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x397, 0x2a5, 0x300, 0x309, 0x044,
+ 0x1e9, 0x003, 0x141, 0x309, 0x362, 0x309, 0x002, 0x009,
+ 0x3e0, 0x044, 0x3f0, 0x002, 0x347, 0x2a5, 0x2e3, 0x342,
+ 0x2ae, 0x2e3, 0x00b, 0x007, 0x348, 0x2e3, 0x2e3, 0x141,
+ 0x2e3, 0x140, 0x307, 0x009, 0x005, 0x347, 0x2e3, 0x307,
+ 0x045, 0x364, 0x2cf, 0x080, 0x009, 0x0ac, 0x0c1, 0x2ce,
+ 0x055, 0x3c5, 0x004, 0x2cf, 0x3a7, 0x071, 0x044, 0x192,
+ 0x000, 0x247, 0x305, 0x054, 0x080, 0x044, 0x1bd, 0x000,
+ 0x3a7, 0x02c, 0x387, 0x30f, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x022, 0x08a, 0x044, 0x1bd, 0x000, 0x081, 0x189, 0x0a0,
+ 0x044, 0x1a9, 0x000, 0x30e, 0x21d, 0x102, 0x0b0, 0x041,
+ 0x0a0, 0x041, 0x0a2, 0x044, 0x0fb, 0x002, 0x080, 0x0a0,
+ 0x262, 0x001, 0x00b, 0x004, 0x3a7, 0x0ff, 0x040, 0x397,
+ 0x2a1, 0x04d, 0x101, 0x04d, 0x101, 0x04d, 0x101, 0x04d,
+ 0x113, 0x043, 0x280, 0x3a7, 0x080, 0x04d, 0x040, 0x392,
+ 0x2a1, 0x262, 0x005, 0x00b, 0x007, 0x040, 0x044, 0x3e4,
+ 0x002, 0x042, 0x227, 0x3a4, 0x001, 0x121, 0x1a5, 0x191,
+ 0x285, 0x187, 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x042,
+ 0x044, 0x1e9, 0x003, 0x041, 0x101, 0x392, 0x2a1, 0x264,
+ 0x008, 0x001, 0x3c6, 0x390, 0x2a1, 0x111, 0x043, 0x04d,
+ 0x272, 0x2a1, 0x003, 0x3fb, 0x044, 0x150, 0x003, 0x044,
+ 0x3ea, 0x002, 0x044, 0x06e, 0x003, 0x364, 0x292, 0x008,
+ 0x009, 0x008, 0x080, 0x3a7, 0x022, 0x044, 0x1bd, 0x000,
+ 0x307, 0x020, 0x192, 0x002, 0x013, 0x19a, 0x002, 0x010,
+ 0x081, 0x189, 0x0a0, 0x044, 0x1b7, 0x000, 0x055, 0x3a7,
+ 0x071, 0x307, 0x305, 0x044, 0x1bd, 0x000, 0x044, 0x3f0,
+ 0x002, 0x3c4, 0x3fb, 0x2cf, 0x3c5, 0x080, 0x2cf, 0x054,
+ 0x045, 0x397, 0x36a, 0x247, 0x369, 0x0a0, 0x101, 0x04d,
+ 0x272, 0x38e, 0x009, 0x3fc, 0x045, 0x0c3, 0x34e, 0x1cb,
+ 0x34e, 0x3c5, 0x03f, 0x34e, 0x0c3, 0x362, 0x0c4, 0x363,
+ 0x0c0, 0x33c, 0x0c0, 0x356, 0x0c0, 0x357, 0x3d7, 0x2f9,
+ 0x352, 0x1c2, 0x352, 0x3d7, 0x254, 0x353, 0x1c0, 0x353,
+ 0x08d, 0x18b, 0x385, 0x05e, 0x247, 0x354, 0x3c7, 0x0b2,
+ 0x355, 0x345, 0x21d, 0x355, 0x0c4, 0x341, 0x0c2, 0x33f,
+ 0x0d2, 0x340, 0x0d3, 0x342, 0x0ce, 0x344, 0x0de, 0x343,
+ 0x0c0, 0x348, 0x045, 0x055, 0x3c4, 0x008, 0x33b, 0x144,
+ 0x33b, 0x0c6, 0x33d, 0x3c4, 0x3fb, 0x200, 0x307, 0x027,
+ 0x384, 0x324, 0x101, 0x247, 0x027, 0x045, 0x055, 0x3c4,
+ 0x3ec, 0x052, 0x044, 0x05b, 0x001, 0x3c4, 0x3fb, 0x00c,
+ 0x0c0, 0x359, 0x3c4, 0x1ff, 0x35b, 0x045, 0x055, 0x0c0,
+ 0x33d, 0x3c4, 0x3fe, 0x03d, 0x0df, 0x053, 0x3c4, 0x373,
+ 0x00c, 0x0c0, 0x349, 0x0c0, 0x35a, 0x3c4, 0x330, 0x027,
+ 0x0c0, 0x33b, 0x344, 0x21f, 0x33c, 0x045, 0x044, 0x050,
+ 0x005, 0x307, 0x33c, 0x00b, 0x018, 0x362, 0x33d, 0x003,
+ 0x007, 0x013, 0x009, 0x005, 0x044, 0x073, 0x004, 0x055,
+ 0x364, 0x33c, 0x040, 0x001, 0x008, 0x3c4, 0x3fb, 0x200,
+ 0x044, 0x103, 0x004, 0x044, 0x1bb, 0x005, 0x020, 0x0ad,
+ 0x000, 0x364, 0x33c, 0x002, 0x001, 0x055, 0x3c5, 0x008,
+ 0x00c, 0x364, 0x33c, 0x080, 0x009, 0x023, 0x0c0, 0x392,
+ 0x347, 0x354, 0x03e, 0x347, 0x355, 0x03f, 0x347, 0x352,
+ 0x038, 0x347, 0x353, 0x039, 0x307, 0x352, 0x040, 0x307,
+ 0x353, 0x040, 0x307, 0x354, 0x040, 0x307, 0x355, 0x305,
+ 0x21d, 0x040, 0x397, 0x2a0, 0x044, 0x01a, 0x002, 0x0c0,
+ 0x290, 0x044, 0x034, 0x003, 0x044, 0x361, 0x003, 0x0c7,
+ 0x2d1, 0x0c3, 0x2cc, 0x0c0, 0x2c8, 0x0c1, 0x2c7, 0x1cd,
+ 0x2c7, 0x0c1, 0x33d, 0x142, 0x027, 0x0c0, 0x33e, 0x0c0,
+ 0x365, 0x020, 0x04c, 0x004, 0x085, 0x364, 0x33b, 0x020,
+ 0x001, 0x003, 0x102, 0x18a, 0x385, 0x11f, 0x247, 0x053,
+ 0x045, 0x0c2, 0x33b, 0x0c5, 0x33d, 0x3c5, 0x010, 0x027,
+ 0x0df, 0x053, 0x045, 0x044, 0x3b6, 0x003, 0x044, 0x3c6,
+ 0x003, 0x044, 0x38c, 0x002, 0x044, 0x39f, 0x002, 0x3c4,
+ 0x3fb, 0x200, 0x045, 0x055, 0x364, 0x33c, 0x020, 0x009,
+ 0x007, 0x0c4, 0x33d, 0x044, 0x103, 0x004, 0x0c5, 0x33d,
+ 0x3c4, 0x3fe, 0x03d, 0x3c5, 0x00a, 0x33b, 0x307, 0x027,
+ 0x384, 0x334, 0x305, 0x392, 0x247, 0x027, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x377, 0x00c, 0x020, 0x38c, 0x002, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x044, 0x006,
+ 0x005, 0x364, 0x33b, 0x010, 0x3c4, 0x3bf, 0x33c, 0x364,
+ 0x33c, 0x020, 0x009, 0x007, 0x364, 0x33c, 0x002, 0x001,
+ 0x015, 0x3c5, 0x004, 0x00c, 0x364, 0x33c, 0x020, 0x001,
+ 0x005, 0x3c5, 0x080, 0x058, 0x0c0, 0x350, 0x044, 0x0c5,
+ 0x004, 0x3c5, 0x001, 0x052, 0x045, 0x0c0, 0x04c, 0x347,
+ 0x357, 0x04d, 0x347, 0x356, 0x04e, 0x347, 0x357, 0x364,
+ 0x347, 0x356, 0x365, 0x387, 0x067, 0x187, 0x385, 0x080,
+ 0x3c4, 0x04c, 0x052, 0x245, 0x052, 0x309, 0x026, 0x003,
+ 0x00c, 0x0c0, 0x029, 0x3c7, 0x022, 0x029, 0x345, 0x21d,
+ 0x029, 0x000, 0x00f, 0x347, 0x362, 0x02c, 0x399, 0x380,
+ 0x208, 0x244, 0x029, 0x307, 0x363, 0x187, 0x245, 0x029,
+ 0x081, 0x18c, 0x245, 0x053, 0x347, 0x021, 0x35c, 0x3c4,
+ 0x3cf, 0x021, 0x045, 0x055, 0x364, 0x052, 0x001, 0x001,
+ 0x02a, 0x142, 0x052, 0x086, 0x187, 0x385, 0x072, 0x364,
+ 0x33c, 0x020, 0x001, 0x008, 0x044, 0x18d, 0x000, 0x055,
+ 0x000, 0x005, 0x044, 0x189, 0x000, 0x347, 0x35c, 0x021,
+ 0x3c7, 0x080, 0x052, 0x3c4, 0x3fb, 0x00c, 0x364, 0x33c,
+ 0x200, 0x009, 0x008, 0x3c5, 0x020, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x045, 0x055, 0x140, 0x33d, 0x001, 0x3fc, 0x055,
+ 0x307, 0x368, 0x304, 0x34e, 0x0c0, 0x351, 0x227, 0x1a2,
+ 0x002, 0x071, 0x00b, 0x08b, 0x044, 0x03b, 0x005, 0x044,
+ 0x006, 0x005, 0x141, 0x374, 0x140, 0x346, 0x001, 0x00b,
+ 0x342, 0x342, 0x346, 0x00b, 0x019, 0x0c4, 0x350, 0x000,
+ 0x019, 0x340, 0x347, 0x345, 0x001, 0x00a, 0x342, 0x347,
+ 0x345, 0x364, 0x33c, 0x004, 0x001, 0x3ec, 0x0cc, 0x350,
+ 0x141, 0x384, 0x000, 0x009, 0x0c8, 0x350, 0x141, 0x383,
+ 0x340, 0x342, 0x346, 0x140, 0x345, 0x001, 0x00c, 0x342,
+ 0x344, 0x345, 0x00b, 0x00e, 0x3c5, 0x001, 0x350, 0x000,
+ 0x00e, 0x3c5, 0x003, 0x350, 0x141, 0x37c, 0x000, 0x00a,
+ 0x3c5, 0x002, 0x350, 0x141, 0x37b, 0x340, 0x344, 0x345,
+ 0x140, 0x347, 0x001, 0x00c, 0x342, 0x343, 0x347, 0x00b,
+ 0x00f, 0x3c5, 0x010, 0x350, 0x000, 0x00f, 0x3c5, 0x030,
+ 0x350, 0x141, 0x38c, 0x020, 0x3fa, 0x004, 0x3c5, 0x020,
+ 0x350, 0x141, 0x38b, 0x340, 0x343, 0x347, 0x020, 0x3fa,
+ 0x004, 0x044, 0x03b, 0x005, 0x141, 0x375, 0x0c0, 0x346,
+ 0x0c0, 0x347, 0x0c0, 0x345, 0x044, 0x01e, 0x005, 0x3c7,
+ 0x03f, 0x350, 0x141, 0x37c, 0x141, 0x384, 0x141, 0x38c,
+ 0x0c0, 0x348, 0x020, 0x3fa, 0x004, 0x205, 0x001, 0x182,
+ 0x044, 0x03b, 0x005, 0x227, 0x3a4, 0x00c, 0x1b1, 0x001,
+ 0x013, 0x364, 0x350, 0x00c, 0x009, 0x05c, 0x364, 0x04d,
+ 0x00c, 0x009, 0x057, 0x132, 0x003, 0x00b, 0x001, 0x027,
+ 0x000, 0x050, 0x3c4, 0x3f3, 0x350, 0x000, 0x04b, 0x140,
+ 0x351, 0x009, 0x004, 0x141, 0x37f, 0x140, 0x346, 0x001,
+ 0x00b, 0x151, 0x348, 0x151, 0x346, 0x3c5, 0x004, 0x350,
+ 0x000, 0x038, 0x3c5, 0x00c, 0x350, 0x140, 0x351, 0x001,
+ 0x031, 0x141, 0x384, 0x000, 0x02d, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x380, 0x342, 0x23e, 0x348, 0x001, 0x016,
+ 0x342, 0x342, 0x346, 0x001, 0x00e, 0x141, 0x348, 0x141,
+ 0x346, 0x340, 0x342, 0x346, 0x3c5, 0x004, 0x350, 0x000,
+ 0x00e, 0x340, 0x342, 0x346, 0x3c5, 0x008, 0x350, 0x140,
+ 0x351, 0x001, 0x004, 0x141, 0x383, 0x340, 0x23e, 0x348,
+ 0x227, 0x3a4, 0x003, 0x001, 0x013, 0x364, 0x350, 0x003,
+ 0x009, 0x071, 0x364, 0x04d, 0x003, 0x009, 0x06c, 0x132,
+ 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065, 0x3c4, 0x3fc,
+ 0x350, 0x000, 0x060, 0x140, 0x351, 0x009, 0x004, 0x141,
+ 0x377, 0x140, 0x345, 0x009, 0x00d, 0x3c5, 0x003, 0x350,
+ 0x140, 0x351, 0x001, 0x04f, 0x141, 0x37c, 0x000, 0x04b,
+ 0x342, 0x342, 0x346, 0x009, 0x007, 0x340, 0x342, 0x346,
+ 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151, 0x348, 0x141,
+ 0x346, 0x152, 0x345, 0x3c5, 0x001, 0x350, 0x000, 0x033,
+ 0x140, 0x351, 0x009, 0x004, 0x141, 0x378, 0x342, 0x23e,
+ 0x348, 0x001, 0x01c, 0x342, 0x344, 0x345, 0x001, 0x014,
+ 0x140, 0x346, 0x001, 0x010, 0x141, 0x348, 0x151, 0x346,
+ 0x142, 0x345, 0x340, 0x344, 0x345, 0x3c5, 0x001, 0x350,
+ 0x000, 0x00e, 0x340, 0x344, 0x345, 0x3c5, 0x002, 0x350,
+ 0x140, 0x351, 0x001, 0x004, 0x141, 0x37b, 0x340, 0x23e,
+ 0x348, 0x227, 0x1b3, 0x3a4, 0x003, 0x001, 0x013, 0x364,
+ 0x350, 0x030, 0x009, 0x071, 0x364, 0x04d, 0x030, 0x009,
+ 0x06c, 0x132, 0x003, 0x00b, 0x001, 0x036, 0x000, 0x065,
+ 0x3c4, 0x3cf, 0x350, 0x000, 0x060, 0x140, 0x351, 0x009,
+ 0x004, 0x141, 0x387, 0x140, 0x347, 0x009, 0x00d, 0x3c5,
+ 0x030, 0x350, 0x140, 0x351, 0x001, 0x04f, 0x141, 0x38c,
+ 0x000, 0x04b, 0x342, 0x342, 0x346, 0x009, 0x007, 0x340,
+ 0x342, 0x346, 0x000, 0x3ed, 0x340, 0x342, 0x346, 0x151,
+ 0x348, 0x141, 0x346, 0x152, 0x347, 0x3c5, 0x010, 0x350,
+ 0x000, 0x033, 0x140, 0x351, 0x009, 0x004, 0x141, 0x388,
+ 0x342, 0x23e, 0x348, 0x001, 0x01c, 0x342, 0x343, 0x347,
+ 0x001, 0x014, 0x140, 0x346, 0x001, 0x010, 0x141, 0x348,
+ 0x151, 0x346, 0x142, 0x347, 0x340, 0x343, 0x347, 0x3c5,
+ 0x010, 0x350, 0x000, 0x00e, 0x340, 0x343, 0x347, 0x3c5,
+ 0x020, 0x350, 0x140, 0x351, 0x001, 0x004, 0x141, 0x38b,
+ 0x340, 0x23e, 0x348, 0x328, 0x04d, 0x324, 0x350, 0x2e4,
+ 0x02a, 0x001, 0x026, 0x140, 0x351, 0x009, 0x022, 0x141,
+ 0x351, 0x2e4, 0x020, 0x001, 0x005, 0x3c4, 0x3cf, 0x350,
+ 0x2e4, 0x008, 0x001, 0x005, 0x3c4, 0x3f3, 0x350, 0x2e4,
+ 0x002, 0x001, 0x28a, 0x3c4, 0x3fc, 0x350, 0x000, 0x285,
+ 0x0c0, 0x350, 0x3c4, 0x3bf, 0x027, 0x000, 0x0a8, 0x328,
+ 0x04d, 0x324, 0x350, 0x287, 0x3a4, 0x003, 0x131, 0x009,
+ 0x029, 0x140, 0x345, 0x001, 0x01c, 0x342, 0x344, 0x345,
+ 0x044, 0x030, 0x005, 0x003, 0x006, 0x140, 0x345, 0x009,
+ 0x00b, 0x364, 0x368, 0x002, 0x001, 0x006, 0x141, 0x350,
+ 0x141, 0x37b, 0x340, 0x344, 0x345, 0x000, 0x00b, 0x364,
+ 0x368, 0x001, 0x001, 0x006, 0x142, 0x350, 0x141, 0x37c,
+ 0x227, 0x3a4, 0x00c, 0x134, 0x009, 0x039, 0x140, 0x346,
+ 0x001, 0x01c, 0x342, 0x342, 0x346, 0x044, 0x030, 0x005,
+ 0x003, 0x006, 0x140, 0x346, 0x009, 0x00b, 0x364, 0x368,
+ 0x008, 0x001, 0x006, 0x144, 0x350, 0x141, 0x383, 0x340,
+ 0x342, 0x346, 0x000, 0x01b, 0x364, 0x368, 0x008, 0x009,
+ 0x3e3, 0x364, 0x368, 0x004, 0x001, 0x011, 0x327, 0x345,
+ 0x320, 0x347, 0x001, 0x007, 0x364, 0x33c, 0x004, 0x001,
+ 0x006, 0x148, 0x350, 0x141, 0x384, 0x227, 0x3a4, 0x030,
+ 0x3a2, 0x010, 0x009, 0x02b, 0x140, 0x347, 0x001, 0x01d,
+ 0x342, 0x343, 0x347, 0x044, 0x030, 0x005, 0x003, 0x006,
+ 0x140, 0x347, 0x009, 0x00c, 0x364, 0x368, 0x020, 0x001,
+ 0x007, 0x3c0, 0x010, 0x350, 0x141, 0x38b, 0x340, 0x343,
+ 0x347, 0x000, 0x00c, 0x364, 0x368, 0x010, 0x001, 0x007,
+ 0x3c0, 0x020, 0x350, 0x141, 0x38c, 0x044, 0x3fa, 0x004,
+ 0x000, 0x026, 0x3c5, 0x040, 0x027, 0x307, 0x04d, 0x384,
+ 0x200, 0x305, 0x350, 0x247, 0x04d, 0x045, 0x08f, 0x3b7,
+ 0x211, 0x044, 0x1b8, 0x000, 0x347, 0x33f, 0x346, 0x347,
+ 0x340, 0x347, 0x347, 0x341, 0x345, 0x347, 0x347, 0x348,
+ 0x340, 0x345, 0x348, 0x340, 0x346, 0x348, 0x307, 0x347,
+ 0x183, 0x305, 0x345, 0x184, 0x305, 0x346, 0x32a, 0x346,
+ 0x3a4, 0x008, 0x285, 0x3b7, 0x210, 0x020, 0x1bd, 0x000,
+ 0x364, 0x33c, 0x008, 0x001, 0x007, 0x327, 0x23e, 0x322,
+ 0x348, 0x131, 0x045, 0x040, 0x227, 0x306, 0x373, 0x001,
+ 0x00f, 0x307, 0x369, 0x101, 0x272, 0x372, 0x009, 0x003,
+ 0x118, 0x04d, 0x247, 0x369, 0x2c7, 0x373, 0x042, 0x045,
+ 0x140, 0x35a, 0x00b, 0x0b0, 0x344, 0x21f, 0x35a, 0x0a3,
+ 0x1ad, 0x2a8, 0x2c4, 0x027, 0x055, 0x044, 0x340, 0x001,
+ 0x054, 0x264, 0x002, 0x001, 0x005, 0x0c0, 0x33d, 0x045,
+ 0x083, 0x18b, 0x304, 0x050, 0x305, 0x221, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3f4, 0x308, 0x221, 0x244, 0x027, 0x387,
+ 0x030, 0x304, 0x050, 0x187, 0x044, 0x0d3, 0x005, 0x001,
+ 0x3e6, 0x08c, 0x304, 0x050, 0x189, 0x044, 0x0d3, 0x005,
+ 0x001, 0x3dd, 0x083, 0x304, 0x050, 0x18b, 0x044, 0x0d3,
+ 0x005, 0x001, 0x3d4, 0x083, 0x18b, 0x304, 0x365, 0x044,
+ 0x0d3, 0x005, 0x001, 0x3cb, 0x387, 0x030, 0x304, 0x365,
+ 0x187, 0x044, 0x0d3, 0x005, 0x001, 0x3c1, 0x08c, 0x304,
+ 0x365, 0x189, 0x044, 0x0d3, 0x005, 0x001, 0x3b8, 0x083,
+ 0x304, 0x365, 0x18b, 0x044, 0x0d3, 0x005, 0x001, 0x3af,
+ 0x307, 0x21d, 0x304, 0x364, 0x192, 0x364, 0x052, 0x020,
+ 0x001, 0x00d, 0x327, 0x21d, 0x324, 0x04f, 0x001, 0x005,
+ 0x345, 0x21d, 0x33c, 0x1b1, 0x285, 0x044, 0x0d3, 0x005,
+ 0x001, 0x395, 0x045, 0x364, 0x33b, 0x001, 0x001, 0x02c,
+ 0x0a3, 0x1ab, 0x2a8, 0x2c4, 0x027, 0x245, 0x027, 0x345,
+ 0x21d, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x01d, 0x044,
+ 0x103, 0x005, 0x364, 0x35b, 0x002, 0x001, 0x3f5, 0x344,
+ 0x21f, 0x027, 0x364, 0x33b, 0x001, 0x001, 0x00d, 0x044,
+ 0x103, 0x005, 0x307, 0x35b, 0x264, 0x002, 0x009, 0x3f4,
+ 0x264, 0x001, 0x045, 0x055, 0x364, 0x059, 0x080, 0x001,
+ 0x012, 0x364, 0x24b, 0x080, 0x009, 0x00d, 0x364, 0x001,
+ 0x080, 0x009, 0x008, 0x3c5, 0x080, 0x24b, 0x044, 0x12d,
+ 0x005, 0x044, 0x340, 0x001, 0x247, 0x35b, 0x374, 0x027,
+ 0x200, 0x001, 0x00a, 0x264, 0x080, 0x009, 0x006, 0x3b8,
+ 0x200, 0x2c4, 0x027, 0x054, 0x045, 0x044, 0x340, 0x001,
+ 0x264, 0x080, 0x001, 0x088, 0x3b8, 0x300, 0x2c4, 0x027,
+ 0x264, 0x040, 0x001, 0x00f, 0x140, 0x359, 0x001, 0x00b,
+ 0x0c0, 0x33d, 0x0c0, 0x359, 0x3d5, 0x200, 0x027, 0x000,
+ 0x060, 0x384, 0x030, 0x327, 0x359, 0x2e2, 0x008, 0x003,
+ 0x3f1, 0x2a9, 0x3b0, 0x15a, 0x041, 0x1b9, 0x3b0, 0x005,
+ 0x041, 0x045, 0x000, 0x012, 0x000, 0x015, 0x000, 0x015,
+ 0x000, 0x016, 0x000, 0x017, 0x000, 0x01a, 0x000, 0x01c,
+ 0x000, 0x01f, 0x000, 0x022, 0x187, 0x247, 0x367, 0x000,
+ 0x033, 0x000, 0x02f, 0x191, 0x000, 0x02c, 0x193, 0x000,
+ 0x029, 0x187, 0x247, 0x366, 0x000, 0x026, 0x245, 0x366,
+ 0x000, 0x022, 0x191, 0x245, 0x366, 0x000, 0x01d, 0x193,
+ 0x245, 0x366, 0x000, 0x018, 0x229, 0x3a4, 0x040, 0x2c5,
+ 0x33c, 0x329, 0x04d, 0x18b, 0x2cc, 0x04d, 0x347, 0x367,
+ 0x368, 0x347, 0x366, 0x04e, 0x0c0, 0x359, 0x000, 0x006,
+ 0x245, 0x367, 0x141, 0x359, 0x3d5, 0x300, 0x027, 0x140,
+ 0x359, 0x009, 0x011, 0x140, 0x04d, 0x00b, 0x00a, 0x364,
+ 0x33c, 0x040, 0x001, 0x005, 0x020, 0x103, 0x004, 0x044,
+ 0x137, 0x004, 0x045, 0x362, 0x33e, 0x007, 0x009, 0x003,
+ 0x045, 0x055, 0x362, 0x33d, 0x000, 0x006, 0x077, 0x362,
+ 0x33d, 0x005, 0x00f, 0x072, 0x140, 0x34d, 0x009, 0x007,
+ 0x044, 0x23e, 0x005, 0x000, 0x069, 0x308, 0x356, 0x304,
+ 0x365, 0x001, 0x007, 0x044, 0x36a, 0x005, 0x000, 0x05e,
+ 0x140, 0x349, 0x001, 0x017, 0x364, 0x04f, 0x03f, 0x009,
+ 0x055, 0x0c0, 0x349, 0x362, 0x34f, 0x000, 0x006, 0x04e,
+ 0x307, 0x34f, 0x044, 0x000, 0x006, 0x0c0, 0x34f, 0x000,
+ 0x045, 0x140, 0x33e, 0x009, 0x007, 0x044, 0x23e, 0x005,
+ 0x000, 0x03c, 0x362, 0x33e, 0x001, 0x009, 0x007, 0x044,
+ 0x269, 0x005, 0x000, 0x032, 0x362, 0x33e, 0x002, 0x009,
+ 0x007, 0x044, 0x26f, 0x005, 0x000, 0x028, 0x362, 0x33e,
+ 0x003, 0x009, 0x007, 0x044, 0x27f, 0x005, 0x000, 0x01e,
+ 0x362, 0x33e, 0x004, 0x009, 0x007, 0x044, 0x29a, 0x005,
+ 0x000, 0x014, 0x362, 0x33e, 0x005, 0x009, 0x007, 0x044,
+ 0x2b7, 0x005, 0x000, 0x00a, 0x362, 0x33e, 0x006, 0x009,
+ 0x005, 0x044, 0x34f, 0x005, 0x054, 0x045, 0x081, 0x189,
+ 0x304, 0x053, 0x009, 0x00c, 0x0c1, 0x33d, 0x083, 0x189,
+ 0x395, 0x3f7, 0x244, 0x027, 0x000, 0x01c, 0x0c2, 0x33d,
+ 0x0c1, 0x34d, 0x3c5, 0x008, 0x027, 0x0c0, 0x34b, 0x0c0,
+ 0x34c, 0x140, 0x35d, 0x009, 0x00b, 0x0c0, 0x34f, 0x088,
+ 0x189, 0x245, 0x365, 0x044, 0x3c2, 0x005, 0x0c1, 0x33e,
+ 0x045, 0x347, 0x35e, 0x34a, 0x0c2, 0x33e, 0x045, 0x307,
+ 0x34b, 0x302, 0x360, 0x006, 0x009, 0x141, 0x34b, 0x044,
+ 0x3f3, 0x005, 0x000, 0x004, 0x0c3, 0x33e, 0x045, 0x307,
+ 0x34c, 0x302, 0x361, 0x006, 0x014, 0x141, 0x34c, 0x088,
+ 0x3c7, 0x020, 0x34f, 0x3c7, 0x028, 0x372, 0x141, 0x38a,
+ 0x141, 0x382, 0x044, 0x000, 0x006, 0x000, 0x004, 0x0c4,
+ 0x33e, 0x045, 0x0c5, 0x33e, 0x0c0, 0x2d3, 0x364, 0x02b,
+ 0x080, 0x009, 0x015, 0x3c5, 0x010, 0x2cf, 0x3c5, 0x040,
+ 0x200, 0x362, 0x201, 0x002, 0x001, 0x00a, 0x0c1, 0x244,
+ 0x3c5, 0x004, 0x245, 0x3c4, 0x3fb, 0x200, 0x045, 0x364,
+ 0x2cf, 0x010, 0x009, 0x094, 0x364, 0x34a, 0x001, 0x001,
+ 0x009, 0x0c6, 0x33e, 0x044, 0x33f, 0x002, 0x000, 0x088,
+ 0x140, 0x259, 0x009, 0x009, 0x044, 0x346, 0x002, 0x0c0,
+ 0x330, 0x0c0, 0x331, 0x044, 0x012, 0x006, 0x140, 0x330,
+ 0x009, 0x00c, 0x140, 0x331, 0x009, 0x008, 0x347, 0x32e,
+ 0x330, 0x347, 0x32f, 0x331, 0x362, 0x331, 0x000, 0x00f,
+ 0x00c, 0x362, 0x32f, 0x000, 0x00f, 0x007, 0x044, 0x3f3,
+ 0x005, 0x000, 0x057, 0x362, 0x331, 0x000, 0x007, 0x00c,
+ 0x362, 0x32f, 0x000, 0x007, 0x007, 0x044, 0x3e6, 0x005,
+ 0x000, 0x048, 0x044, 0x33f, 0x002, 0x0c6, 0x33e, 0x362,
+ 0x331, 0x000, 0x00f, 0x021, 0x308, 0x330, 0x348, 0x331,
+ 0x336, 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x330, 0x347,
+ 0x336, 0x331, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302,
+ 0x32e, 0x343, 0x32f, 0x336, 0x002, 0x024, 0x044, 0x3e6,
+ 0x005, 0x000, 0x01f, 0x308, 0x32e, 0x348, 0x32f, 0x336,
+ 0x101, 0x3c1, 0x000, 0x336, 0x247, 0x32e, 0x347, 0x336,
+ 0x32f, 0x307, 0x330, 0x347, 0x331, 0x336, 0x302, 0x32e,
+ 0x343, 0x32f, 0x336, 0x002, 0x005, 0x044, 0x3f3, 0x005,
+ 0x347, 0x32e, 0x330, 0x347, 0x32f, 0x331, 0x045, 0x3c5,
+ 0x008, 0x33b, 0x0c3, 0x33d, 0x3c5, 0x004, 0x027, 0x0c7,
+ 0x33e, 0x364, 0x33c, 0x020, 0x001, 0x00a, 0x345, 0x21d,
+ 0x364, 0x347, 0x21d, 0x35a, 0x000, 0x005, 0x345, 0x21d,
+ 0x04d, 0x045, 0x364, 0x04f, 0x03f, 0x001, 0x054, 0x307,
+ 0x04f, 0x384, 0x003, 0x113, 0x009, 0x007, 0x3c5, 0x001,
+ 0x34a, 0x141, 0x37e, 0x307, 0x04f, 0x384, 0x003, 0x112,
+ 0x009, 0x007, 0x3c5, 0x001, 0x34a, 0x141, 0x37d, 0x307,
+ 0x04f, 0x384, 0x00c, 0x11c, 0x009, 0x007, 0x3c5, 0x002,
+ 0x34a, 0x141, 0x386, 0x307, 0x04f, 0x384, 0x00c, 0x118,
+ 0x009, 0x007, 0x3c5, 0x002, 0x34a, 0x141, 0x385, 0x307,
+ 0x04f, 0x384, 0x030, 0x382, 0x030, 0x009, 0x007, 0x3c5,
+ 0x004, 0x34a, 0x141, 0x38e, 0x307, 0x04f, 0x384, 0x030,
+ 0x382, 0x020, 0x009, 0x007, 0x3c5, 0x004, 0x34a, 0x141,
+ 0x38d, 0x347, 0x356, 0x365, 0x0c1, 0x349, 0x044, 0x3c2,
+ 0x005, 0x045, 0x088, 0x189, 0x304, 0x365, 0x001, 0x004,
+ 0x141, 0x376, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x347,
+ 0x21d, 0x35a, 0x000, 0x013, 0x347, 0x365, 0x04e, 0x307,
+ 0x04e, 0x306, 0x356, 0x001, 0x007, 0x3c5, 0x080, 0x027,
+ 0x000, 0x005, 0x3c4, 0x37f, 0x027, 0x045, 0x081, 0x0c4,
+ 0x34f, 0x0c5, 0x372, 0x141, 0x379, 0x141, 0x381, 0x044,
+ 0x000, 0x006, 0x045, 0x088, 0x0c2, 0x34f, 0x0ca, 0x372,
+ 0x141, 0x37a, 0x141, 0x382, 0x044, 0x000, 0x006, 0x045,
+ 0x364, 0x33c, 0x010, 0x001, 0x009, 0x22a, 0x3a4, 0x015,
+ 0x209, 0x384, 0x02a, 0x285, 0x245, 0x365, 0x044, 0x3c2,
+ 0x005, 0x045, 0x0c2, 0x2ce, 0x0c0, 0x304, 0x044, 0x25b,
+ 0x003, 0x0c1, 0x2c7, 0x0c1, 0x2c8, 0x086, 0x189, 0x395,
+ 0x104, 0x0a2, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003,
+ 0x3c5, 0x100, 0x00d, 0x044, 0x30e, 0x002, 0x347, 0x00e,
+ 0x32e, 0x347, 0x00f, 0x32f, 0x045, 0x307, 0x000, 0x183,
+ 0x247, 0x21b, 0x003, 0x00b, 0x347, 0x21e, 0x008, 0x307,
+ 0x006, 0x327, 0x007, 0x000, 0x009, 0x347, 0x21e, 0x00b,
+ 0x307, 0x009, 0x327, 0x00a, 0x020, 0x1bb, 0x006, 0x347,
+ 0x059, 0x24b, 0x347, 0x248, 0x24a, 0x044, 0x340, 0x001,
+ 0x247, 0x248, 0x054, 0x364, 0x001, 0x080, 0x009, 0x059,
+ 0x347, 0x24a, 0x248, 0x364, 0x001, 0x001, 0x009, 0x171,
+ 0x364, 0x001, 0x002, 0x009, 0x171, 0x364, 0x001, 0x004,
+ 0x009, 0x17c, 0x364, 0x001, 0x008, 0x009, 0x1a4, 0x364,
+ 0x001, 0x010, 0x029, 0x042, 0x008, 0x364, 0x001, 0x020,
+ 0x029, 0x01f, 0x008, 0x364, 0x001, 0x040, 0x009, 0x1b1,
+ 0x364, 0x001, 0x100, 0x009, 0x1ac, 0x327, 0x21c, 0x324,
+ 0x020, 0x029, 0x0ab, 0x006, 0x364, 0x23f, 0x003, 0x001,
+ 0x00c, 0x044, 0x340, 0x001, 0x227, 0x1ba, 0x326, 0x248,
+ 0x054, 0x009, 0x026, 0x307, 0x24b, 0x306, 0x059, 0x384,
+ 0x080, 0x009, 0x01b, 0x055, 0x142, 0x000, 0x047, 0x3c5,
+ 0x020, 0x000, 0x3c5, 0x001, 0x233, 0x000, 0x012, 0x364,
+ 0x233, 0x001, 0x001, 0x00a, 0x3c5, 0x020, 0x000, 0x347,
+ 0x24a, 0x248, 0x000, 0x3e9, 0x044, 0x340, 0x001, 0x247,
+ 0x248, 0x347, 0x059, 0x24b, 0x044, 0x34c, 0x001, 0x009,
+ 0x012, 0x364, 0x33c, 0x020, 0x001, 0x007, 0x364, 0x24b,
+ 0x080, 0x009, 0x008, 0x306, 0x249, 0x19a, 0x001, 0x3cd,
+ 0x054, 0x364, 0x23f, 0x002, 0x001, 0x052, 0x307, 0x248,
+ 0x19e, 0x00a, 0x01c, 0x3c5, 0x084, 0x021, 0x3c4, 0x3fe,
+ 0x026, 0x307, 0x248, 0x306, 0x249, 0x209, 0x00b, 0x022,
+ 0x3b7, 0x210, 0x044, 0x192, 0x000, 0x247, 0x257, 0x080,
+ 0x044, 0x1bd, 0x000, 0x000, 0x015, 0x364, 0x026, 0x001,
+ 0x009, 0x010, 0x3c4, 0x37b, 0x021, 0x364, 0x234, 0x001,
+ 0x001, 0x008, 0x3c5, 0x001, 0x026, 0x044, 0x342, 0x006,
+ 0x307, 0x248, 0x19d, 0x00a, 0x007, 0x3c4, 0x3fd, 0x026,
+ 0x000, 0x00d, 0x364, 0x234, 0x002, 0x001, 0x008, 0x3c5,
+ 0x002, 0x026, 0x044, 0x141, 0x001, 0x327, 0x248, 0x326,
+ 0x249, 0x00b, 0x005, 0x044, 0x27d, 0x001, 0x364, 0x23f,
+ 0x001, 0x001, 0x013, 0x044, 0x34c, 0x001, 0x001, 0x007,
+ 0x2a8, 0x2c4, 0x233, 0x000, 0x00b, 0x327, 0x248, 0x326,
+ 0x249, 0x1ba, 0x3a4, 0x003, 0x001, 0x01f, 0x327, 0x248,
+ 0x1ba, 0x287, 0x190, 0x002, 0x006, 0x3a4, 0x3fb, 0x000,
+ 0x004, 0x3a5, 0x004, 0x3a4, 0x007, 0x307, 0x234, 0x384,
+ 0x3f8, 0x225, 0x344, 0x21f, 0x232, 0x347, 0x248, 0x249,
+ 0x020, 0x022, 0x008, 0x347, 0x248, 0x249, 0x364, 0x33c,
+ 0x020, 0x001, 0x005, 0x044, 0x12d, 0x005, 0x364, 0x233,
+ 0x001, 0x001, 0x314, 0x3c4, 0x3fe, 0x233, 0x020, 0x0f4,
+ 0x000, 0x328, 0x21b, 0x044, 0x32d, 0x001, 0x043, 0x307,
+ 0x21b, 0x009, 0x00e, 0x2c7, 0x005, 0x144, 0x000, 0x364,
+ 0x233, 0x001, 0x001, 0x2fb, 0x020, 0x0a3, 0x006, 0x00b,
+ 0x009, 0x0c0, 0x21b, 0x2c7, 0x00b, 0x148, 0x000, 0x047,
+ 0x0c0, 0x21b, 0x2c7, 0x008, 0x141, 0x000, 0x047, 0x3c4,
+ 0x3df, 0x000, 0x364, 0x000, 0x100, 0x001, 0x2a2, 0x364,
+ 0x233, 0x001, 0x001, 0x005, 0x3c5, 0x020, 0x000, 0x307,
+ 0x003, 0x327, 0x004, 0x041, 0x044, 0x32c, 0x001, 0x043,
+ 0x044, 0x32d, 0x001, 0x040, 0x209, 0x002, 0x083, 0x003,
+ 0x089, 0x188, 0x199, 0x041, 0x229, 0x280, 0x043, 0x390,
+ 0x057, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x3c4,
+ 0x3f3, 0x012, 0x000, 0x2b3, 0x0af, 0x1a7, 0x2a8, 0x2c4,
+ 0x01f, 0x3c5, 0x001, 0x200, 0x344, 0x21f, 0x232, 0x151,
+ 0x2fa, 0x020, 0x247, 0x002, 0x364, 0x03a, 0x040, 0x009,
+ 0x007, 0x3c4, 0x3f7, 0x03a, 0x000, 0x299, 0x3c4, 0x3f0,
+ 0x03a, 0x364, 0x2cf, 0x001, 0x001, 0x01b, 0x364, 0x2cf,
+ 0x002, 0x009, 0x016, 0x142, 0x03a, 0x141, 0x2e6, 0x3c1,
+ 0x000, 0x2e7, 0x32a, 0x03a, 0x2f4, 0x200, 0x001, 0x3fc,
+ 0x143, 0x03a, 0x374, 0x25e, 0x200, 0x009, 0x002, 0x000,
+ 0x276, 0x364, 0x03d, 0x040, 0x009, 0x007, 0x3c4, 0x3f7,
+ 0x03d, 0x000, 0x26c, 0x3c4, 0x3f0, 0x03d, 0x364, 0x33b,
+ 0x001, 0x001, 0x007, 0x044, 0x3a3, 0x003, 0x000, 0x25f,
+ 0x374, 0x25f, 0x200, 0x009, 0x002, 0x000, 0x258, 0x158,
+ 0x025, 0x044, 0x141, 0x001, 0x020, 0x0a3, 0x006, 0x042,
+ 0x3a7, 0x020, 0x1a7, 0x3a5, 0x055, 0x020, 0x1de, 0x007,
+ 0x042, 0x394, 0x3ff, 0x04d, 0x040, 0x020, 0x181, 0x006,
+ 0x042, 0x394, 0x3ff, 0x04c, 0x020, 0x1de, 0x007, 0x364,
+ 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x042, 0x186, 0x003,
+ 0x06b, 0x002, 0x06d, 0x2e4, 0x001, 0x001, 0x059, 0x287,
+ 0x384, 0x00e, 0x190, 0x3a4, 0x370, 0x285, 0x041, 0x0a0,
+ 0x044, 0x18a, 0x002, 0x043, 0x1b3, 0x3c4, 0x3fd, 0x00c,
+ 0x3c4, 0x3fc, 0x018, 0x287, 0x384, 0x070, 0x382, 0x030,
+ 0x001, 0x007, 0x3a4, 0x007, 0x133, 0x009, 0x007, 0x3c4,
+ 0x3ef, 0x021, 0x141, 0x018, 0x142, 0x00c, 0x0a1, 0x1aa,
+ 0x364, 0x026, 0x004, 0x009, 0x004, 0x2c5, 0x02a, 0x2a8,
+ 0x082, 0x044, 0x189, 0x000, 0x2c4, 0x02a, 0x3c4, 0x3f7,
+ 0x00d, 0x3c5, 0x001, 0x00d, 0x3c4, 0x3fe, 0x00d, 0x140,
+ 0x013, 0x009, 0x00b, 0x140, 0x014, 0x009, 0x007, 0x3c5,
+ 0x008, 0x00d, 0x000, 0x014, 0x0c0, 0x012, 0x0c2, 0x012,
+ 0x0c0, 0x012, 0x0c5, 0x012, 0x000, 0x00a, 0x0c0, 0x00d,
+ 0x044, 0x33f, 0x002, 0x3c4, 0x3fd, 0x00c, 0x0a3, 0x020,
+ 0x1de, 0x007, 0x2c7, 0x013, 0x000, 0x3fa, 0x2c7, 0x014,
+ 0x000, 0x3f6, 0x020, 0x181, 0x006, 0x2e4, 0x100, 0x009,
+ 0x042, 0x041, 0x3a4, 0x007, 0x387, 0x006, 0x189, 0x395,
+ 0x321, 0x280, 0x050, 0x0c3, 0x265, 0x262, 0x005, 0x00b,
+ 0x004, 0x0c7, 0x265, 0x043, 0x1b3, 0x2e2, 0x003, 0x009,
+ 0x00b, 0x041, 0x040, 0x084, 0x0b1, 0x044, 0x1b7, 0x000,
+ 0x042, 0x043, 0x2a9, 0x349, 0x265, 0x265, 0x111, 0x00b,
+ 0x3fb, 0x2aa, 0x34a, 0x265, 0x265, 0x348, 0x265, 0x265,
+ 0x041, 0x3a7, 0x030, 0x044, 0x192, 0x000, 0x043, 0x304,
+ 0x265, 0x285, 0x3a7, 0x030, 0x044, 0x1bd, 0x000, 0x000,
+ 0x007, 0x084, 0x0b1, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x000, 0x006, 0x009, 0x002, 0x004, 0x2c7, 0x22c,
+ 0x2f4, 0x100, 0x001, 0x013, 0x044, 0x342, 0x006, 0x364,
+ 0x23f, 0x002, 0x009, 0x005, 0x020, 0x181, 0x006, 0x345,
+ 0x21d, 0x232, 0x020, 0x2ea, 0x002, 0x044, 0x1ef, 0x001,
+ 0x000, 0x3ef, 0x044, 0x1ef, 0x001, 0x091, 0x044, 0x189,
+ 0x000, 0x3b7, 0x210, 0x307, 0x257, 0x020, 0x1bd, 0x000,
+ 0x1ad, 0x1b9, 0x3c4, 0x3cf, 0x232, 0x2c5, 0x232, 0x020,
+ 0x181, 0x006, 0x287, 0x2e4, 0x100, 0x001, 0x016, 0x041,
+ 0x384, 0x001, 0x182, 0x040, 0x3b7, 0x211, 0x044, 0x1a9,
+ 0x000, 0x042, 0x385, 0x3f3, 0x3b7, 0x211, 0x044, 0x1b8,
+ 0x000, 0x043, 0x287, 0x2f4, 0x200, 0x001, 0x016, 0x384,
+ 0x010, 0x190, 0x227, 0x190, 0x285, 0x040, 0x3a7, 0x060,
+ 0x044, 0x1a9, 0x000, 0x042, 0x385, 0x3f3, 0x3a7, 0x060,
+ 0x044, 0x1b8, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x100,
+ 0x009, 0x059, 0x287, 0x384, 0x0ff, 0x264, 0x080, 0x001,
+ 0x004, 0x385, 0x300, 0x1bd, 0x3a4, 0x003, 0x009, 0x004,
+ 0x247, 0x23a, 0x131, 0x009, 0x006, 0x247, 0x23c, 0x003,
+ 0x036, 0x131, 0x009, 0x004, 0x247, 0x23b, 0x131, 0x009,
+ 0x00a, 0x247, 0x23a, 0x003, 0x02a, 0x247, 0x23c, 0x247,
+ 0x23b, 0x327, 0x23a, 0x00b, 0x004, 0x2a8, 0x121, 0x140,
+ 0x23b, 0x00b, 0x006, 0x322, 0x23b, 0x000, 0x004, 0x320,
+ 0x23b, 0x320, 0x23c, 0x322, 0x23e, 0x007, 0x010, 0x347,
+ 0x23a, 0x237, 0x347, 0x23b, 0x239, 0x347, 0x23c, 0x238,
+ 0x044, 0x05b, 0x001, 0x000, 0x038, 0x347, 0x237, 0x23a,
+ 0x347, 0x239, 0x23b, 0x347, 0x238, 0x23c, 0x020, 0x1dc,
+ 0x007, 0x140, 0x33b, 0x009, 0x015, 0x289, 0x002, 0x00c,
+ 0x003, 0x006, 0x327, 0x237, 0x000, 0x008, 0x327, 0x238,
+ 0x000, 0x004, 0x327, 0x239, 0x042, 0x020, 0x1de, 0x007,
+ 0x289, 0x002, 0x00c, 0x003, 0x006, 0x327, 0x345, 0x000,
+ 0x3f5, 0x327, 0x346, 0x000, 0x3f1, 0x327, 0x347, 0x000,
+ 0x3ed, 0x2c7, 0x01a, 0x020, 0x181, 0x006, 0x327, 0x00c,
+ 0x3c5, 0x002, 0x00c, 0x3c5, 0x001, 0x00d, 0x08f, 0x044,
+ 0x189, 0x000, 0x3c4, 0x3fe, 0x00d, 0x2c7, 0x00c, 0x020,
+ 0x181, 0x006, 0x130, 0x001, 0x013, 0x083, 0x044, 0x189,
+ 0x000, 0x3c6, 0x001, 0x02b, 0x083, 0x044, 0x189, 0x000,
+ 0x3c6, 0x001, 0x02b, 0x131, 0x009, 0x3f1, 0x020, 0x181,
+ 0x006, 0x327, 0x00c, 0x3c5, 0x002, 0x00c, 0x387, 0x07c,
+ 0x187, 0x385, 0x08f, 0x244, 0x02a, 0x088, 0x187, 0x107,
+ 0x245, 0x02a, 0x082, 0x044, 0x189, 0x000, 0x081, 0x18a,
+ 0x208, 0x244, 0x02a, 0x2c7, 0x00c, 0x020, 0x181, 0x006,
+ 0x364, 0x026, 0x002, 0x001, 0x007, 0x364, 0x012, 0x040,
+ 0x001, 0x3f8, 0x020, 0x181, 0x006, 0x042, 0x327, 0x00d,
+ 0x000, 0x16e, 0x042, 0x327, 0x012, 0x000, 0x169, 0x042,
+ 0x197, 0x264, 0x001, 0x009, 0x011, 0x264, 0x002, 0x009,
+ 0x015, 0x264, 0x004, 0x009, 0x017, 0x264, 0x010, 0x009,
+ 0x01b, 0x0a0, 0x000, 0x154, 0x325, 0x21d, 0x2c7, 0x054,
+ 0x384, 0x3fe, 0x000, 0x3e7, 0x0c0, 0x054, 0x384, 0x3fd,
+ 0x000, 0x3e1, 0x324, 0x21f, 0x2c7, 0x054, 0x384, 0x3fb,
+ 0x000, 0x3d9, 0x044, 0x048, 0x008, 0x3c6, 0x100, 0x02b,
+ 0x0c0, 0x000, 0x364, 0x000, 0x020, 0x009, 0x3f5, 0x000,
+ 0x3db, 0x042, 0x040, 0x197, 0x001, 0x00d, 0x111, 0x001,
+ 0x028, 0x111, 0x001, 0x029, 0x111, 0x001, 0x02a, 0x000,
+ 0x041, 0x2c7, 0x254, 0x3b7, 0x241, 0x387, 0x100, 0x044,
+ 0x1a9, 0x000, 0x121, 0x044, 0x192, 0x000, 0x384, 0x030,
+ 0x040, 0x132, 0x044, 0x192, 0x000, 0x3b8, 0x300, 0x284,
+ 0x043, 0x1a3, 0x285, 0x395, 0x080, 0x000, 0x01e, 0x2c7,
+ 0x254, 0x000, 0x01f, 0x2c7, 0x255, 0x000, 0x01b, 0x287,
+ 0x040, 0x3b7, 0x241, 0x044, 0x1bd, 0x000, 0x131, 0x044,
+ 0x192, 0x000, 0x3a7, 0x061, 0x1a6, 0x2a8, 0x284, 0x043,
+ 0x3a4, 0x200, 0x285, 0x3b7, 0x240, 0x044, 0x1bd, 0x000,
+ 0x020, 0x181, 0x006, 0x362, 0x391, 0x001, 0x021, 0x181,
+ 0x006, 0x0c1, 0x391, 0x042, 0x0c0, 0x38f, 0x0c0, 0x390,
+ 0x080, 0x051, 0x2c0, 0x38f, 0x3c1, 0x000, 0x390, 0x3c6,
+ 0x0d8, 0x38f, 0x349, 0x38f, 0x38f, 0x34b, 0x390, 0x390,
+ 0x00a, 0x008, 0x3c0, 0x0e5, 0x38f, 0x3c1, 0x000, 0x390,
+ 0x3b7, 0x00b, 0x1a9, 0x3b5, 0x382, 0x222, 0x001, 0x005,
+ 0x101, 0x000, 0x3e0, 0x101, 0x051, 0x326, 0x38f, 0x3a4,
+ 0x0ff, 0x2e2, 0x0ef, 0x009, 0x029, 0x101, 0x051, 0x040,
+ 0x307, 0x38f, 0x197, 0x226, 0x2e2, 0x0be, 0x009, 0x01e,
+ 0x042, 0x101, 0x051, 0x326, 0x390, 0x3a4, 0x0ff, 0x2e2,
+ 0x0ad, 0x009, 0x013, 0x101, 0x050, 0x327, 0x390, 0x1b7,
+ 0x286, 0x262, 0x0de, 0x009, 0x009, 0x080, 0x040, 0x0c0,
+ 0x391, 0x020, 0x181, 0x006, 0x387, 0x0ff, 0x000, 0x3f8,
+ 0x043, 0x0a0, 0x1a7, 0x3a5, 0x045, 0x000, 0x071, 0x345,
+ 0x221, 0x233, 0x28a, 0x00a, 0x00c, 0x3a4, 0x300, 0x3c4,
+ 0x0ff, 0x027, 0x2c5, 0x027, 0x020, 0x181, 0x006, 0x327,
+ 0x059, 0x1a7, 0x042, 0x000, 0x05b, 0x2e4, 0x020, 0x001,
+ 0x012, 0x081, 0x18c, 0x245, 0x233, 0x2e4, 0x010, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x044, 0x11a, 0x001, 0x000,
+ 0x042, 0x041, 0x091, 0x189, 0x2e4, 0x002, 0x001, 0x015,
+ 0x3c5, 0x040, 0x233, 0x245, 0x024, 0x364, 0x026, 0x002,
+ 0x001, 0x01d, 0x3a7, 0x022, 0x397, 0x200, 0x044, 0x1b7,
+ 0x000, 0x000, 0x014, 0x3c4, 0x3bf, 0x233, 0x364, 0x026,
+ 0x002, 0x001, 0x00c, 0x208, 0x244, 0x024, 0x3a7, 0x022,
+ 0x397, 0x200, 0x044, 0x1a9, 0x000, 0x043, 0x081, 0x189,
+ 0x2e4, 0x001, 0x001, 0x009, 0x3a7, 0x020, 0x044, 0x1a9,
+ 0x000, 0x000, 0x008, 0x3a7, 0x020, 0x208, 0x044, 0x1b8,
+ 0x000, 0x020, 0x181, 0x006, 0x043, 0x0a0, 0x041, 0x020,
+ 0x181, 0x006, 0x2e2, 0x00f, 0x001, 0x060, 0x307, 0x2cf,
+ 0x0c0, 0x2d3, 0x120, 0x001, 0x05f, 0x00b, 0x018, 0x0c1,
+ 0x2d3, 0x3c4, 0x00f, 0x2cf, 0x3c5, 0x100, 0x200, 0x000,
+ 0x048, 0x0c0, 0x2d8, 0x3a4, 0x3f7, 0x001, 0x042, 0x0c1,
+ 0x2d8, 0x2c7, 0x2da, 0x000, 0x03c, 0x2e4, 0x008, 0x009,
+ 0x3f2, 0x2e2, 0x004, 0x009, 0x021, 0x364, 0x026, 0x002,
+ 0x001, 0x3cc, 0x364, 0x027, 0x010, 0x001, 0x3c7, 0x140,
+ 0x2f9, 0x009, 0x3c3, 0x140, 0x2d8, 0x001, 0x3bf, 0x327,
+ 0x2da, 0x364, 0x026, 0x002, 0x001, 0x3b8, 0x364, 0x2cf,
+ 0x073, 0x029, 0x1dc, 0x007, 0x3c4, 0x08f, 0x2cf, 0x2e4,
+ 0x003, 0x001, 0x010, 0x287, 0x1b5, 0x2c7, 0x2d4, 0x384,
+ 0x007, 0x183, 0x245, 0x2cf, 0x3c5, 0x040, 0x200, 0x0c0,
+ 0x2f9, 0x020, 0x181, 0x006, 0x30a, 0x2cf, 0x384, 0x020,
+ 0x000, 0x3f2, 0x347, 0x2cf, 0x2f9, 0x3c4, 0x3df, 0x2cf,
+ 0x000, 0x3f1, 0x042, 0x2c7, 0x269, 0x287, 0x384, 0x0ff,
+ 0x1a1, 0x002, 0x00b, 0x00b, 0x014, 0x247, 0x28d, 0x044,
+ 0x2fb, 0x002, 0x000, 0x005, 0x044, 0x125, 0x003, 0x140,
+ 0x269, 0x00b, 0x01b, 0x0a1, 0x044, 0x30f, 0x002, 0x140,
+ 0x269, 0x00b, 0x013, 0x364, 0x012, 0x040, 0x001, 0x3fd,
+ 0x347, 0x00e, 0x266, 0x347, 0x00f, 0x267, 0x397, 0x200,
+ 0x0a1, 0x044, 0x0fb, 0x002, 0x347, 0x28d, 0x268, 0x307,
+ 0x269, 0x197, 0x384, 0x003, 0x390, 0x266, 0x04c, 0x041,
+ 0x020, 0x181, 0x006, 0x042, 0x247, 0x265, 0x041, 0x397,
+ 0x007, 0x189, 0x395, 0x2ca, 0x1bb, 0x3a4, 0x007, 0x280,
+ 0x050, 0x043, 0x041, 0x1b7, 0x3a4, 0x00f, 0x280, 0x043,
+ 0x364, 0x265, 0x100, 0x009, 0x01b, 0x120, 0x00b, 0x008,
+ 0x3a4, 0x0ff, 0x2a8, 0x121, 0x000, 0x004, 0x3a4, 0x0ff,
+ 0x04d, 0x041, 0x044, 0x3f0, 0x002, 0x044, 0x3de, 0x002,
+ 0x044, 0x06e, 0x003, 0x020, 0x181, 0x006, 0x04c, 0x041,
+ 0x000, 0x3fb, 0x2c4, 0x2a1, 0x28e, 0x293, 0x2b2, 0x2d4,
+ 0x2e4, 0x347, 0x22e, 0x22f, 0x2c7, 0x22e, 0x2e4, 0x040,
+ 0x001, 0x01a, 0x3a7, 0x080, 0x044, 0x192, 0x000, 0x384,
+ 0x3c3, 0x327, 0x22e, 0x1b5, 0x3a4, 0x03c, 0x285, 0x3a7,
+ 0x080, 0x044, 0x1bd, 0x000, 0x347, 0x22f, 0x22e, 0x020,
+ 0x181, 0x006, 0x3c5, 0x004, 0x233, 0x364, 0x22e, 0x001,
+ 0x009, 0x005, 0x3c4, 0x3fb, 0x233, 0x364, 0x22e, 0x020,
+ 0x009, 0x005, 0x080, 0x000, 0x003, 0x081, 0x044, 0x254,
+ 0x001, 0x364, 0x22e, 0x001, 0x001, 0x007, 0x0c8, 0x291,
+ 0x044, 0x06e, 0x003, 0x307, 0x22e, 0x20a, 0x384, 0x001,
+ 0x044, 0x045, 0x008, 0x020, 0x181, 0x006, 0x2c7, 0x223,
+ 0x0c0, 0x224, 0x000, 0x01e, 0x309, 0x223, 0x002, 0x01d,
+ 0x003, 0x020, 0x001, 0x030, 0x362, 0x224, 0x001, 0x003,
+ 0x011, 0x112, 0x001, 0x01f, 0x112, 0x001, 0x057, 0x116,
+ 0x001, 0x06a, 0x114, 0x001, 0x055, 0x112, 0x001, 0x060,
+ 0x020, 0x181, 0x006, 0x20e, 0x384, 0x0ff, 0x000, 0x055,
+ 0x041, 0x229, 0x1b1, 0x042, 0x044, 0x1bd, 0x000, 0x000,
+ 0x3f1, 0x344, 0x21f, 0x02a, 0x083, 0x18b, 0x245, 0x02a,
+ 0x000, 0x005, 0x344, 0x21f, 0x029, 0x3b4, 0x3ff, 0x307,
+ 0x224, 0x262, 0x007, 0x00e, 0x3dd, 0x20a, 0x183, 0x247,
+ 0x02d, 0x30a, 0x224, 0x002, 0x006, 0x2c7, 0x02c, 0x000,
+ 0x009, 0x287, 0x189, 0x245, 0x02c, 0x1b5, 0x2c5, 0x02d,
+ 0x307, 0x223, 0x001, 0x00a, 0x3c5, 0x080, 0x02d, 0x3c4,
+ 0x37f, 0x02d, 0x000, 0x030, 0x3c5, 0x040, 0x02d, 0x3c4,
+ 0x3bf, 0x02d, 0x000, 0x028, 0x397, 0x013, 0x000, 0x004,
+ 0x397, 0x2c7, 0x300, 0x224, 0x140, 0x224, 0x009, 0x005,
+ 0x04d, 0x101, 0x0a0, 0x04d, 0x000, 0x016, 0x2c7, 0x2cb,
+ 0x000, 0x3a0, 0x140, 0x224, 0x009, 0x006, 0x2c7, 0x019,
+ 0x000, 0x00a, 0x08f, 0x18b, 0x208, 0x244, 0x018, 0x1ab,
+ 0x2c5, 0x018, 0x141, 0x224, 0x000, 0x38c, 0x042, 0x309,
+ 0x223, 0x002, 0x03e, 0x003, 0x043, 0x116, 0x001, 0x049,
+ 0x112, 0x001, 0x00e, 0x114, 0x029, 0x1dd, 0x007, 0x397,
+ 0x200, 0x300, 0x224, 0x04c, 0x041, 0x000, 0x3e5, 0x307,
+ 0x224, 0x262, 0x007, 0x02e, 0x1dd, 0x007, 0x364, 0x026,
+ 0x004, 0x029, 0x1dd, 0x007, 0x20a, 0x187, 0x0a3, 0x1a7,
+ 0x2a8, 0x2c4, 0x02a, 0x245, 0x02a, 0x327, 0x02e, 0x307,
+ 0x224, 0x20a, 0x00a, 0x009, 0x1b9, 0x307, 0x02f, 0x384,
+ 0x00f, 0x185, 0x225, 0x3b4, 0x3ff, 0x000, 0x022, 0x20a,
+ 0x384, 0x0ff, 0x04c, 0x041, 0x000, 0x344, 0x304, 0x21f,
+ 0x22e, 0x044, 0x192, 0x000, 0x040, 0x000, 0x33b, 0x30a,
+ 0x224, 0x002, 0x011, 0x307, 0x00d, 0x3c4, 0x3f7, 0x00d,
+ 0x347, 0x00f, 0x26a, 0x327, 0x00e, 0x247, 0x00d, 0x041,
+ 0x000, 0x39a, 0x327, 0x26a, 0x000, 0x3fb, 0x04e, 0x020,
+ 0x23b, 0x009, 0x020, 0x28e, 0x00a, 0x020, 0x03a, 0x00a,
+ 0x020, 0x0fe, 0x00a, 0x020, 0x3f3, 0x008, 0x020, 0x013,
+ 0x009, 0x020, 0x075, 0x00a, 0x0c1, 0x244, 0x3c5, 0x040,
+ 0x245, 0x3c4, 0x3bf, 0x200, 0x044, 0x172, 0x000, 0x020,
+ 0x0d3, 0x000, 0x020, 0x2c3, 0x008, 0x020, 0x375, 0x00b,
+ 0x0c1, 0x201, 0x045, 0x020, 0x04a, 0x009, 0x020, 0x04e,
+ 0x009, 0x020, 0x256, 0x002, 0x020, 0x17f, 0x009, 0x020,
+ 0x23f, 0x006, 0x020, 0x285, 0x00a, 0x020, 0x209, 0x00b,
+ 0x020, 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x394,
+ 0x00a, 0x020, 0x048, 0x00b, 0x020, 0x1dc, 0x007, 0x020,
+ 0x27f, 0x00a, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007,
+ 0x020, 0x295, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x277,
+ 0x00b, 0x020, 0x2ba, 0x00b, 0x020, 0x2d5, 0x006, 0x020,
+ 0x326, 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007,
+ 0x020, 0x35a, 0x006, 0x020, 0x0a4, 0x00b, 0x020, 0x38e,
+ 0x006, 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020,
+ 0x31e, 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007,
+ 0x020, 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060,
+ 0x007, 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020,
+ 0x2d1, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252,
+ 0x007, 0x020, 0x293, 0x007, 0x020, 0x2d8, 0x00b, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007,
+ 0x020, 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x30e, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007,
+ 0x020, 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5,
+ 0x008, 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287,
+ 0x264, 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5,
+ 0x020, 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004,
+ 0x3a4, 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009,
+ 0x093, 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff,
+ 0x053, 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe,
+ 0x052, 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de,
+ 0x007, 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c,
+ 0x100, 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c,
+ 0x3d5, 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066,
+ 0x004, 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b,
+ 0x262, 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4,
+ 0x3ef, 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009,
+ 0x007, 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200,
+ 0x004, 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001,
+ 0x028, 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344,
+ 0x21f, 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183,
+ 0x19b, 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001,
+ 0x004, 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5,
+ 0x080, 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044,
+ 0x001, 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5,
+ 0x004, 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044,
+ 0x097, 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d,
+ 0x001, 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390,
+ 0x1d7, 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000,
+ 0x021, 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000,
+ 0x025, 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000,
+ 0x041, 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000,
+ 0x031, 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007,
+ 0x044, 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072,
+ 0x2c7, 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a,
+ 0x2c7, 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062,
+ 0x2c7, 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a,
+ 0x2c7, 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052,
+ 0x2c7, 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a,
+ 0x307, 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007,
+ 0x006, 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020,
+ 0x1de, 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3,
+ 0x052, 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c,
+ 0x2c5, 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7,
+ 0x3ff, 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf,
+ 0x354, 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222,
+ 0x355, 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a,
+ 0x006, 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357,
+ 0x020, 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4,
+ 0x00f, 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020,
+ 0x131, 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001,
+ 0x025, 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132,
+ 0x182, 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de,
+ 0x007, 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373,
+ 0x000, 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3,
+ 0x322, 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7,
+ 0x327, 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df,
+ 0x307, 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108,
+ 0x04c, 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b,
+ 0x364, 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f,
+ 0x181, 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5,
+ 0x080, 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7,
+ 0x392, 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5,
+ 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364,
+ 0x33c, 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000,
+ 0x018, 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140,
+ 0x04f, 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3,
+ 0x2c5, 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081,
+ 0x189, 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027,
+ 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053,
+ 0x3c4, 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a,
+ 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004,
+ 0x020, 0x08d, 0x006, 0x2c7, 0x231, 0x2e4, 0x100, 0x001,
+ 0x045, 0x2e4, 0x001, 0x001, 0x020, 0x3b7, 0x213, 0x084,
+ 0x044, 0x1a9, 0x000, 0x3b7, 0x250, 0x084, 0x044, 0x1a9,
+ 0x000, 0x399, 0x305, 0x3a7, 0x022, 0x044, 0x1a9, 0x000,
+ 0x044, 0x322, 0x001, 0x3c5, 0x002, 0x024, 0x3c5, 0x008,
+ 0x292, 0x000, 0x020, 0x3c4, 0x3fd, 0x024, 0x364, 0x234,
+ 0x002, 0x001, 0x009, 0x3a7, 0x022, 0x399, 0x305, 0x044,
+ 0x1b7, 0x000, 0x3b7, 0x213, 0x084, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x250, 0x084, 0x044, 0x1b7, 0x000, 0x3c4, 0x007,
+ 0x292, 0x044, 0x06e, 0x003, 0x307, 0x231, 0x274, 0x200,
+ 0x001, 0x01b, 0x274, 0x010, 0x001, 0x00b, 0x387, 0x020,
+ 0x245, 0x021, 0x185, 0x245, 0x024, 0x000, 0x00b, 0x387,
+ 0x3df, 0x244, 0x021, 0x185, 0x385, 0x0ff, 0x244, 0x024,
+ 0x044, 0x03a, 0x00a, 0x045, 0x327, 0x230, 0x364, 0x233,
+ 0x004, 0x001, 0x003, 0x045, 0x0c2, 0x22a, 0x347, 0x275,
+ 0x270, 0x0c0, 0x27b, 0x264, 0x010, 0x009, 0x00a, 0x0c1,
+ 0x22a, 0x347, 0x274, 0x270, 0x3d7, 0x200, 0x27b, 0x327,
+ 0x230, 0x264, 0x004, 0x001, 0x012, 0x347, 0x277, 0x270,
+ 0x264, 0x010, 0x009, 0x007, 0x347, 0x276, 0x270, 0x384,
+ 0x00f, 0x112, 0x1b7, 0x000, 0x004, 0x3a4, 0x0ff, 0x2c7,
+ 0x225, 0x384, 0x00f, 0x262, 0x003, 0x001, 0x00b, 0x101,
+ 0x3c2, 0x100, 0x270, 0x34a, 0x225, 0x225, 0x000, 0x3f3,
+ 0x044, 0x237, 0x00a, 0x044, 0x35f, 0x009, 0x364, 0x22a,
+ 0x002, 0x001, 0x006, 0x347, 0x270, 0x272, 0x045, 0x347,
+ 0x270, 0x271, 0x045, 0x044, 0x2f6, 0x002, 0x009, 0x015,
+ 0x307, 0x241, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x3d7, 0x200, 0x27b,
+ 0x044, 0x35f, 0x009, 0x347, 0x241, 0x226, 0x0c1, 0x22a,
+ 0x020, 0x032, 0x009, 0x044, 0x2f6, 0x002, 0x009, 0x014,
+ 0x307, 0x243, 0x197, 0x384, 0x007, 0x001, 0x008, 0x3c2,
+ 0x100, 0x270, 0x111, 0x009, 0x3fc, 0x0c0, 0x27b, 0x044,
+ 0x35f, 0x009, 0x347, 0x243, 0x226, 0x0c2, 0x22a, 0x020,
+ 0x032, 0x009, 0x327, 0x226, 0x2c7, 0x225, 0x3c4, 0x0ff,
+ 0x225, 0x1b7, 0x3a4, 0x007, 0x001, 0x00d, 0x34a, 0x225,
+ 0x225, 0x002, 0x008, 0x131, 0x009, 0x3fa, 0x044, 0x237,
+ 0x00a, 0x045, 0x0c1, 0x26f, 0x000, 0x050, 0x327, 0x01d,
+ 0x2e4, 0x380, 0x009, 0x00a, 0x13a, 0x00b, 0x00a, 0x3c5,
+ 0x008, 0x233, 0x000, 0x005, 0x3c4, 0x3f7, 0x233, 0x327,
+ 0x01d, 0x021, 0x2ed, 0x009, 0x364, 0x020, 0x004, 0x009,
+ 0x01a, 0x309, 0x232, 0x00b, 0x009, 0x347, 0x270, 0x271,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x271, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x04c, 0x347, 0x270, 0x271, 0x000,
+ 0x018, 0x309, 0x232, 0x001, 0x009, 0x347, 0x270, 0x272,
+ 0x308, 0x221, 0x244, 0x232, 0x347, 0x272, 0x270, 0x044,
+ 0x270, 0x009, 0x001, 0x089, 0x347, 0x270, 0x272, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x364, 0x26f, 0x002, 0x009,
+ 0x005, 0x3c4, 0x3fe, 0x200, 0x364, 0x020, 0x005, 0x029,
+ 0x2e4, 0x002, 0x364, 0x020, 0x010, 0x009, 0x0e9, 0x364,
+ 0x020, 0x100, 0x009, 0x12b, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3f7, 0x233, 0x364, 0x020, 0x001, 0x009, 0x00c, 0x364,
+ 0x020, 0x004, 0x009, 0x05d, 0x000, 0x3d8, 0x3c5, 0x080,
+ 0x020, 0x364, 0x233, 0x006, 0x001, 0x031, 0x081, 0x189,
+ 0x304, 0x232, 0x009, 0x017, 0x347, 0x271, 0x274, 0x080,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x085, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245, 0x232, 0x000,
+ 0x0c8, 0x347, 0x271, 0x276, 0x081, 0x189, 0x208, 0x244,
+ 0x232, 0x307, 0x22b, 0x197, 0x384, 0x007, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x000, 0x005, 0x044, 0x3f3, 0x008,
+ 0x3b7, 0x200, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x203, 0x387, 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020,
+ 0x385, 0x042, 0x384, 0x3ce, 0x247, 0x020, 0x044, 0x044,
+ 0x001, 0x000, 0x383, 0x0a1, 0x1aa, 0x2c5, 0x020, 0x364,
+ 0x233, 0x006, 0x001, 0x039, 0x081, 0x189, 0x304, 0x232,
+ 0x009, 0x017, 0x347, 0x272, 0x275, 0x080, 0x364, 0x233,
+ 0x004, 0x009, 0x003, 0x095, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x081, 0x189, 0x245, 0x232, 0x000, 0x0ce, 0x347,
+ 0x272, 0x277, 0x081, 0x189, 0x208, 0x244, 0x232, 0x307,
+ 0x22b, 0x197, 0x364, 0x233, 0x002, 0x001, 0x007, 0x193,
+ 0x384, 0x007, 0x385, 0x010, 0x0c2, 0x22a, 0x044, 0x39c,
+ 0x008, 0x000, 0x008, 0x347, 0x272, 0x270, 0x044, 0x013,
+ 0x009, 0x081, 0x18a, 0x208, 0x244, 0x232, 0x0a3, 0x387,
+ 0x3f7, 0x044, 0x1b8, 0x000, 0x307, 0x020, 0x0a4, 0x1a7,
+ 0x3a5, 0x008, 0x285, 0x3b7, 0x304, 0x2a8, 0x284, 0x247,
+ 0x020, 0x0a0, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x044,
+ 0x2f6, 0x002, 0x009, 0x00f, 0x364, 0x2cf, 0x080, 0x009,
+ 0x00a, 0x0c3, 0x26f, 0x044, 0x1b6, 0x001, 0x020, 0x2e4,
+ 0x002, 0x044, 0x051, 0x001, 0x000, 0x308, 0x3c4, 0x3f7,
+ 0x233, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x083, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x3d7,
+ 0x200, 0x27b, 0x0c0, 0x271, 0x3b7, 0x300, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x221, 0x091, 0x189, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x201, 0x387, 0x0a6, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x021, 0x020, 0x3c4, 0x3ed, 0x020, 0x3b7,
+ 0x200, 0x387, 0x020, 0x044, 0x020, 0x00a, 0x009, 0x32a,
+ 0x347, 0x271, 0x270, 0x000, 0x055, 0x3a7, 0x071, 0x307,
+ 0x21d, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x233, 0x044,
+ 0x2f6, 0x002, 0x009, 0x007, 0x0c1, 0x2d3, 0x3c5, 0x100,
+ 0x200, 0x081, 0x18a, 0x245, 0x232, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x093, 0x0c2, 0x22a,
+ 0x044, 0x39c, 0x008, 0x0c0, 0x27b, 0x0c0, 0x272, 0x3b7,
+ 0x300, 0x387, 0x020, 0x044, 0x1a9, 0x000, 0x0a1, 0x397,
+ 0x205, 0x247, 0x27f, 0x044, 0x1bd, 0x000, 0x3d5, 0x204,
+ 0x020, 0x3c4, 0x2ff, 0x020, 0x0a0, 0x387, 0x021, 0x184,
+ 0x044, 0x020, 0x00a, 0x009, 0x336, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3fe, 0x200, 0x3c4, 0x3fe,
+ 0x26f, 0x344, 0x21f, 0x232, 0x0c0, 0x020, 0x0a3, 0x1ab,
+ 0x2c5, 0x020, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000,
+ 0x044, 0x3b5, 0x000, 0x044, 0x3f0, 0x000, 0x0a9, 0x1ab,
+ 0x09f, 0x18a, 0x208, 0x304, 0x055, 0x225, 0x044, 0x394,
+ 0x00a, 0x307, 0x055, 0x19b, 0x384, 0x003, 0x227, 0x1a3,
+ 0x225, 0x044, 0x0a4, 0x00b, 0x0a7, 0x020, 0x022, 0x008,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x006, 0x387, 0x07f,
+ 0x000, 0x013, 0x399, 0x240, 0x385, 0x07f, 0x306, 0x270,
+ 0x001, 0x06c, 0x399, 0x240, 0x327, 0x270, 0x3a4, 0x380,
+ 0x226, 0x001, 0x015, 0x327, 0x220, 0x324, 0x232, 0x009,
+ 0x019, 0x364, 0x232, 0x001, 0x001, 0x02b, 0x327, 0x01d,
+ 0x307, 0x01b, 0x190, 0x282, 0x003, 0x041, 0x307, 0x01b,
+ 0x191, 0x380, 0x010, 0x282, 0x003, 0x034, 0x000, 0x019,
+ 0x327, 0x220, 0x324, 0x232, 0x001, 0x00a, 0x327, 0x270,
+ 0x384, 0x07f, 0x3a4, 0x07f, 0x000, 0x008, 0x327, 0x270,
+ 0x3a4, 0x07f, 0x001, 0x005, 0x286, 0x001, 0x02f, 0x307,
+ 0x270, 0x384, 0x07f, 0x386, 0x07f, 0x009, 0x00a, 0x387,
+ 0x080, 0x382, 0x07f, 0x240, 0x270, 0x000, 0x01c, 0x141,
+ 0x270, 0x364, 0x270, 0x07f, 0x009, 0x015, 0x000, 0x013,
+ 0x3c0, 0x040, 0x270, 0x000, 0x00e, 0x307, 0x270, 0x196,
+ 0x384, 0x00f, 0x266, 0x009, 0x001, 0x008, 0x3c0, 0x080,
+ 0x270, 0x044, 0x35f, 0x009, 0x045, 0x309, 0x232, 0x00b,
+ 0x013, 0x347, 0x271, 0x270, 0x374, 0x27b, 0x200, 0x009,
+ 0x005, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x020,
+ 0x0b7, 0x009, 0x327, 0x232, 0x1a2, 0x023, 0x0b7, 0x009,
+ 0x307, 0x270, 0x196, 0x119, 0x021, 0x0b7, 0x009, 0x307,
+ 0x232, 0x181, 0x023, 0x0b7, 0x009, 0x364, 0x270, 0x080,
+ 0x029, 0x0b7, 0x009, 0x083, 0x18c, 0x245, 0x232, 0x3c4,
+ 0x3f7, 0x233, 0x3c4, 0x380, 0x270, 0x3c0, 0x080, 0x270,
+ 0x044, 0x35f, 0x009, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x020, 0x09c, 0x009, 0x000, 0x000, 0x000, 0x002, 0x000,
+ 0x00a, 0x000, 0x02a, 0x000, 0x0aa, 0x002, 0x0aa, 0x00a,
+ 0x0aa, 0x02a, 0x0aa, 0x06a, 0x0aa, 0x07a, 0x0aa, 0x07e,
+ 0x0aa, 0x07f, 0x0aa, 0x07f, 0x0ba, 0x07f, 0x0be, 0x07f,
+ 0x0fe, 0x07f, 0x0ff, 0x007, 0x000, 0x017, 0x001, 0x027,
+ 0x007, 0x037, 0x00f, 0x047, 0x03f, 0x080, 0x0ff, 0x364,
+ 0x270, 0x07f, 0x307, 0x270, 0x384, 0x07f, 0x009, 0x00d,
+ 0x089, 0x186, 0x306, 0x270, 0x001, 0x007, 0x3c5, 0x001,
+ 0x232, 0x000, 0x005, 0x3c4, 0x3fe, 0x232, 0x347, 0x270,
+ 0x27c, 0x3c4, 0x007, 0x27c, 0x374, 0x27b, 0x200, 0x001,
+ 0x00a, 0x364, 0x233, 0x008, 0x001, 0x005, 0x044, 0x0fe,
+ 0x00a, 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384,
+ 0x3f8, 0x305, 0x27c, 0x044, 0x1bd, 0x000, 0x307, 0x270,
+ 0x192, 0x384, 0x00f, 0x3b9, 0x200, 0x324, 0x233, 0x001,
+ 0x014, 0x229, 0x387, 0x009, 0x189, 0x395, 0x333, 0x280,
+ 0x051, 0x2c7, 0x27d, 0x1c7, 0x27d, 0x101, 0x051, 0x2c5,
+ 0x27d, 0x000, 0x011, 0x0c0, 0x27d, 0x384, 0x00f, 0x001,
+ 0x00b, 0x349, 0x27d, 0x27d, 0x3c5, 0x001, 0x27d, 0x111,
+ 0x000, 0x3f7, 0x0a2, 0x325, 0x27b, 0x307, 0x27d, 0x044,
+ 0x1bd, 0x000, 0x374, 0x27b, 0x200, 0x001, 0x02c, 0x347,
+ 0x270, 0x27a, 0x3c4, 0x07f, 0x27a, 0x0c0, 0x279, 0x387,
+ 0x009, 0x189, 0x395, 0x353, 0x051, 0x041, 0x101, 0x051,
+ 0x2c7, 0x279, 0x101, 0x043, 0x322, 0x27a, 0x00b, 0x004,
+ 0x000, 0x3f4, 0x1c7, 0x279, 0x0a3, 0x325, 0x27b, 0x044,
+ 0x192, 0x000, 0x384, 0x0ff, 0x305, 0x279, 0x044, 0x1bd,
+ 0x000, 0x327, 0x220, 0x324, 0x232, 0x009, 0x022, 0x0c0,
+ 0x27e, 0x307, 0x270, 0x197, 0x00a, 0x004, 0x000, 0x005,
+ 0x3c5, 0x018, 0x27e, 0x0a4, 0x222, 0x2c5, 0x27e, 0x0a4,
+ 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x3e0, 0x3c4,
+ 0x01f, 0x27e, 0x305, 0x27e, 0x044, 0x1bd, 0x000, 0x045,
+ 0x044, 0x1a9, 0x000, 0x083, 0x18c, 0x208, 0x244, 0x232,
+ 0x3c5, 0x001, 0x00c, 0x0a3, 0x325, 0x27b, 0x088, 0x044,
+ 0x1a9, 0x000, 0x387, 0x3f8, 0x044, 0x1b8, 0x000, 0x020,
+ 0x2f6, 0x002, 0x3c7, 0x020, 0x280, 0x3b7, 0x215, 0x044,
+ 0x192, 0x000, 0x247, 0x247, 0x151, 0x280, 0x001, 0x016,
+ 0x0a1, 0x044, 0x182, 0x000, 0x3b7, 0x215, 0x044, 0x192,
+ 0x000, 0x306, 0x247, 0x001, 0x3f1, 0x0a2, 0x364, 0x22b,
+ 0x003, 0x001, 0x003, 0x0a5, 0x020, 0x182, 0x000, 0x1f6,
+ 0x185, 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122,
+ 0x0ff, 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194,
+ 0x0ce, 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020,
+ 0x009, 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9,
+ 0x286, 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x05f,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051,
+ 0x2c7, 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f,
+ 0x040, 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b,
+ 0x005, 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4,
+ 0x00f, 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288,
+ 0x286, 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b,
+ 0x3fc, 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285,
+ 0x307, 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd,
+ 0x000, 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2,
+ 0x12c, 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002,
+ 0x005, 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b,
+ 0x1f0, 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081,
+ 0x138, 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f,
+ 0x080, 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d,
+ 0x1e0, 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b,
+ 0x347, 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044,
+ 0x1bc, 0x00a, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364,
+ 0x233, 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff,
+ 0x242, 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289,
+ 0x397, 0x00a, 0x189, 0x395, 0x0ca, 0x044, 0x2ea, 0x001,
+ 0x009, 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9,
+ 0x121, 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f,
+ 0x282, 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000,
+ 0x3ed, 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233,
+ 0x002, 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff,
+ 0x281, 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366,
+ 0x282, 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282,
+ 0x044, 0x1bc, 0x00a, 0x001, 0x00d, 0x141, 0x28b, 0x362,
+ 0x28b, 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5,
+ 0x362, 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397,
+ 0x00a, 0x189, 0x395, 0x0d8, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005,
+ 0x105, 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f,
+ 0x121, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042,
+ 0x003, 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387,
+ 0x00a, 0x189, 0x395, 0x0d3, 0x300, 0x28a, 0x050, 0x247,
+ 0x28a, 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a,
+ 0x044, 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282,
+ 0x281, 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045,
+ 0x000, 0x301, 0x103, 0x307, 0x006, 0x10e, 0x21e, 0x33e,
+ 0x13c, 0x038, 0x078, 0x1f8, 0x0f0, 0x000, 0x001, 0x003,
+ 0x002, 0x006, 0x187, 0x227, 0x387, 0x00a, 0x189, 0x395,
+ 0x1c8, 0x300, 0x227, 0x111, 0x050, 0x247, 0x24c, 0x1d7,
+ 0x24c, 0x384, 0x0ff, 0x285, 0x044, 0x226, 0x00a, 0x364,
+ 0x22a, 0x001, 0x001, 0x00e, 0x3d5, 0x200, 0x232, 0x040,
+ 0x1aa, 0x285, 0x3b7, 0x220, 0x044, 0x1bd, 0x000, 0x042,
+ 0x364, 0x22a, 0x002, 0x001, 0x022, 0x040, 0x3a7, 0x073,
+ 0x044, 0x192, 0x000, 0x384, 0x31f, 0x044, 0x226, 0x00a,
+ 0x1a4, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000, 0x3a7,
+ 0x070, 0x044, 0x192, 0x000, 0x19a, 0x18a, 0x043, 0x285,
+ 0x3a7, 0x070, 0x044, 0x1bd, 0x000, 0x045, 0x040, 0x387,
+ 0x00a, 0x189, 0x395, 0x1d5, 0x300, 0x278, 0x051, 0x042,
+ 0x045, 0x00a, 0x058, 0x026, 0x015, 0x044, 0x032, 0x0c0,
+ 0x278, 0x0a0, 0x397, 0x00a, 0x189, 0x395, 0x231, 0x280,
+ 0x050, 0x247, 0x228, 0x3c4, 0x00f, 0x228, 0x347, 0x228,
+ 0x229, 0x040, 0x307, 0x278, 0x001, 0x008, 0x340, 0x229,
+ 0x228, 0x111, 0x000, 0x3fa, 0x042, 0x193, 0x0c0, 0x227,
+ 0x347, 0x225, 0x226, 0x141, 0x227, 0x366, 0x227, 0x00e,
+ 0x001, 0x00b, 0x342, 0x228, 0x226, 0x007, 0x3f6, 0x140,
+ 0x226, 0x001, 0x011, 0x121, 0x2e2, 0x006, 0x009, 0x3cc,
+ 0x141, 0x278, 0x366, 0x278, 0x005, 0x009, 0x3c4, 0x0a1,
+ 0x000, 0x006, 0x044, 0x1da, 0x00a, 0x0a0, 0x045, 0x044,
+ 0x333, 0x008, 0x020, 0x181, 0x006, 0x345, 0x21d, 0x232,
+ 0x364, 0x23f, 0x001, 0x029, 0x2e4, 0x002, 0x2c7, 0x234,
+ 0x399, 0x200, 0x245, 0x233, 0x364, 0x234, 0x020, 0x009,
+ 0x005, 0x208, 0x244, 0x233, 0x083, 0x18c, 0x208, 0x244,
+ 0x036, 0x2e4, 0x008, 0x001, 0x007, 0x345, 0x220, 0x232,
+ 0x000, 0x007, 0x081, 0x18b, 0x208, 0x244, 0x232, 0x307,
+ 0x235, 0x226, 0x3a4, 0x001, 0x001, 0x010, 0x327, 0x234,
+ 0x2e4, 0x001, 0x009, 0x041, 0x041, 0x044, 0x20d, 0x000,
+ 0x044, 0x2f3, 0x000, 0x043, 0x0a3, 0x1ad, 0x3a5, 0x039,
+ 0x326, 0x236, 0x001, 0x02f, 0x3a7, 0x0bf, 0x1a7, 0x3a5,
+ 0x039, 0x326, 0x236, 0x001, 0x026, 0x327, 0x234, 0x307,
+ 0x235, 0x286, 0x384, 0x002, 0x001, 0x01d, 0x2e4, 0x002,
+ 0x009, 0x051, 0x041, 0x0a1, 0x1ad, 0x2c5, 0x024, 0x307,
+ 0x024, 0x19a, 0x002, 0x005, 0x2a8, 0x2c4, 0x024, 0x044,
+ 0x3b5, 0x000, 0x044, 0x0f2, 0x001, 0x044, 0x3f0, 0x000,
+ 0x043, 0x000, 0x067, 0x364, 0x026, 0x001, 0x009, 0x3c6,
+ 0x041, 0x0a0, 0x044, 0x301, 0x000, 0x364, 0x26f, 0x002,
+ 0x021, 0x310, 0x00a, 0x044, 0x290, 0x001, 0x0c1, 0x26f,
+ 0x043, 0x307, 0x020, 0x193, 0x384, 0x007, 0x110, 0x001,
+ 0x00c, 0x364, 0x232, 0x010, 0x009, 0x007, 0x044, 0x044,
+ 0x001, 0x000, 0x3a3, 0x3c4, 0x3fe, 0x26f, 0x3c5, 0x001,
+ 0x200, 0x3c5, 0x010, 0x020, 0x3c4, 0x31f, 0x020, 0x000,
+ 0x395, 0x364, 0x026, 0x002, 0x009, 0x3c5, 0x041, 0x044,
+ 0x003, 0x001, 0x043, 0x307, 0x020, 0x197, 0x384, 0x007,
+ 0x110, 0x001, 0x00f, 0x364, 0x232, 0x020, 0x009, 0x00a,
+ 0x044, 0x051, 0x001, 0x3c5, 0x001, 0x027, 0x000, 0x3ab,
+ 0x3c4, 0x3fc, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x100,
+ 0x020, 0x087, 0x188, 0x208, 0x244, 0x020, 0x000, 0x39b,
+ 0x044, 0x11a, 0x001, 0x307, 0x234, 0x306, 0x235, 0x384,
+ 0x003, 0x021, 0x2e4, 0x002, 0x347, 0x234, 0x235, 0x366,
+ 0x26f, 0x002, 0x021, 0x2e4, 0x002, 0x364, 0x234, 0x003,
+ 0x021, 0x2e4, 0x002, 0x327, 0x21c, 0x324, 0x020, 0x021,
+ 0x385, 0x00a, 0x020, 0x247, 0x002, 0x364, 0x23f, 0x001,
+ 0x021, 0x38e, 0x00a, 0x020, 0x247, 0x002, 0x345, 0x21d,
+ 0x232, 0x020, 0x247, 0x002, 0x2c7, 0x241, 0x307, 0x242,
+ 0x286, 0x001, 0x092, 0x0c1, 0x22a, 0x327, 0x242, 0x326,
+ 0x241, 0x3a4, 0x0ff, 0x001, 0x005, 0x3c4, 0x38f, 0x020,
+ 0x347, 0x241, 0x242, 0x347, 0x241, 0x226, 0x307, 0x21d,
+ 0x304, 0x241, 0x001, 0x00a, 0x0c3, 0x22a, 0x347, 0x241,
+ 0x243, 0x044, 0x39f, 0x002, 0x327, 0x241, 0x3a4, 0x0ff,
+ 0x2c7, 0x225, 0x3c4, 0x300, 0x230, 0x2c5, 0x230, 0x044,
+ 0x099, 0x00b, 0x081, 0x18b, 0x304, 0x226, 0x009, 0x00d,
+ 0x364, 0x232, 0x100, 0x009, 0x008, 0x081, 0x18a, 0x208,
+ 0x044, 0x1b8, 0x000, 0x044, 0x186, 0x001, 0x327, 0x226,
+ 0x003, 0x032, 0x044, 0x2f6, 0x002, 0x001, 0x019, 0x044,
+ 0x237, 0x00a, 0x347, 0x241, 0x226, 0x081, 0x306, 0x22a,
+ 0x384, 0x003, 0x001, 0x005, 0x347, 0x243, 0x226, 0x044,
+ 0x032, 0x009, 0x002, 0x02d, 0x000, 0x031, 0x044, 0x237,
+ 0x00a, 0x009, 0x01c, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x001, 0x023, 0x364, 0x233, 0x001, 0x000,
+ 0x01e, 0x045, 0x044, 0x186, 0x001, 0x3c5, 0x003, 0x22a,
+ 0x347, 0x241, 0x243, 0x000, 0x3c7, 0x3a4, 0x3ff, 0x001,
+ 0x00e, 0x327, 0x21c, 0x324, 0x020, 0x009, 0x006, 0x043,
+ 0x3a7, 0x0ff, 0x041, 0x0c0, 0x270, 0x044, 0x082, 0x00b,
+ 0x3c4, 0x3fc, 0x22a, 0x327, 0x21c, 0x324, 0x020, 0x009,
+ 0x010, 0x307, 0x270, 0x003, 0x005, 0x020, 0x181, 0x006,
+ 0x043, 0x3a7, 0x0ff, 0x041, 0x020, 0x181, 0x006, 0x045,
+ 0x2c7, 0x243, 0x307, 0x250, 0x286, 0x001, 0x3de, 0x0c2,
+ 0x22a, 0x041, 0x044, 0x39f, 0x002, 0x043, 0x2c7, 0x226,
+ 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x0ff, 0x230, 0x1a7,
+ 0x2c5, 0x230, 0x307, 0x21d, 0x304, 0x243, 0x001, 0x013,
+ 0x3c4, 0x0ff, 0x243, 0x347, 0x243, 0x241, 0x347, 0x243,
+ 0x242, 0x3c5, 0x003, 0x22a, 0x044, 0x186, 0x001, 0x000,
+ 0x005, 0x044, 0x191, 0x001, 0x327, 0x243, 0x003, 0x394,
+ 0x000, 0x362, 0x364, 0x232, 0x004, 0x001, 0x008, 0x044,
+ 0x232, 0x000, 0x3c4, 0x3fb, 0x232, 0x364, 0x232, 0x008,
+ 0x001, 0x008, 0x044, 0x326, 0x000, 0x3c4, 0x3f7, 0x232,
+ 0x045, 0x3b7, 0x221, 0x083, 0x189, 0x044, 0x1a9, 0x000,
+ 0x083, 0x020, 0x1b7, 0x000, 0x2c7, 0x24d, 0x364, 0x233,
+ 0x002, 0x001, 0x019, 0x3b9, 0x381, 0x121, 0x081, 0x18a,
+ 0x225, 0x304, 0x24d, 0x001, 0x005, 0x044, 0x1ff, 0x00b,
+ 0x140, 0x24d, 0x00b, 0x008, 0x1a3, 0x044, 0x1ff, 0x00b,
+ 0x000, 0x012, 0x307, 0x220, 0x284, 0x001, 0x00d, 0x3c4,
+ 0x33f, 0x232, 0x1b6, 0x3a4, 0x0c0, 0x2c5, 0x232, 0x020,
+ 0x18a, 0x00b, 0x327, 0x24d, 0x364, 0x020, 0x070, 0x001,
+ 0x00a, 0x399, 0x383, 0x101, 0x326, 0x22b, 0x234, 0x001,
+ 0x02e, 0x3c5, 0x080, 0x233, 0x3d5, 0x200, 0x232, 0x364,
+ 0x026, 0x001, 0x001, 0x00a, 0x3c5, 0x004, 0x232, 0x044,
+ 0x20d, 0x000, 0x307, 0x24d, 0x307, 0x24d, 0x384, 0x007,
+ 0x001, 0x09a, 0x113, 0x001, 0x0a8, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3f8, 0x103, 0x044, 0x1bd, 0x000,
+ 0x3c4, 0x3f7, 0x021, 0x000, 0x0af, 0x0a7, 0x1a7, 0x324,
+ 0x020, 0x001, 0x014, 0x327, 0x24d, 0x326, 0x22b, 0x397,
+ 0x370, 0x364, 0x233, 0x002, 0x001, 0x006, 0x087, 0x18b,
+ 0x385, 0x070, 0x234, 0x001, 0x02b, 0x3c5, 0x100, 0x233,
+ 0x364, 0x026, 0x002, 0x001, 0x008, 0x3c5, 0x008, 0x232,
+ 0x044, 0x3b5, 0x000, 0x307, 0x24d, 0x384, 0x070, 0x001,
+ 0x091, 0x382, 0x030, 0x001, 0x0a6, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x018, 0x044, 0x1bd,
+ 0x000, 0x3c4, 0x3f7, 0x024, 0x000, 0x0a4, 0x347, 0x24d,
+ 0x22b, 0x364, 0x233, 0x006, 0x001, 0x033, 0x307, 0x22b,
+ 0x364, 0x233, 0x002, 0x001, 0x023, 0x364, 0x233, 0x080,
+ 0x001, 0x00a, 0x307, 0x24d, 0x197, 0x384, 0x007, 0x044,
+ 0x3a4, 0x008, 0x364, 0x233, 0x100, 0x001, 0x00c, 0x307,
+ 0x24d, 0x19b, 0x384, 0x007, 0x385, 0x010, 0x044, 0x3a4,
+ 0x008, 0x3c4, 0x27f, 0x233, 0x000, 0x00b, 0x3c5, 0x003,
+ 0x22a, 0x197, 0x044, 0x39c, 0x008, 0x003, 0x298, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x021, 0x181,
+ 0x006, 0x045, 0x3b7, 0x301, 0x087, 0x044, 0x1b7, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7,
+ 0x021, 0x000, 0x36c, 0x3b7, 0x301, 0x044, 0x192, 0x000,
+ 0x384, 0x3f8, 0x105, 0x044, 0x1bd, 0x000, 0x3b7, 0x211,
+ 0x082, 0x044, 0x1a9, 0x000, 0x3c5, 0x008, 0x021, 0x344,
+ 0x21f, 0x021, 0x3b7, 0x250, 0x088, 0x044, 0x1a9, 0x000,
+ 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000, 0x000, 0x347,
+ 0x3b7, 0x301, 0x387, 0x3c7, 0x044, 0x1b8, 0x000, 0x3a7,
+ 0x060, 0x083, 0x044, 0x1b7, 0x000, 0x3b7, 0x070, 0x083,
+ 0x18a, 0x044, 0x1b7, 0x000, 0x3c4, 0x3f7, 0x024, 0x000,
+ 0x36f, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x028, 0x044, 0x1bd, 0x000, 0x3c5, 0x048, 0x024,
+ 0x3a7, 0x060, 0x083, 0x044, 0x1a9, 0x000, 0x3b7, 0x070,
+ 0x083, 0x18a, 0x044, 0x1a9, 0x000, 0x000, 0x351, 0x307,
+ 0x22b, 0x284, 0x2a8, 0x2c4, 0x24d, 0x245, 0x24d, 0x2a8,
+ 0x045, 0x2f4, 0x200, 0x001, 0x008, 0x364, 0x2cf, 0x007,
+ 0x029, 0x1dc, 0x007, 0x287, 0x385, 0x300, 0x208, 0x001,
+ 0x04b, 0x364, 0x33b, 0x010, 0x029, 0x1dc, 0x007, 0x3c5,
+ 0x002, 0x00c, 0x287, 0x384, 0x0ff, 0x305, 0x21d, 0x1b7,
+ 0x2e4, 0x001, 0x001, 0x01d, 0x040, 0x384, 0x007, 0x117,
+ 0x001, 0x00c, 0x3c5, 0x040, 0x02d, 0x081, 0x044, 0x189,
+ 0x000, 0x3c4, 0x3bf, 0x02d, 0x042, 0x247, 0x029, 0x364,
+ 0x018, 0x001, 0x009, 0x005, 0x3c5, 0x010, 0x021, 0x2e4,
+ 0x002, 0x001, 0x017, 0x264, 0x080, 0x001, 0x005, 0x0a4,
+ 0x1a7, 0x285, 0x040, 0x3c5, 0x080, 0x02d, 0x081, 0x044,
+ 0x189, 0x000, 0x3c4, 0x37f, 0x02d, 0x042, 0x247, 0x02a,
+ 0x000, 0x014, 0x38e, 0x3ff, 0x1b8, 0x00a, 0x00a, 0x244,
+ 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4, 0x3fd, 0x00c, 0x120,
+ 0x001, 0x004, 0x244, 0x02a, 0x020, 0x181, 0x006, 0x289,
+ 0x00a, 0x004, 0x2c7, 0x251, 0x22e, 0x364, 0x026, 0x001,
+ 0x021, 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005,
+ 0x044, 0x0d9, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044,
+ 0x182, 0x000, 0x020, 0x181, 0x006, 0x2e4, 0x002, 0x009,
+ 0x01b, 0x3c4, 0x2ff, 0x232, 0x120, 0x001, 0x012, 0x3d5,
+ 0x300, 0x232, 0x044, 0x099, 0x00b, 0x32a, 0x026, 0x00a,
+ 0x008, 0x044, 0x20d, 0x000, 0x044, 0x232, 0x000, 0x020,
+ 0x181, 0x006, 0x042, 0x307, 0x240, 0x20a, 0x040, 0x020,
+ 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x256, 0x22e,
+ 0x364, 0x026, 0x002, 0x021, 0x181, 0x006, 0x041, 0x3a4,
+ 0x01f, 0x001, 0x005, 0x044, 0x0de, 0x001, 0x043, 0x1b7,
+ 0x001, 0x005, 0x044, 0x0e2, 0x001, 0x020, 0x181, 0x006,
+ 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044, 0x340, 0x001,
+ 0x247, 0x249, 0x054, 0x347, 0x249, 0x248, 0x2c7, 0x23f,
+ 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f, 0x001, 0x001,
+ 0x006, 0x245, 0x057, 0x000, 0x005, 0x208, 0x244, 0x057,
+ 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f, 0x002, 0x001,
+ 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001, 0x000, 0x005,
+ 0x208, 0x244, 0x057, 0x020, 0x181, 0x006, 0x2c7, 0x246,
+ 0x3b7, 0x300, 0x044, 0x192, 0x000, 0x384, 0x33f, 0x327,
+ 0x246, 0x3a4, 0x0c0, 0x285, 0x3b7, 0x300, 0x044, 0x1bd,
+ 0x000, 0x3b7, 0x200, 0x044, 0x192, 0x000, 0x0a3, 0x1a9,
+ 0x2a8, 0x284, 0x040, 0x307, 0x246, 0x384, 0x030, 0x264,
+ 0x010, 0x001, 0x012, 0x040, 0x041, 0x3b7, 0x201, 0x044,
+ 0x192, 0x000, 0x384, 0x3e0, 0x385, 0x006, 0x044, 0x1bd,
+ 0x000, 0x043, 0x042, 0x185, 0x2a8, 0x284, 0x227, 0x042,
+ 0x285, 0x3b7, 0x200, 0x044, 0x1bd, 0x000, 0x307, 0x246,
+ 0x384, 0x007, 0x044, 0x375, 0x00b, 0x3a7, 0x073, 0x307,
+ 0x246, 0x184, 0x00a, 0x014, 0x044, 0x192, 0x000, 0x384,
+ 0x3e2, 0x327, 0x246, 0x1b5, 0x3a4, 0x01c, 0x285, 0x3a7,
+ 0x073, 0x044, 0x1bd, 0x000, 0x000, 0x006, 0x081, 0x044,
+ 0x1a9, 0x000, 0x020, 0x181, 0x006, 0x3c5, 0x001, 0x00c,
+ 0x187, 0x0a7, 0x1a7, 0x2a8, 0x324, 0x00c, 0x225, 0x2c7,
+ 0x00c, 0x045, 0x04e, 0x087, 0x011, 0x054, 0x084
+
+}; /* end fm10000_serdes_spico_code_prd2 */
+
+
+
+/* Spico Swap code (production version 2)
+ * source file: serdes.0x2055_0045.swap
+ * Note: if swap code exists, if must be loaded on top of
+ * the sbus master image.
+ */
+
+const uint32_t fm10000_serdes_swap_code_version_build_id_prd2 = 0x20550045;
+const uint32_t fm10000_serdes_swap_code_size_prd2 = 7842;
+
+const uint16_t fm10000_serdes_swap_code_prd2[] = {
+ 0x01e, 0x09d, 0x002, 0x000, 0x000, 0x008, 0x00f, 0x07b,
+ 0x020, 0x01e, 0x00f, 0x069, 0x01d, 0x029, 0x00b, 0x01d,
+ 0x02c, 0x382, 0x04e, 0x020, 0x23b, 0x009, 0x020, 0x28e,
+ 0x00a, 0x020, 0x03a, 0x00a, 0x020, 0x0fe, 0x00a, 0x020,
+ 0x3f3, 0x008, 0x020, 0x013, 0x009, 0x020, 0x075, 0x00a,
+ 0x0c1, 0x244, 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200,
+ 0x044, 0x172, 0x000, 0x020, 0x0d3, 0x000, 0x020, 0x2c3,
+ 0x008, 0x020, 0x375, 0x00b, 0x0c1, 0x201, 0x045, 0x020,
+ 0x04a, 0x009, 0x020, 0x04e, 0x009, 0x020, 0x256, 0x002,
+ 0x020, 0x17f, 0x009, 0x020, 0x23f, 0x006, 0x020, 0x285,
+ 0x00a, 0x020, 0x209, 0x00b, 0x020, 0x257, 0x006, 0x020,
+ 0x117, 0x008, 0x020, 0x394, 0x00a, 0x020, 0x048, 0x00b,
+ 0x020, 0x1dc, 0x007, 0x020, 0x27f, 0x00a, 0x020, 0x2d2,
+ 0x006, 0x020, 0x1e2, 0x007, 0x020, 0x295, 0x00b, 0x020,
+ 0x3a8, 0x001, 0x020, 0x277, 0x00b, 0x020, 0x2ba, 0x00b,
+ 0x020, 0x2d5, 0x006, 0x020, 0x326, 0x006, 0x020, 0x350,
+ 0x006, 0x020, 0x1dc, 0x007, 0x020, 0x35a, 0x006, 0x020,
+ 0x0a4, 0x00b, 0x020, 0x38e, 0x006, 0x020, 0x011, 0x007,
+ 0x020, 0x016, 0x007, 0x020, 0x31e, 0x007, 0x020, 0x324,
+ 0x007, 0x020, 0x3b6, 0x007, 0x020, 0x02a, 0x007, 0x020,
+ 0x041, 0x007, 0x020, 0x060, 0x007, 0x020, 0x06d, 0x007,
+ 0x020, 0x072, 0x007, 0x020, 0x2d1, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x252, 0x007, 0x020, 0x293, 0x007,
+ 0x020, 0x2d8, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x16f, 0x007, 0x020, 0x185, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x30e, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x077, 0x007, 0x020, 0x0b1, 0x007, 0x020,
+ 0x103, 0x007, 0x020, 0x1c5, 0x008, 0x020, 0x1dc, 0x007,
+ 0x020, 0x168, 0x007, 0x287, 0x264, 0x004, 0x001, 0x049,
+ 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020, 0x33b, 0x18e, 0x00a,
+ 0x013, 0x044, 0x04c, 0x004, 0x3a4, 0x001, 0x001, 0x014,
+ 0x364, 0x052, 0x001, 0x009, 0x093, 0x044, 0x0bc, 0x004,
+ 0x000, 0x08e, 0x3c4, 0x2ff, 0x053, 0x3c4, 0x030, 0x027,
+ 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052, 0x000, 0x081, 0x043,
+ 0x327, 0x33b, 0x020, 0x1de, 0x007, 0x2e4, 0x080, 0x009,
+ 0x00d, 0x3d5, 0x220, 0x33c, 0x100, 0x009, 0x06e, 0x044,
+ 0x3b6, 0x003, 0x000, 0x06c, 0x3d5, 0x120, 0x33c, 0x100,
+ 0x009, 0x022, 0x044, 0x066, 0x004, 0x000, 0x061, 0x384,
+ 0x003, 0x3c4, 0x3df, 0x33b, 0x262, 0x003, 0x001, 0x3d9,
+ 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef, 0x027, 0x2e4, 0x040,
+ 0x009, 0x3d5, 0x100, 0x009, 0x007, 0x044, 0x063, 0x004,
+ 0x000, 0x046, 0x364, 0x200, 0x004, 0x029, 0x1dc, 0x007,
+ 0x0c0, 0x2d8, 0x111, 0x001, 0x028, 0x3c5, 0x002, 0x33c,
+ 0x3c5, 0x001, 0x33b, 0x344, 0x21f, 0x33c, 0x287, 0x19b,
+ 0x247, 0x361, 0x287, 0x183, 0x19b, 0x247, 0x360, 0x0c0,
+ 0x35e, 0x2e4, 0x020, 0x001, 0x004, 0x0c5, 0x35e, 0x2e4,
+ 0x010, 0x001, 0x005, 0x3c5, 0x080, 0x33c, 0x289, 0x384,
+ 0x010, 0x245, 0x33c, 0x044, 0x001, 0x004, 0x364, 0x33c,
+ 0x002, 0x001, 0x005, 0x3c5, 0x004, 0x200, 0x364, 0x33c,
+ 0x020, 0x009, 0x005, 0x044, 0x097, 0x004, 0x020, 0x181,
+ 0x006, 0x287, 0x19b, 0x11d, 0x001, 0x002, 0x287, 0x1a3,
+ 0x1b3, 0x19b, 0x180, 0x390, 0x1d7, 0x040, 0x199, 0x390,
+ 0x008, 0x040, 0x045, 0x000, 0x021, 0x000, 0x073, 0x000,
+ 0x05f, 0x000, 0x023, 0x000, 0x025, 0x000, 0x033, 0x000,
+ 0x035, 0x000, 0x037, 0x000, 0x041, 0x000, 0x01f, 0x000,
+ 0x021, 0x000, 0x023, 0x000, 0x031, 0x000, 0x082, 0x000,
+ 0x002, 0x020, 0x1dc, 0x007, 0x044, 0x378, 0x003, 0x3c4,
+ 0x3fd, 0x233, 0x000, 0x072, 0x2c7, 0x363, 0x000, 0x06e,
+ 0x2c7, 0x362, 0x000, 0x06a, 0x2c7, 0x341, 0x000, 0x066,
+ 0x2c7, 0x33f, 0x000, 0x062, 0x2c7, 0x340, 0x000, 0x05e,
+ 0x2c7, 0x344, 0x000, 0x05a, 0x2c7, 0x342, 0x000, 0x056,
+ 0x2c7, 0x343, 0x000, 0x052, 0x2c7, 0x341, 0x2c7, 0x33f,
+ 0x2c7, 0x340, 0x000, 0x04a, 0x307, 0x340, 0x300, 0x341,
+ 0x300, 0x33f, 0x282, 0x007, 0x006, 0x2c7, 0x23e, 0x000,
+ 0x03d, 0x043, 0x0a1, 0x020, 0x1de, 0x007, 0x287, 0x384,
+ 0x003, 0x181, 0x3c4, 0x3f3, 0x052, 0x245, 0x052, 0x3a4,
+ 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5, 0x33c, 0x000, 0x026,
+ 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff, 0x352, 0x3c7, 0x3ff,
+ 0x353, 0x1d3, 0x353, 0x0cf, 0x354, 0x1cb, 0x354, 0x3c5,
+ 0x0c0, 0x354, 0x3c9, 0x222, 0x355, 0x3c5, 0x001, 0x33c,
+ 0x0c0, 0x356, 0x1b0, 0x00a, 0x006, 0x081, 0x18a, 0x247,
+ 0x356, 0x1ab, 0x2c7, 0x357, 0x020, 0x181, 0x006, 0x287,
+ 0x001, 0x025, 0x197, 0x3a4, 0x00f, 0x262, 0x003, 0x001,
+ 0x03d, 0x131, 0x001, 0x020, 0x131, 0x001, 0x021, 0x131,
+ 0x001, 0x022, 0x131, 0x001, 0x025, 0x131, 0x001, 0x026,
+ 0x131, 0x001, 0x027, 0x132, 0x182, 0x390, 0x377, 0x280,
+ 0x04c, 0x042, 0x020, 0x1de, 0x007, 0x044, 0x361, 0x003,
+ 0x000, 0x3d4, 0x327, 0x373, 0x000, 0x3f5, 0x327, 0x372,
+ 0x000, 0x3f1, 0x327, 0x2b3, 0x322, 0x2b2, 0x000, 0x3eb,
+ 0x327, 0x374, 0x000, 0x3e7, 0x327, 0x375, 0x000, 0x3e3,
+ 0x327, 0x376, 0x000, 0x3df, 0x307, 0x369, 0x282, 0x272,
+ 0x369, 0x00e, 0x003, 0x108, 0x04c, 0x000, 0x3d4, 0x364,
+ 0x33c, 0x001, 0x001, 0x01b, 0x364, 0x33b, 0x020, 0x009,
+ 0x016, 0x0a0, 0x307, 0x04f, 0x181, 0x00a, 0x003, 0x0a4,
+ 0x100, 0x00b, 0x004, 0x3a5, 0x080, 0x307, 0x050, 0x194,
+ 0x384, 0x040, 0x225, 0x2c7, 0x392, 0x081, 0x189, 0x314,
+ 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x027, 0x364, 0x33c, 0x020, 0x001, 0x007,
+ 0x347, 0x21d, 0x35a, 0x000, 0x018, 0x347, 0x050, 0x368,
+ 0x3b7, 0x201, 0x1a5, 0x140, 0x04f, 0x00b, 0x00b, 0x364,
+ 0x052, 0x020, 0x001, 0x3f3, 0x2c5, 0x33c, 0x000, 0x005,
+ 0x2a8, 0x2c4, 0x33c, 0x081, 0x189, 0x304, 0x053, 0x001,
+ 0x007, 0x3c5, 0x008, 0x027, 0x000, 0x005, 0x3c4, 0x3f7,
+ 0x027, 0x3c5, 0x080, 0x053, 0x3c4, 0x37f, 0x053, 0x364,
+ 0x33b, 0x020, 0x009, 0x00a, 0x364, 0x33c, 0x020, 0x009,
+ 0x005, 0x044, 0x132, 0x004, 0x020, 0x08d, 0x006, 0x2c7,
+ 0x231, 0x2e4, 0x100, 0x001, 0x045, 0x2e4, 0x001, 0x001,
+ 0x020, 0x3b7, 0x213, 0x084, 0x044, 0x1a9, 0x000, 0x3b7,
+ 0x250, 0x084, 0x044, 0x1a9, 0x000, 0x399, 0x305, 0x3a7,
+ 0x022, 0x044, 0x1a9, 0x000, 0x044, 0x322, 0x001, 0x3c5,
+ 0x002, 0x024, 0x3c5, 0x008, 0x292, 0x000, 0x020, 0x3c4,
+ 0x3fd, 0x024, 0x364, 0x234, 0x002, 0x001, 0x009, 0x3a7,
+ 0x022, 0x399, 0x305, 0x044, 0x1b7, 0x000, 0x3b7, 0x213,
+ 0x084, 0x044, 0x1b7, 0x000, 0x3b7, 0x250, 0x084, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x007, 0x292, 0x044, 0x06e, 0x003,
+ 0x307, 0x231, 0x274, 0x200, 0x001, 0x01b, 0x274, 0x010,
+ 0x001, 0x00b, 0x387, 0x020, 0x245, 0x021, 0x185, 0x245,
+ 0x024, 0x000, 0x00b, 0x387, 0x3df, 0x244, 0x021, 0x185,
+ 0x385, 0x0ff, 0x244, 0x024, 0x044, 0x03a, 0x00a, 0x045,
+ 0x327, 0x230, 0x364, 0x233, 0x004, 0x001, 0x003, 0x045,
+ 0x0c2, 0x22a, 0x347, 0x275, 0x270, 0x0c0, 0x27b, 0x264,
+ 0x010, 0x009, 0x00a, 0x0c1, 0x22a, 0x347, 0x274, 0x270,
+ 0x3d7, 0x200, 0x27b, 0x327, 0x230, 0x264, 0x004, 0x001,
+ 0x012, 0x347, 0x277, 0x270, 0x264, 0x010, 0x009, 0x007,
+ 0x347, 0x276, 0x270, 0x384, 0x00f, 0x112, 0x1b7, 0x000,
+ 0x004, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x384, 0x00f, 0x262,
+ 0x003, 0x001, 0x00b, 0x101, 0x3c2, 0x100, 0x270, 0x34a,
+ 0x225, 0x225, 0x000, 0x3f3, 0x044, 0x237, 0x00a, 0x044,
+ 0x35f, 0x009, 0x364, 0x22a, 0x002, 0x001, 0x006, 0x347,
+ 0x270, 0x272, 0x045, 0x347, 0x270, 0x271, 0x045, 0x044,
+ 0x2f6, 0x002, 0x009, 0x015, 0x307, 0x241, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x3d7, 0x200, 0x27b, 0x044, 0x35f, 0x009, 0x347,
+ 0x241, 0x226, 0x0c1, 0x22a, 0x020, 0x032, 0x009, 0x044,
+ 0x2f6, 0x002, 0x009, 0x014, 0x307, 0x243, 0x197, 0x384,
+ 0x007, 0x001, 0x008, 0x3c2, 0x100, 0x270, 0x111, 0x009,
+ 0x3fc, 0x0c0, 0x27b, 0x044, 0x35f, 0x009, 0x347, 0x243,
+ 0x226, 0x0c2, 0x22a, 0x020, 0x032, 0x009, 0x327, 0x226,
+ 0x2c7, 0x225, 0x3c4, 0x0ff, 0x225, 0x1b7, 0x3a4, 0x007,
+ 0x001, 0x00d, 0x34a, 0x225, 0x225, 0x002, 0x008, 0x131,
+ 0x009, 0x3fa, 0x044, 0x237, 0x00a, 0x045, 0x0c1, 0x26f,
+ 0x000, 0x050, 0x327, 0x01d, 0x2e4, 0x380, 0x009, 0x00a,
+ 0x13a, 0x00b, 0x00a, 0x3c5, 0x008, 0x233, 0x000, 0x005,
+ 0x3c4, 0x3f7, 0x233, 0x327, 0x01d, 0x021, 0x2ed, 0x009,
+ 0x364, 0x020, 0x004, 0x009, 0x01a, 0x309, 0x232, 0x00b,
+ 0x009, 0x347, 0x270, 0x271, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x271, 0x270, 0x044, 0x270, 0x009, 0x001, 0x04c,
+ 0x347, 0x270, 0x271, 0x000, 0x018, 0x309, 0x232, 0x001,
+ 0x009, 0x347, 0x270, 0x272, 0x308, 0x221, 0x244, 0x232,
+ 0x347, 0x272, 0x270, 0x044, 0x270, 0x009, 0x001, 0x089,
+ 0x347, 0x270, 0x272, 0x397, 0x200, 0x044, 0x1b1, 0x001,
+ 0x364, 0x26f, 0x002, 0x009, 0x005, 0x3c4, 0x3fe, 0x200,
+ 0x364, 0x020, 0x005, 0x029, 0x2e4, 0x002, 0x364, 0x020,
+ 0x010, 0x009, 0x0e9, 0x364, 0x020, 0x100, 0x009, 0x12b,
+ 0x020, 0x2e4, 0x002, 0x3c4, 0x3f7, 0x233, 0x364, 0x020,
+ 0x001, 0x009, 0x00c, 0x364, 0x020, 0x004, 0x009, 0x05d,
+ 0x000, 0x3d8, 0x3c5, 0x080, 0x020, 0x364, 0x233, 0x006,
+ 0x001, 0x031, 0x081, 0x189, 0x304, 0x232, 0x009, 0x017,
+ 0x347, 0x271, 0x274, 0x080, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x085, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x081,
+ 0x189, 0x245, 0x232, 0x000, 0x0c8, 0x347, 0x271, 0x276,
+ 0x081, 0x189, 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197,
+ 0x384, 0x007, 0x0c1, 0x22a, 0x044, 0x39c, 0x008, 0x000,
+ 0x005, 0x044, 0x3f3, 0x008, 0x3b7, 0x200, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x3b7, 0x203, 0x387, 0x3f7, 0x044,
+ 0x1b8, 0x000, 0x307, 0x020, 0x385, 0x042, 0x384, 0x3ce,
+ 0x247, 0x020, 0x044, 0x044, 0x001, 0x000, 0x383, 0x0a1,
+ 0x1aa, 0x2c5, 0x020, 0x364, 0x233, 0x006, 0x001, 0x039,
+ 0x081, 0x189, 0x304, 0x232, 0x009, 0x017, 0x347, 0x272,
+ 0x275, 0x080, 0x364, 0x233, 0x004, 0x009, 0x003, 0x095,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x081, 0x189, 0x245,
+ 0x232, 0x000, 0x0ce, 0x347, 0x272, 0x277, 0x081, 0x189,
+ 0x208, 0x244, 0x232, 0x307, 0x22b, 0x197, 0x364, 0x233,
+ 0x002, 0x001, 0x007, 0x193, 0x384, 0x007, 0x385, 0x010,
+ 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x000, 0x008, 0x347,
+ 0x272, 0x270, 0x044, 0x013, 0x009, 0x081, 0x18a, 0x208,
+ 0x244, 0x232, 0x0a3, 0x387, 0x3f7, 0x044, 0x1b8, 0x000,
+ 0x307, 0x020, 0x0a4, 0x1a7, 0x3a5, 0x008, 0x285, 0x3b7,
+ 0x304, 0x2a8, 0x284, 0x247, 0x020, 0x0a0, 0x387, 0x3df,
+ 0x044, 0x1b8, 0x000, 0x044, 0x2f6, 0x002, 0x009, 0x00f,
+ 0x364, 0x2cf, 0x080, 0x009, 0x00a, 0x0c3, 0x26f, 0x044,
+ 0x1b6, 0x001, 0x020, 0x2e4, 0x002, 0x044, 0x051, 0x001,
+ 0x000, 0x308, 0x3c4, 0x3f7, 0x233, 0x364, 0x233, 0x006,
+ 0x001, 0x013, 0x044, 0x2f6, 0x002, 0x009, 0x00e, 0x081,
+ 0x364, 0x233, 0x004, 0x009, 0x003, 0x083, 0x0c1, 0x22a,
+ 0x044, 0x39c, 0x008, 0x3d7, 0x200, 0x27b, 0x0c0, 0x271,
+ 0x3b7, 0x300, 0x387, 0x3df, 0x044, 0x1b8, 0x000, 0x3b7,
+ 0x221, 0x091, 0x189, 0x044, 0x1a9, 0x000, 0x3b7, 0x201,
+ 0x387, 0x0a6, 0x044, 0x1bd, 0x000, 0x3c5, 0x021, 0x020,
+ 0x3c4, 0x3ed, 0x020, 0x3b7, 0x200, 0x387, 0x020, 0x044,
+ 0x020, 0x00a, 0x009, 0x32a, 0x347, 0x271, 0x270, 0x000,
+ 0x055, 0x3a7, 0x071, 0x307, 0x21d, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x233, 0x044, 0x2f6, 0x002, 0x009, 0x007,
+ 0x0c1, 0x2d3, 0x3c5, 0x100, 0x200, 0x081, 0x18a, 0x245,
+ 0x232, 0x364, 0x233, 0x006, 0x001, 0x013, 0x044, 0x2f6,
+ 0x002, 0x009, 0x00e, 0x081, 0x364, 0x233, 0x004, 0x009,
+ 0x003, 0x093, 0x0c2, 0x22a, 0x044, 0x39c, 0x008, 0x0c0,
+ 0x27b, 0x0c0, 0x272, 0x3b7, 0x300, 0x387, 0x020, 0x044,
+ 0x1a9, 0x000, 0x0a1, 0x397, 0x205, 0x247, 0x27f, 0x044,
+ 0x1bd, 0x000, 0x3d5, 0x204, 0x020, 0x3c4, 0x2ff, 0x020,
+ 0x0a0, 0x387, 0x021, 0x184, 0x044, 0x020, 0x00a, 0x009,
+ 0x336, 0x347, 0x272, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x2e4, 0x002, 0x3c4,
+ 0x3fe, 0x200, 0x3c4, 0x3fe, 0x26f, 0x344, 0x21f, 0x232,
+ 0x0c0, 0x020, 0x0a3, 0x1ab, 0x2c5, 0x020, 0x044, 0x20d,
+ 0x000, 0x044, 0x2f3, 0x000, 0x044, 0x3b5, 0x000, 0x044,
+ 0x3f0, 0x000, 0x0a9, 0x1ab, 0x09f, 0x18a, 0x208, 0x304,
+ 0x055, 0x225, 0x044, 0x394, 0x00a, 0x307, 0x055, 0x19b,
+ 0x384, 0x003, 0x227, 0x1a3, 0x225, 0x044, 0x0a4, 0x00b,
+ 0x0a7, 0x020, 0x022, 0x008, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x006, 0x387, 0x07f, 0x000, 0x013, 0x399, 0x240,
+ 0x385, 0x07f, 0x306, 0x270, 0x001, 0x06c, 0x399, 0x240,
+ 0x327, 0x270, 0x3a4, 0x380, 0x226, 0x001, 0x015, 0x327,
+ 0x220, 0x324, 0x232, 0x009, 0x019, 0x364, 0x232, 0x001,
+ 0x001, 0x02b, 0x327, 0x01d, 0x307, 0x01b, 0x190, 0x282,
+ 0x003, 0x041, 0x307, 0x01b, 0x191, 0x380, 0x010, 0x282,
+ 0x003, 0x034, 0x000, 0x019, 0x327, 0x220, 0x324, 0x232,
+ 0x001, 0x00a, 0x327, 0x270, 0x384, 0x07f, 0x3a4, 0x07f,
+ 0x000, 0x008, 0x327, 0x270, 0x3a4, 0x07f, 0x001, 0x005,
+ 0x286, 0x001, 0x02f, 0x307, 0x270, 0x384, 0x07f, 0x386,
+ 0x07f, 0x009, 0x00a, 0x387, 0x080, 0x382, 0x07f, 0x240,
+ 0x270, 0x000, 0x01c, 0x141, 0x270, 0x364, 0x270, 0x07f,
+ 0x009, 0x015, 0x000, 0x013, 0x3c0, 0x040, 0x270, 0x000,
+ 0x00e, 0x307, 0x270, 0x196, 0x384, 0x00f, 0x266, 0x009,
+ 0x001, 0x008, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009,
+ 0x045, 0x309, 0x232, 0x00b, 0x013, 0x347, 0x271, 0x270,
+ 0x374, 0x27b, 0x200, 0x009, 0x005, 0x347, 0x272, 0x270,
+ 0x044, 0x35f, 0x009, 0x020, 0x0b7, 0x009, 0x327, 0x232,
+ 0x1a2, 0x023, 0x0b7, 0x009, 0x307, 0x270, 0x196, 0x119,
+ 0x021, 0x0b7, 0x009, 0x307, 0x232, 0x181, 0x023, 0x0b7,
+ 0x009, 0x364, 0x270, 0x080, 0x029, 0x0b7, 0x009, 0x083,
+ 0x18c, 0x245, 0x232, 0x3c4, 0x3f7, 0x233, 0x3c4, 0x380,
+ 0x270, 0x3c0, 0x080, 0x270, 0x044, 0x35f, 0x009, 0x397,
+ 0x200, 0x044, 0x1b1, 0x001, 0x020, 0x09c, 0x009, 0x000,
+ 0x000, 0x000, 0x002, 0x000, 0x00a, 0x000, 0x02a, 0x000,
+ 0x0aa, 0x002, 0x0aa, 0x00a, 0x0aa, 0x02a, 0x0aa, 0x06a,
+ 0x0aa, 0x07a, 0x0aa, 0x07e, 0x0aa, 0x07f, 0x0aa, 0x07f,
+ 0x0ba, 0x07f, 0x0be, 0x07f, 0x0fe, 0x07f, 0x0ff, 0x007,
+ 0x000, 0x017, 0x001, 0x027, 0x007, 0x037, 0x00f, 0x047,
+ 0x03f, 0x080, 0x0ff, 0x364, 0x270, 0x07f, 0x307, 0x270,
+ 0x384, 0x07f, 0x009, 0x00d, 0x089, 0x186, 0x306, 0x270,
+ 0x001, 0x007, 0x3c5, 0x001, 0x232, 0x000, 0x005, 0x3c4,
+ 0x3fe, 0x232, 0x347, 0x270, 0x27c, 0x3c4, 0x007, 0x27c,
+ 0x374, 0x27b, 0x200, 0x001, 0x00a, 0x364, 0x233, 0x008,
+ 0x001, 0x005, 0x044, 0x0fe, 0x00a, 0x0a3, 0x325, 0x27b,
+ 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x305, 0x27c, 0x044,
+ 0x1bd, 0x000, 0x307, 0x270, 0x192, 0x384, 0x00f, 0x3b9,
+ 0x200, 0x324, 0x233, 0x001, 0x014, 0x229, 0x387, 0x009,
+ 0x189, 0x395, 0x333, 0x280, 0x051, 0x2c7, 0x27d, 0x1c7,
+ 0x27d, 0x101, 0x051, 0x2c5, 0x27d, 0x000, 0x011, 0x0c0,
+ 0x27d, 0x384, 0x00f, 0x001, 0x00b, 0x349, 0x27d, 0x27d,
+ 0x3c5, 0x001, 0x27d, 0x111, 0x000, 0x3f7, 0x0a2, 0x325,
+ 0x27b, 0x307, 0x27d, 0x044, 0x1bd, 0x000, 0x374, 0x27b,
+ 0x200, 0x001, 0x02c, 0x347, 0x270, 0x27a, 0x3c4, 0x07f,
+ 0x27a, 0x0c0, 0x279, 0x387, 0x009, 0x189, 0x395, 0x353,
+ 0x051, 0x041, 0x101, 0x051, 0x2c7, 0x279, 0x101, 0x043,
+ 0x322, 0x27a, 0x00b, 0x004, 0x000, 0x3f4, 0x1c7, 0x279,
+ 0x0a3, 0x325, 0x27b, 0x044, 0x192, 0x000, 0x384, 0x0ff,
+ 0x305, 0x279, 0x044, 0x1bd, 0x000, 0x327, 0x220, 0x324,
+ 0x232, 0x009, 0x022, 0x0c0, 0x27e, 0x307, 0x270, 0x197,
+ 0x00a, 0x004, 0x000, 0x005, 0x3c5, 0x018, 0x27e, 0x0a4,
+ 0x222, 0x2c5, 0x27e, 0x0a4, 0x325, 0x27b, 0x044, 0x192,
+ 0x000, 0x384, 0x3e0, 0x3c4, 0x01f, 0x27e, 0x305, 0x27e,
+ 0x044, 0x1bd, 0x000, 0x045, 0x044, 0x1a9, 0x000, 0x083,
+ 0x18c, 0x208, 0x244, 0x232, 0x3c5, 0x001, 0x00c, 0x0a3,
+ 0x325, 0x27b, 0x088, 0x044, 0x1a9, 0x000, 0x387, 0x3f8,
+ 0x044, 0x1b8, 0x000, 0x020, 0x2f6, 0x002, 0x3c7, 0x020,
+ 0x280, 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x247, 0x247,
+ 0x151, 0x280, 0x001, 0x016, 0x0a1, 0x044, 0x182, 0x000,
+ 0x3b7, 0x215, 0x044, 0x192, 0x000, 0x306, 0x247, 0x001,
+ 0x3f1, 0x0a2, 0x364, 0x22b, 0x003, 0x001, 0x003, 0x0a5,
+ 0x020, 0x182, 0x000, 0x1f6, 0x185, 0x1f5, 0x129, 0x1f4,
+ 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff, 0x1f6, 0x189, 0x1f5,
+ 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce, 0x123, 0x0f3, 0x0d2,
+ 0x0ff, 0x364, 0x233, 0x020, 0x009, 0x051, 0x307, 0x272,
+ 0x196, 0x384, 0x00f, 0x0c9, 0x286, 0x242, 0x286, 0x397,
+ 0x00a, 0x189, 0x395, 0x05f, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x00a, 0x051, 0x2c7, 0x284, 0x101, 0x051,
+ 0x2c7, 0x288, 0x3a4, 0x07f, 0x040, 0x307, 0x272, 0x384,
+ 0x07f, 0x222, 0x042, 0x00b, 0x005, 0x101, 0x000, 0x3ed,
+ 0x347, 0x284, 0x285, 0x3c4, 0x00f, 0x284, 0x1d3, 0x285,
+ 0x1d6, 0x288, 0x342, 0x288, 0x286, 0x003, 0x008, 0x156,
+ 0x285, 0x151, 0x286, 0x00b, 0x3fc, 0x362, 0x285, 0x003,
+ 0x003, 0x004, 0x0c3, 0x285, 0x307, 0x285, 0x0a1, 0x184,
+ 0x305, 0x284, 0x044, 0x1bd, 0x000, 0x045, 0x3db, 0x071,
+ 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c, 0x176, 0x1bf, 0x00d,
+ 0x007, 0x006, 0x003, 0x002, 0x005, 0x0b3, 0x156, 0x1fa,
+ 0x27f, 0x080, 0x09e, 0x14b, 0x1f0, 0x27f, 0x080, 0x08b,
+ 0x140, 0x1e9, 0x27f, 0x081, 0x138, 0x1e3, 0x27f, 0x00f,
+ 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080, 0x0a3, 0x145, 0x1e7,
+ 0x27f, 0x080, 0x095, 0x13d, 0x1e0, 0x27f, 0x08b, 0x136,
+ 0x1db, 0x27f, 0x0c5, 0x28b, 0x347, 0x271, 0x281, 0x3c4,
+ 0x07f, 0x281, 0x0c5, 0x282, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x0c2, 0x282, 0x044, 0x1bc, 0x00a, 0x347, 0x283,
+ 0x289, 0x307, 0x241, 0x364, 0x233, 0x002, 0x001, 0x004,
+ 0x307, 0x225, 0x384, 0x0ff, 0x242, 0x283, 0x00b, 0x056,
+ 0x0c3, 0x28b, 0x302, 0x289, 0x397, 0x00a, 0x189, 0x395,
+ 0x0ca, 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x004,
+ 0x051, 0x2c7, 0x28c, 0x2a9, 0x121, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003, 0x007,
+ 0x001, 0x005, 0x101, 0x000, 0x3ed, 0x1d5, 0x28c, 0x347,
+ 0x241, 0x281, 0x364, 0x233, 0x002, 0x001, 0x005, 0x347,
+ 0x225, 0x281, 0x3c4, 0x0ff, 0x281, 0x342, 0x289, 0x281,
+ 0x347, 0x28c, 0x282, 0x366, 0x282, 0x00f, 0x009, 0x006,
+ 0x1c0, 0x281, 0x0c1, 0x282, 0x044, 0x1bc, 0x00a, 0x001,
+ 0x00d, 0x141, 0x28b, 0x362, 0x28b, 0x03f, 0x001, 0x00d,
+ 0x151, 0x283, 0x000, 0x3f5, 0x362, 0x28b, 0x00a, 0x003,
+ 0x004, 0x0ca, 0x28b, 0x397, 0x00a, 0x189, 0x395, 0x0d8,
+ 0x044, 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x013, 0x327,
+ 0x24c, 0x131, 0x003, 0x005, 0x105, 0x000, 0x3fc, 0x051,
+ 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121, 0x040, 0x307, 0x271,
+ 0x384, 0x07f, 0x282, 0x042, 0x003, 0x005, 0x101, 0x000,
+ 0x3f0, 0x1d6, 0x28a, 0x387, 0x00a, 0x189, 0x395, 0x0d3,
+ 0x300, 0x28a, 0x050, 0x247, 0x28a, 0x30a, 0x28b, 0x3b7,
+ 0x201, 0x184, 0x305, 0x28a, 0x044, 0x1bd, 0x000, 0x045,
+ 0x0c1, 0x283, 0x342, 0x282, 0x281, 0x003, 0x006, 0x141,
+ 0x283, 0x000, 0x3f9, 0x045, 0x000, 0x301, 0x103, 0x307,
+ 0x006, 0x10e, 0x21e, 0x33e, 0x13c, 0x038, 0x078, 0x1f8,
+ 0x0f0, 0x000, 0x001, 0x003, 0x002, 0x006, 0x187, 0x227,
+ 0x387, 0x00a, 0x189, 0x395, 0x1c8, 0x300, 0x227, 0x111,
+ 0x050, 0x247, 0x24c, 0x1d7, 0x24c, 0x384, 0x0ff, 0x285,
+ 0x044, 0x226, 0x00a, 0x364, 0x22a, 0x001, 0x001, 0x00e,
+ 0x3d5, 0x200, 0x232, 0x040, 0x1aa, 0x285, 0x3b7, 0x220,
+ 0x044, 0x1bd, 0x000, 0x042, 0x364, 0x22a, 0x002, 0x001,
+ 0x022, 0x040, 0x3a7, 0x073, 0x044, 0x192, 0x000, 0x384,
+ 0x31f, 0x044, 0x226, 0x00a, 0x1a4, 0x285, 0x3a7, 0x073,
+ 0x044, 0x1bd, 0x000, 0x3a7, 0x070, 0x044, 0x192, 0x000,
+ 0x19a, 0x18a, 0x043, 0x285, 0x3a7, 0x070, 0x044, 0x1bd,
+ 0x000, 0x045, 0x040, 0x387, 0x00a, 0x189, 0x395, 0x1d5,
+ 0x300, 0x278, 0x051, 0x042, 0x045, 0x00a, 0x058, 0x026,
+ 0x015, 0x044, 0x032, 0x0c0, 0x278, 0x0a0, 0x397, 0x00a,
+ 0x189, 0x395, 0x231, 0x280, 0x050, 0x247, 0x228, 0x3c4,
+ 0x00f, 0x228, 0x347, 0x228, 0x229, 0x040, 0x307, 0x278,
+ 0x001, 0x008, 0x340, 0x229, 0x228, 0x111, 0x000, 0x3fa,
+ 0x042, 0x193, 0x0c0, 0x227, 0x347, 0x225, 0x226, 0x141,
+ 0x227, 0x366, 0x227, 0x00e, 0x001, 0x00b, 0x342, 0x228,
+ 0x226, 0x007, 0x3f6, 0x140, 0x226, 0x001, 0x011, 0x121,
+ 0x2e2, 0x006, 0x009, 0x3cc, 0x141, 0x278, 0x366, 0x278,
+ 0x005, 0x009, 0x3c4, 0x0a1, 0x000, 0x006, 0x044, 0x1da,
+ 0x00a, 0x0a0, 0x045, 0x044, 0x333, 0x008, 0x020, 0x181,
+ 0x006, 0x345, 0x21d, 0x232, 0x364, 0x23f, 0x001, 0x029,
+ 0x2e4, 0x002, 0x2c7, 0x234, 0x399, 0x200, 0x245, 0x233,
+ 0x364, 0x234, 0x020, 0x009, 0x005, 0x208, 0x244, 0x233,
+ 0x083, 0x18c, 0x208, 0x244, 0x036, 0x2e4, 0x008, 0x001,
+ 0x007, 0x345, 0x220, 0x232, 0x000, 0x007, 0x081, 0x18b,
+ 0x208, 0x244, 0x232, 0x307, 0x235, 0x226, 0x3a4, 0x001,
+ 0x001, 0x010, 0x327, 0x234, 0x2e4, 0x001, 0x009, 0x041,
+ 0x041, 0x044, 0x20d, 0x000, 0x044, 0x2f3, 0x000, 0x043,
+ 0x0a3, 0x1ad, 0x3a5, 0x039, 0x326, 0x236, 0x001, 0x02f,
+ 0x3a7, 0x0bf, 0x1a7, 0x3a5, 0x039, 0x326, 0x236, 0x001,
+ 0x026, 0x327, 0x234, 0x307, 0x235, 0x286, 0x384, 0x002,
+ 0x001, 0x01d, 0x2e4, 0x002, 0x009, 0x051, 0x041, 0x0a1,
+ 0x1ad, 0x2c5, 0x024, 0x307, 0x024, 0x19a, 0x002, 0x005,
+ 0x2a8, 0x2c4, 0x024, 0x044, 0x3b5, 0x000, 0x044, 0x0f2,
+ 0x001, 0x044, 0x3f0, 0x000, 0x043, 0x000, 0x067, 0x364,
+ 0x026, 0x001, 0x009, 0x3c6, 0x041, 0x0a0, 0x044, 0x301,
+ 0x000, 0x364, 0x26f, 0x002, 0x021, 0x310, 0x00a, 0x044,
+ 0x290, 0x001, 0x0c1, 0x26f, 0x043, 0x307, 0x020, 0x193,
+ 0x384, 0x007, 0x110, 0x001, 0x00c, 0x364, 0x232, 0x010,
+ 0x009, 0x007, 0x044, 0x044, 0x001, 0x000, 0x3a3, 0x3c4,
+ 0x3fe, 0x26f, 0x3c5, 0x001, 0x200, 0x3c5, 0x010, 0x020,
+ 0x3c4, 0x31f, 0x020, 0x000, 0x395, 0x364, 0x026, 0x002,
+ 0x009, 0x3c5, 0x041, 0x044, 0x003, 0x001, 0x043, 0x307,
+ 0x020, 0x197, 0x384, 0x007, 0x110, 0x001, 0x00f, 0x364,
+ 0x232, 0x020, 0x009, 0x00a, 0x044, 0x051, 0x001, 0x3c5,
+ 0x001, 0x027, 0x000, 0x3ab, 0x3c4, 0x3fc, 0x26f, 0x3c5,
+ 0x001, 0x200, 0x3c5, 0x100, 0x020, 0x087, 0x188, 0x208,
+ 0x244, 0x020, 0x000, 0x39b, 0x044, 0x11a, 0x001, 0x307,
+ 0x234, 0x306, 0x235, 0x384, 0x003, 0x021, 0x2e4, 0x002,
+ 0x347, 0x234, 0x235, 0x366, 0x26f, 0x002, 0x021, 0x2e4,
+ 0x002, 0x364, 0x234, 0x003, 0x021, 0x2e4, 0x002, 0x327,
+ 0x21c, 0x324, 0x020, 0x021, 0x385, 0x00a, 0x020, 0x247,
+ 0x002, 0x364, 0x23f, 0x001, 0x021, 0x38e, 0x00a, 0x020,
+ 0x247, 0x002, 0x345, 0x21d, 0x232, 0x020, 0x247, 0x002,
+ 0x2c7, 0x241, 0x307, 0x242, 0x286, 0x001, 0x092, 0x0c1,
+ 0x22a, 0x327, 0x242, 0x326, 0x241, 0x3a4, 0x0ff, 0x001,
+ 0x005, 0x3c4, 0x38f, 0x020, 0x347, 0x241, 0x242, 0x347,
+ 0x241, 0x226, 0x307, 0x21d, 0x304, 0x241, 0x001, 0x00a,
+ 0x0c3, 0x22a, 0x347, 0x241, 0x243, 0x044, 0x39f, 0x002,
+ 0x327, 0x241, 0x3a4, 0x0ff, 0x2c7, 0x225, 0x3c4, 0x300,
+ 0x230, 0x2c5, 0x230, 0x044, 0x099, 0x00b, 0x081, 0x18b,
+ 0x304, 0x226, 0x009, 0x00d, 0x364, 0x232, 0x100, 0x009,
+ 0x008, 0x081, 0x18a, 0x208, 0x044, 0x1b8, 0x000, 0x044,
+ 0x186, 0x001, 0x327, 0x226, 0x003, 0x032, 0x044, 0x2f6,
+ 0x002, 0x001, 0x019, 0x044, 0x237, 0x00a, 0x347, 0x241,
+ 0x226, 0x081, 0x306, 0x22a, 0x384, 0x003, 0x001, 0x005,
+ 0x347, 0x243, 0x226, 0x044, 0x032, 0x009, 0x002, 0x02d,
+ 0x000, 0x031, 0x044, 0x237, 0x00a, 0x009, 0x01c, 0x044,
+ 0x082, 0x00b, 0x0a2, 0x1ab, 0x324, 0x020, 0x001, 0x023,
+ 0x364, 0x233, 0x001, 0x000, 0x01e, 0x045, 0x044, 0x186,
+ 0x001, 0x3c5, 0x003, 0x22a, 0x347, 0x241, 0x243, 0x000,
+ 0x3c7, 0x3a4, 0x3ff, 0x001, 0x00e, 0x327, 0x21c, 0x324,
+ 0x020, 0x009, 0x006, 0x043, 0x3a7, 0x0ff, 0x041, 0x0c0,
+ 0x270, 0x044, 0x082, 0x00b, 0x3c4, 0x3fc, 0x22a, 0x327,
+ 0x21c, 0x324, 0x020, 0x009, 0x010, 0x307, 0x270, 0x003,
+ 0x005, 0x020, 0x181, 0x006, 0x043, 0x3a7, 0x0ff, 0x041,
+ 0x020, 0x181, 0x006, 0x045, 0x2c7, 0x243, 0x307, 0x250,
+ 0x286, 0x001, 0x3de, 0x0c2, 0x22a, 0x041, 0x044, 0x39f,
+ 0x002, 0x043, 0x2c7, 0x226, 0x3a4, 0x0ff, 0x2c7, 0x225,
+ 0x3c4, 0x0ff, 0x230, 0x1a7, 0x2c5, 0x230, 0x307, 0x21d,
+ 0x304, 0x243, 0x001, 0x013, 0x3c4, 0x0ff, 0x243, 0x347,
+ 0x243, 0x241, 0x347, 0x243, 0x242, 0x3c5, 0x003, 0x22a,
+ 0x044, 0x186, 0x001, 0x000, 0x005, 0x044, 0x191, 0x001,
+ 0x327, 0x243, 0x003, 0x394, 0x000, 0x362, 0x364, 0x232,
+ 0x004, 0x001, 0x008, 0x044, 0x232, 0x000, 0x3c4, 0x3fb,
+ 0x232, 0x364, 0x232, 0x008, 0x001, 0x008, 0x044, 0x326,
+ 0x000, 0x3c4, 0x3f7, 0x232, 0x045, 0x3b7, 0x221, 0x083,
+ 0x189, 0x044, 0x1a9, 0x000, 0x083, 0x020, 0x1b7, 0x000,
+ 0x2c7, 0x24d, 0x364, 0x233, 0x002, 0x001, 0x019, 0x3b9,
+ 0x381, 0x121, 0x081, 0x18a, 0x225, 0x304, 0x24d, 0x001,
+ 0x005, 0x044, 0x1ff, 0x00b, 0x140, 0x24d, 0x00b, 0x008,
+ 0x1a3, 0x044, 0x1ff, 0x00b, 0x000, 0x012, 0x307, 0x220,
+ 0x284, 0x001, 0x00d, 0x3c4, 0x33f, 0x232, 0x1b6, 0x3a4,
+ 0x0c0, 0x2c5, 0x232, 0x020, 0x18a, 0x00b, 0x327, 0x24d,
+ 0x364, 0x020, 0x070, 0x001, 0x00a, 0x399, 0x383, 0x101,
+ 0x326, 0x22b, 0x234, 0x001, 0x02e, 0x3c5, 0x080, 0x233,
+ 0x3d5, 0x200, 0x232, 0x364, 0x026, 0x001, 0x001, 0x00a,
+ 0x3c5, 0x004, 0x232, 0x044, 0x20d, 0x000, 0x307, 0x24d,
+ 0x307, 0x24d, 0x384, 0x007, 0x001, 0x09a, 0x113, 0x001,
+ 0x0a8, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8,
+ 0x103, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x021, 0x000,
+ 0x0af, 0x0a7, 0x1a7, 0x324, 0x020, 0x001, 0x014, 0x327,
+ 0x24d, 0x326, 0x22b, 0x397, 0x370, 0x364, 0x233, 0x002,
+ 0x001, 0x006, 0x087, 0x18b, 0x385, 0x070, 0x234, 0x001,
+ 0x02b, 0x3c5, 0x100, 0x233, 0x364, 0x026, 0x002, 0x001,
+ 0x008, 0x3c5, 0x008, 0x232, 0x044, 0x3b5, 0x000, 0x307,
+ 0x24d, 0x384, 0x070, 0x001, 0x091, 0x382, 0x030, 0x001,
+ 0x0a6, 0x3b7, 0x301, 0x044, 0x192, 0x000, 0x384, 0x3c7,
+ 0x385, 0x018, 0x044, 0x1bd, 0x000, 0x3c4, 0x3f7, 0x024,
+ 0x000, 0x0a4, 0x347, 0x24d, 0x22b, 0x364, 0x233, 0x006,
+ 0x001, 0x033, 0x307, 0x22b, 0x364, 0x233, 0x002, 0x001,
+ 0x023, 0x364, 0x233, 0x080, 0x001, 0x00a, 0x307, 0x24d,
+ 0x197, 0x384, 0x007, 0x044, 0x3a4, 0x008, 0x364, 0x233,
+ 0x100, 0x001, 0x00c, 0x307, 0x24d, 0x19b, 0x384, 0x007,
+ 0x385, 0x010, 0x044, 0x3a4, 0x008, 0x3c4, 0x27f, 0x233,
+ 0x000, 0x00b, 0x3c5, 0x003, 0x22a, 0x197, 0x044, 0x39c,
+ 0x008, 0x003, 0x298, 0x044, 0x082, 0x00b, 0x0a2, 0x1ab,
+ 0x324, 0x020, 0x021, 0x181, 0x006, 0x045, 0x3b7, 0x301,
+ 0x087, 0x044, 0x1b7, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1b7, 0x000, 0x3c4, 0x3f7, 0x021, 0x000, 0x36c, 0x3b7,
+ 0x301, 0x044, 0x192, 0x000, 0x384, 0x3f8, 0x105, 0x044,
+ 0x1bd, 0x000, 0x3b7, 0x211, 0x082, 0x044, 0x1a9, 0x000,
+ 0x3c5, 0x008, 0x021, 0x344, 0x21f, 0x021, 0x3b7, 0x250,
+ 0x088, 0x044, 0x1a9, 0x000, 0x3b7, 0x211, 0x082, 0x044,
+ 0x1a9, 0x000, 0x000, 0x347, 0x3b7, 0x301, 0x387, 0x3c7,
+ 0x044, 0x1b8, 0x000, 0x3a7, 0x060, 0x083, 0x044, 0x1b7,
+ 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1b7, 0x000,
+ 0x3c4, 0x3f7, 0x024, 0x000, 0x36f, 0x3b7, 0x301, 0x044,
+ 0x192, 0x000, 0x384, 0x3c7, 0x385, 0x028, 0x044, 0x1bd,
+ 0x000, 0x3c5, 0x048, 0x024, 0x3a7, 0x060, 0x083, 0x044,
+ 0x1a9, 0x000, 0x3b7, 0x070, 0x083, 0x18a, 0x044, 0x1a9,
+ 0x000, 0x000, 0x351, 0x307, 0x22b, 0x284, 0x2a8, 0x2c4,
+ 0x24d, 0x245, 0x24d, 0x2a8, 0x045, 0x2f4, 0x200, 0x001,
+ 0x008, 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287,
+ 0x385, 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010,
+ 0x029, 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384,
+ 0x0ff, 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d,
+ 0x040, 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d,
+ 0x042, 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005,
+ 0x3c5, 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264,
+ 0x080, 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5,
+ 0x080, 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f,
+ 0x02d, 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff,
+ 0x1b8, 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021,
+ 0x3c4, 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a,
+ 0x020, 0x181, 0x006, 0x289, 0x00a, 0x004, 0x2c7, 0x251,
+ 0x22e, 0x364, 0x026, 0x001, 0x021, 0x181, 0x006, 0x041,
+ 0x3a4, 0x01f, 0x001, 0x005, 0x044, 0x0d9, 0x001, 0x043,
+ 0x1b7, 0x001, 0x005, 0x044, 0x182, 0x000, 0x020, 0x181,
+ 0x006, 0x2e4, 0x002, 0x009, 0x01b, 0x3c4, 0x2ff, 0x232,
+ 0x120, 0x001, 0x012, 0x3d5, 0x300, 0x232, 0x044, 0x099,
+ 0x00b, 0x32a, 0x026, 0x00a, 0x008, 0x044, 0x20d, 0x000,
+ 0x044, 0x232, 0x000, 0x020, 0x181, 0x006, 0x042, 0x307,
+ 0x240, 0x20a, 0x040, 0x020, 0x181, 0x006, 0x289, 0x00a,
+ 0x004, 0x2c7, 0x256, 0x22e, 0x364, 0x026, 0x002, 0x021,
+ 0x181, 0x006, 0x041, 0x3a4, 0x01f, 0x001, 0x005, 0x044,
+ 0x0de, 0x001, 0x043, 0x1b7, 0x001, 0x005, 0x044, 0x0e2,
+ 0x001, 0x020, 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009,
+ 0x007, 0x044, 0x340, 0x001, 0x247, 0x249, 0x054, 0x347,
+ 0x249, 0x248, 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018,
+ 0x364, 0x23f, 0x001, 0x001, 0x006, 0x245, 0x057, 0x000,
+ 0x005, 0x208, 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0,
+ 0x364, 0x23f, 0x002, 0x001, 0x009, 0x245, 0x057, 0x044,
+ 0x27d, 0x001, 0x000, 0x005, 0x208, 0x244, 0x057, 0x020,
+ 0x181, 0x006, 0x2c7, 0x246, 0x3b7, 0x300, 0x044, 0x192,
+ 0x000, 0x384, 0x33f, 0x327, 0x246, 0x3a4, 0x0c0, 0x285,
+ 0x3b7, 0x300, 0x044, 0x1bd, 0x000, 0x3b7, 0x200, 0x044,
+ 0x192, 0x000, 0x0a3, 0x1a9, 0x2a8, 0x284, 0x040, 0x307,
+ 0x246, 0x384, 0x030, 0x264, 0x010, 0x001, 0x012, 0x040,
+ 0x041, 0x3b7, 0x201, 0x044, 0x192, 0x000, 0x384, 0x3e0,
+ 0x385, 0x006, 0x044, 0x1bd, 0x000, 0x043, 0x042, 0x185,
+ 0x2a8, 0x284, 0x227, 0x042, 0x285, 0x3b7, 0x200, 0x044,
+ 0x1bd, 0x000, 0x307, 0x246, 0x384, 0x007, 0x044, 0x375,
+ 0x00b, 0x3a7, 0x073, 0x307, 0x246, 0x184, 0x00a, 0x014,
+ 0x044, 0x192, 0x000, 0x384, 0x3e2, 0x327, 0x246, 0x1b5,
+ 0x3a4, 0x01c, 0x285, 0x3a7, 0x073, 0x044, 0x1bd, 0x000,
+ 0x000, 0x006, 0x081, 0x044, 0x1a9, 0x000, 0x020, 0x181,
+ 0x006, 0x3c5, 0x001, 0x00c, 0x187, 0x0a7, 0x1a7, 0x2a8,
+ 0x324, 0x00c, 0x225, 0x2c7, 0x00c, 0x045, 0x04e, 0x087,
+ 0x011, 0x054, 0x084, 0x020, 0x01e, 0x00f, 0x018, 0x01d,
+ 0x029, 0x00b, 0x01d, 0x02c, 0x331, 0x04e, 0x020, 0x0d3,
+ 0x000, 0x020, 0x167, 0x00b, 0x020, 0x3f6, 0x00a, 0x020,
+ 0x096, 0x00b, 0x020, 0x0d3, 0x000, 0x020, 0x0d3, 0x000,
+ 0x020, 0x00d, 0x00b, 0x020, 0x179, 0x00b, 0x0c1, 0x244,
+ 0x3c5, 0x040, 0x245, 0x3c4, 0x3bf, 0x200, 0x020, 0x0d3,
+ 0x000, 0x020, 0x2c3, 0x008, 0x04e, 0x04e, 0x045, 0x0c2,
+ 0x201, 0x045, 0x020, 0x3ed, 0x00a, 0x020, 0x256, 0x002,
+ 0x020, 0x256, 0x002, 0x020, 0x256, 0x002, 0x020, 0x23f,
+ 0x006, 0x020, 0x160, 0x00b, 0x020, 0x28d, 0x00b, 0x020,
+ 0x257, 0x006, 0x020, 0x117, 0x008, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x2d2, 0x006, 0x020, 0x1e2, 0x007, 0x020,
+ 0x160, 0x00b, 0x020, 0x3a8, 0x001, 0x020, 0x160, 0x00b,
+ 0x020, 0x160, 0x00b, 0x020, 0x2d5, 0x006, 0x020, 0x326,
+ 0x006, 0x020, 0x350, 0x006, 0x020, 0x1dc, 0x007, 0x020,
+ 0x35a, 0x006, 0x020, 0x177, 0x00b, 0x020, 0x38e, 0x006,
+ 0x020, 0x011, 0x007, 0x020, 0x016, 0x007, 0x020, 0x31e,
+ 0x007, 0x020, 0x324, 0x007, 0x020, 0x3b6, 0x007, 0x020,
+ 0x02a, 0x007, 0x020, 0x041, 0x007, 0x020, 0x060, 0x007,
+ 0x020, 0x06d, 0x007, 0x020, 0x072, 0x007, 0x020, 0x2d1,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x252, 0x007,
+ 0x020, 0x293, 0x007, 0x020, 0x2fb, 0x00b, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x16f, 0x007, 0x020,
+ 0x185, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x160,
+ 0x00b, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020,
+ 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007,
+ 0x020, 0x1dc, 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x1dc,
+ 0x007, 0x020, 0x1dc, 0x007, 0x020, 0x077, 0x007, 0x020,
+ 0x0b1, 0x007, 0x020, 0x103, 0x007, 0x020, 0x1c5, 0x008,
+ 0x020, 0x1dc, 0x007, 0x020, 0x168, 0x007, 0x287, 0x264,
+ 0x004, 0x001, 0x049, 0x3c5, 0x00c, 0x00c, 0x3c5, 0x020,
+ 0x33b, 0x18e, 0x00a, 0x013, 0x044, 0x04c, 0x004, 0x3a4,
+ 0x001, 0x001, 0x014, 0x364, 0x052, 0x001, 0x009, 0x093,
+ 0x044, 0x0bc, 0x004, 0x000, 0x08e, 0x3c4, 0x2ff, 0x053,
+ 0x3c4, 0x030, 0x027, 0x000, 0x3ec, 0x3c4, 0x3fe, 0x052,
+ 0x000, 0x081, 0x043, 0x327, 0x33b, 0x020, 0x1de, 0x007,
+ 0x2e4, 0x080, 0x009, 0x00d, 0x3d5, 0x220, 0x33c, 0x100,
+ 0x009, 0x06e, 0x044, 0x3b6, 0x003, 0x000, 0x06c, 0x3d5,
+ 0x120, 0x33c, 0x100, 0x009, 0x022, 0x044, 0x066, 0x004,
+ 0x000, 0x061, 0x384, 0x003, 0x3c4, 0x3df, 0x33b, 0x262,
+ 0x003, 0x001, 0x3d9, 0x3c4, 0x32d, 0x33c, 0x3c4, 0x3ef,
+ 0x027, 0x2e4, 0x040, 0x009, 0x3d5, 0x100, 0x009, 0x007,
+ 0x044, 0x063, 0x004, 0x000, 0x046, 0x364, 0x200, 0x004,
+ 0x029, 0x1dc, 0x007, 0x0c0, 0x2d8, 0x111, 0x001, 0x028,
+ 0x3c5, 0x002, 0x33c, 0x3c5, 0x001, 0x33b, 0x344, 0x21f,
+ 0x33c, 0x287, 0x19b, 0x247, 0x361, 0x287, 0x183, 0x19b,
+ 0x247, 0x360, 0x0c0, 0x35e, 0x2e4, 0x020, 0x001, 0x004,
+ 0x0c5, 0x35e, 0x2e4, 0x010, 0x001, 0x005, 0x3c5, 0x080,
+ 0x33c, 0x289, 0x384, 0x010, 0x245, 0x33c, 0x044, 0x001,
+ 0x004, 0x364, 0x33c, 0x002, 0x001, 0x005, 0x3c5, 0x004,
+ 0x200, 0x364, 0x33c, 0x020, 0x009, 0x005, 0x044, 0x097,
+ 0x004, 0x020, 0x181, 0x006, 0x287, 0x19b, 0x11d, 0x001,
+ 0x002, 0x287, 0x1a3, 0x1b3, 0x19b, 0x180, 0x390, 0x1d7,
+ 0x040, 0x199, 0x390, 0x008, 0x040, 0x045, 0x000, 0x021,
+ 0x000, 0x073, 0x000, 0x05f, 0x000, 0x023, 0x000, 0x025,
+ 0x000, 0x033, 0x000, 0x035, 0x000, 0x037, 0x000, 0x041,
+ 0x000, 0x01f, 0x000, 0x021, 0x000, 0x023, 0x000, 0x031,
+ 0x000, 0x082, 0x000, 0x002, 0x020, 0x1dc, 0x007, 0x044,
+ 0x378, 0x003, 0x3c4, 0x3fd, 0x233, 0x000, 0x072, 0x2c7,
+ 0x363, 0x000, 0x06e, 0x2c7, 0x362, 0x000, 0x06a, 0x2c7,
+ 0x341, 0x000, 0x066, 0x2c7, 0x33f, 0x000, 0x062, 0x2c7,
+ 0x340, 0x000, 0x05e, 0x2c7, 0x344, 0x000, 0x05a, 0x2c7,
+ 0x342, 0x000, 0x056, 0x2c7, 0x343, 0x000, 0x052, 0x2c7,
+ 0x341, 0x2c7, 0x33f, 0x2c7, 0x340, 0x000, 0x04a, 0x307,
+ 0x340, 0x300, 0x341, 0x300, 0x33f, 0x282, 0x007, 0x006,
+ 0x2c7, 0x23e, 0x000, 0x03d, 0x043, 0x0a1, 0x020, 0x1de,
+ 0x007, 0x287, 0x384, 0x003, 0x181, 0x3c4, 0x3f3, 0x052,
+ 0x245, 0x052, 0x3a4, 0x00c, 0x3c4, 0x3f3, 0x33c, 0x2c5,
+ 0x33c, 0x000, 0x026, 0x3c5, 0x002, 0x233, 0x3c7, 0x3ff,
+ 0x352, 0x3c7, 0x3ff, 0x353, 0x1d3, 0x353, 0x0cf, 0x354,
+ 0x1cb, 0x354, 0x3c5, 0x0c0, 0x354, 0x3c9, 0x222, 0x355,
+ 0x3c5, 0x001, 0x33c, 0x0c0, 0x356, 0x1b0, 0x00a, 0x006,
+ 0x081, 0x18a, 0x247, 0x356, 0x1ab, 0x2c7, 0x357, 0x020,
+ 0x181, 0x006, 0x287, 0x001, 0x025, 0x197, 0x3a4, 0x00f,
+ 0x262, 0x003, 0x001, 0x03d, 0x131, 0x001, 0x020, 0x131,
+ 0x001, 0x021, 0x131, 0x001, 0x022, 0x131, 0x001, 0x025,
+ 0x131, 0x001, 0x026, 0x131, 0x001, 0x027, 0x132, 0x182,
+ 0x390, 0x377, 0x280, 0x04c, 0x042, 0x020, 0x1de, 0x007,
+ 0x044, 0x361, 0x003, 0x000, 0x3d4, 0x327, 0x373, 0x000,
+ 0x3f5, 0x327, 0x372, 0x000, 0x3f1, 0x327, 0x2b3, 0x322,
+ 0x2b2, 0x000, 0x3eb, 0x327, 0x374, 0x000, 0x3e7, 0x327,
+ 0x375, 0x000, 0x3e3, 0x327, 0x376, 0x000, 0x3df, 0x307,
+ 0x369, 0x282, 0x272, 0x369, 0x00e, 0x003, 0x108, 0x04c,
+ 0x000, 0x3d4, 0x364, 0x33c, 0x001, 0x001, 0x01b, 0x364,
+ 0x33b, 0x020, 0x009, 0x016, 0x0a0, 0x307, 0x04f, 0x181,
+ 0x00a, 0x003, 0x0a4, 0x100, 0x00b, 0x004, 0x3a5, 0x080,
+ 0x307, 0x050, 0x194, 0x384, 0x040, 0x225, 0x2c7, 0x392,
+ 0x081, 0x189, 0x314, 0x053, 0x001, 0x007, 0x3c5, 0x008,
+ 0x027, 0x000, 0x005, 0x3c4, 0x3f7, 0x027, 0x364, 0x33c,
+ 0x020, 0x001, 0x007, 0x347, 0x21d, 0x35a, 0x000, 0x018,
+ 0x347, 0x050, 0x368, 0x3b7, 0x201, 0x1a5, 0x140, 0x04f,
+ 0x00b, 0x00b, 0x364, 0x052, 0x020, 0x001, 0x3f3, 0x2c5,
+ 0x33c, 0x000, 0x005, 0x2a8, 0x2c4, 0x33c, 0x081, 0x189,
+ 0x304, 0x053, 0x001, 0x007, 0x3c5, 0x008, 0x027, 0x000,
+ 0x005, 0x3c4, 0x3f7, 0x027, 0x3c5, 0x080, 0x053, 0x3c4,
+ 0x37f, 0x053, 0x364, 0x33b, 0x020, 0x009, 0x00a, 0x364,
+ 0x33c, 0x020, 0x009, 0x005, 0x044, 0x132, 0x004, 0x020,
+ 0x08d, 0x006, 0x364, 0x026, 0x010, 0x009, 0x0e8, 0x0c9,
+ 0x2ce, 0x0c8, 0x30f, 0x362, 0x2d2, 0x002, 0x009, 0x004,
+ 0x0ce, 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x0af, 0x0c4,
+ 0x303, 0x364, 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303,
+ 0x0c1, 0x310, 0x364, 0x30f, 0x001, 0x009, 0x004, 0x152,
+ 0x310, 0x397, 0x2b2, 0x300, 0x30f, 0x04c, 0x2c7, 0x305,
+ 0x397, 0x2aa, 0x300, 0x303, 0x04c, 0x2c0, 0x305, 0x342,
+ 0x310, 0x305, 0x397, 0x2a1, 0x300, 0x303, 0x327, 0x305,
+ 0x04d, 0x044, 0x1bf, 0x003, 0x08c, 0x364, 0x30f, 0x002,
+ 0x001, 0x003, 0x108, 0x364, 0x30f, 0x004, 0x001, 0x003,
+ 0x101, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x364, 0x2cf, 0x002, 0x001, 0x014, 0x309, 0x013, 0x34b,
+ 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b,
+ 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336, 0x014, 0x397,
+ 0x2a1, 0x300, 0x303, 0x04c, 0x320, 0x310, 0x04d, 0x2e4,
+ 0x100, 0x001, 0x004, 0x000, 0x012, 0x044, 0x3f0, 0x002,
+ 0x044, 0x30e, 0x002, 0x140, 0x00f, 0x009, 0x008, 0x307,
+ 0x00e, 0x302, 0x2cd, 0x00a, 0x3e4, 0x0c4, 0x303, 0x364,
+ 0x30f, 0x002, 0x001, 0x004, 0x141, 0x303, 0x397, 0x2a1,
+ 0x300, 0x303, 0x04c, 0x2c7, 0x305, 0x397, 0x2aa, 0x300,
+ 0x303, 0x04c, 0x2c2, 0x305, 0x342, 0x310, 0x305, 0x397,
+ 0x2b2, 0x300, 0x30f, 0x327, 0x305, 0x04d, 0x362, 0x2d2,
+ 0x002, 0x001, 0x00f, 0x364, 0x2cb, 0x002, 0x009, 0x00a,
+ 0x307, 0x30f, 0x118, 0x390, 0x2b2, 0x327, 0x305, 0x04d,
+ 0x141, 0x30f, 0x000, 0x350, 0x044, 0x275, 0x00b, 0x0c0,
+ 0x2a9, 0x0c0, 0x303, 0x362, 0x303, 0x008, 0x00f, 0x017,
+ 0x397, 0x2b2, 0x300, 0x303, 0x04c, 0x2a7, 0x362, 0x303,
+ 0x004, 0x00f, 0x006, 0x2c0, 0x2a9, 0x000, 0x004, 0x2c2,
+ 0x2a9, 0x141, 0x303, 0x000, 0x3e8, 0x30a, 0x2a9, 0x20a,
+ 0x24a, 0x2a9, 0x044, 0x022, 0x009, 0x3c4, 0x3ef, 0x026,
+ 0x045, 0x307, 0x2b2, 0x300, 0x2b3, 0x247, 0x2a1, 0x362,
+ 0x2a1, 0x000, 0x006, 0x004, 0x141, 0x2a1, 0x30a, 0x2a1,
+ 0x300, 0x2aa, 0x247, 0x2a1, 0x307, 0x2b4, 0x300, 0x2b5,
+ 0x247, 0x2a2, 0x362, 0x2a2, 0x000, 0x006, 0x004, 0x141,
+ 0x2a2, 0x30a, 0x2a2, 0x300, 0x2ab, 0x247, 0x2a2, 0x307,
+ 0x2b6, 0x300, 0x2b7, 0x247, 0x2a3, 0x362, 0x2a3, 0x000,
+ 0x006, 0x004, 0x141, 0x2a3, 0x30a, 0x2a3, 0x300, 0x2ac,
+ 0x247, 0x2a3, 0x307, 0x2b8, 0x300, 0x2b9, 0x247, 0x2a4,
+ 0x362, 0x2a4, 0x000, 0x006, 0x004, 0x141, 0x2a4, 0x30a,
+ 0x2a4, 0x300, 0x2ad, 0x247, 0x2a4, 0x347, 0x2ae, 0x2a5,
+ 0x347, 0x2af, 0x2a6, 0x044, 0x3f0, 0x002, 0x045, 0x0c0,
+ 0x30f, 0x362, 0x30f, 0x010, 0x00f, 0x00c, 0x397, 0x2b2,
+ 0x300, 0x30f, 0x0a0, 0x04d, 0x141, 0x30f, 0x000, 0x3f3,
+ 0x045, 0x364, 0x2d4, 0x002, 0x001, 0x003, 0x045, 0x0c7,
+ 0x2ce, 0x347, 0x290, 0x2fb, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x1bf, 0x003, 0x309,
+ 0x013, 0x34b, 0x014, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x209, 0x34b, 0x336, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x249, 0x013, 0x34b, 0x336,
+ 0x014, 0x307, 0x2ae, 0x300, 0x2ba, 0x114, 0x247, 0x2a5,
+ 0x307, 0x2af, 0x300, 0x2bd, 0x104, 0x247, 0x2a6, 0x3c7,
+ 0x3ff, 0x301, 0x151, 0x2a6, 0x141, 0x2a5, 0x364, 0x2a6,
+ 0x100, 0x001, 0x004, 0x000, 0x01a, 0x364, 0x2a5, 0x100,
+ 0x001, 0x004, 0x000, 0x013, 0x044, 0x014, 0x003, 0x044,
+ 0x30e, 0x002, 0x307, 0x2cd, 0x0c0, 0x336, 0x302, 0x00e,
+ 0x343, 0x00f, 0x336, 0x00a, 0x3df, 0x3ce, 0x3ff, 0x2fe,
+ 0x141, 0x2a6, 0x00b, 0x004, 0x0c0, 0x2a6, 0x151, 0x2a5,
+ 0x00b, 0x004, 0x0c0, 0x2a5, 0x362, 0x2a6, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a6, 0x362, 0x2a5, 0x0ff, 0x006,
+ 0x005, 0x3c7, 0x0ff, 0x2a5, 0x044, 0x014, 0x003, 0x309,
+ 0x303, 0x242, 0x290, 0x0c0, 0x304, 0x362, 0x304, 0x003,
+ 0x00f, 0x02c, 0x340, 0x303, 0x290, 0x362, 0x290, 0x000,
+ 0x00f, 0x004, 0x0c0, 0x290, 0x362, 0x290, 0x0ff, 0x006,
+ 0x004, 0x000, 0x01b, 0x044, 0x034, 0x003, 0x044, 0x1cc,
+ 0x003, 0x044, 0x30e, 0x002, 0x0c0, 0x2ff, 0x044, 0x195,
+ 0x003, 0x140, 0x2ff, 0x001, 0x005, 0x347, 0x290, 0x305,
+ 0x141, 0x304, 0x000, 0x3d3, 0x362, 0x301, 0x000, 0x00f,
+ 0x01b, 0x347, 0x305, 0x301, 0x347, 0x2fb, 0x290, 0x3c6,
+ 0x0ff, 0x01a, 0x307, 0x2ae, 0x300, 0x2be, 0x114, 0x247,
+ 0x2a5, 0x307, 0x2af, 0x300, 0x2c1, 0x104, 0x247, 0x2a6,
+ 0x000, 0x36a, 0x307, 0x305, 0x300, 0x301, 0x24a, 0x290,
+ 0x044, 0x034, 0x003, 0x045, 0x0c0, 0x2e4, 0x0c3, 0x2ce,
+ 0x044, 0x1bf, 0x003, 0x0c0, 0x2ea, 0x044, 0x078, 0x009,
+ 0x347, 0x28f, 0x318, 0x347, 0x28e, 0x319, 0x0c0, 0x31b,
+ 0x0c0, 0x31c, 0x3c7, 0x038, 0x317, 0x364, 0x2d4, 0x020,
+ 0x001, 0x005, 0x347, 0x2c6, 0x317, 0x364, 0x2d4, 0x002,
+ 0x001, 0x005, 0x347, 0x290, 0x317, 0x0c0, 0x31e, 0x0cf,
+ 0x324, 0x364, 0x2d4, 0x004, 0x001, 0x008, 0x347, 0x28f,
+ 0x31e, 0x347, 0x28f, 0x324, 0x0c0, 0x31f, 0x0cf, 0x325,
+ 0x364, 0x2d4, 0x008, 0x001, 0x008, 0x347, 0x28e, 0x31f,
+ 0x347, 0x28e, 0x325, 0x364, 0x2d4, 0x010, 0x001, 0x00d,
+ 0x347, 0x291, 0x320, 0x347, 0x291, 0x326, 0x347, 0x291,
+ 0x31a, 0x000, 0x00a, 0x0cd, 0x291, 0x0cd, 0x31a, 0x0c0,
+ 0x320, 0x0cd, 0x326, 0x044, 0x1bf, 0x00a, 0x364, 0x24d,
+ 0x030, 0x009, 0x028, 0x309, 0x2c7, 0x34b, 0x2c8, 0x336,
+ 0x209, 0x34b, 0x336, 0x336, 0x34e, 0x2c8, 0x337, 0x32c,
+ 0x2c7, 0x282, 0x343, 0x337, 0x336, 0x247, 0x332, 0x347,
+ 0x336, 0x333, 0x307, 0x332, 0x347, 0x333, 0x336, 0x300,
+ 0x2c7, 0x341, 0x2c8, 0x336, 0x247, 0x334, 0x347, 0x336,
+ 0x335, 0x0c0, 0x398, 0x364, 0x2df, 0x002, 0x001, 0x006,
+ 0x0c1, 0x39a, 0x000, 0x004, 0x0c4, 0x39a, 0x347, 0x2cd,
+ 0x397, 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x39a, 0x006,
+ 0x076, 0x044, 0x16d, 0x00a, 0x0cf, 0x28f, 0x362, 0x28f,
+ 0x000, 0x007, 0x065, 0x362, 0x28f, 0x00f, 0x00f, 0x009,
+ 0x08f, 0x302, 0x28f, 0x247, 0x28e, 0x000, 0x004, 0x0c0,
+ 0x28e, 0x362, 0x28f, 0x006, 0x00f, 0x004, 0x0cf, 0x28e,
+ 0x0c1, 0x394, 0x362, 0x28e, 0x00a, 0x006, 0x004, 0x0d1,
+ 0x394, 0x0c0, 0x2a9, 0x307, 0x2a9, 0x302, 0x394, 0x006,
+ 0x03b, 0x347, 0x2a9, 0x313, 0x307, 0x2aa, 0x300, 0x2a9,
+ 0x247, 0x2a1, 0x307, 0x2ab, 0x300, 0x2a9, 0x247, 0x2a2,
+ 0x307, 0x2ac, 0x302, 0x2a9, 0x247, 0x2a3, 0x307, 0x2ad,
+ 0x302, 0x2a9, 0x247, 0x2a4, 0x044, 0x12e, 0x00a, 0x347,
+ 0x31d, 0x290, 0x307, 0x290, 0x302, 0x323, 0x007, 0x00a,
+ 0x044, 0x015, 0x00a, 0x340, 0x329, 0x290, 0x000, 0x3f4,
+ 0x307, 0x314, 0x300, 0x315, 0x247, 0x316, 0x141, 0x2a9,
+ 0x000, 0x3c3, 0x151, 0x28f, 0x000, 0x39a, 0x340, 0x317,
+ 0x398, 0x141, 0x399, 0x000, 0x388, 0x347, 0x398, 0x317,
+ 0x362, 0x39a, 0x004, 0x009, 0x007, 0x307, 0x398, 0x191,
+ 0x247, 0x317, 0x044, 0x177, 0x00a, 0x347, 0x317, 0x2e9,
+ 0x347, 0x31b, 0x2a9, 0x347, 0x2a9, 0x313, 0x347, 0x316,
+ 0x2ea, 0x0c0, 0x395, 0x347, 0x31e, 0x28f, 0x307, 0x28f,
+ 0x302, 0x324, 0x007, 0x093, 0x0c3, 0x2ce, 0x347, 0x325,
+ 0x28e, 0x347, 0x326, 0x291, 0x044, 0x3ad, 0x002, 0x0c3,
+ 0x303, 0x044, 0x3d4, 0x009, 0x347, 0x2a5, 0x2e2, 0x364,
+ 0x2d4, 0x008, 0x001, 0x007, 0x364, 0x2d4, 0x010, 0x009,
+ 0x05d, 0x0cf, 0x32e, 0x362, 0x32e, 0x000, 0x007, 0x02d,
+ 0x364, 0x2d4, 0x008, 0x009, 0x00c, 0x347, 0x32e, 0x28e,
+ 0x362, 0x28e, 0x00f, 0x006, 0x004, 0x0cf, 0x28e, 0x364,
+ 0x2d4, 0x010, 0x009, 0x005, 0x347, 0x32e, 0x291, 0x044,
+ 0x3ad, 0x002, 0x0c0, 0x303, 0x044, 0x3d4, 0x009, 0x307,
+ 0x2a5, 0x302, 0x2e2, 0x00e, 0x004, 0x000, 0x006, 0x151,
+ 0x32e, 0x000, 0x3d2, 0x364, 0x2d4, 0x008, 0x001, 0x006,
+ 0x141, 0x291, 0x000, 0x022, 0x307, 0x28e, 0x302, 0x325,
+ 0x006, 0x01c, 0x307, 0x2a5, 0x302, 0x2e2, 0x00e, 0x016,
+ 0x141, 0x28e, 0x307, 0x28e, 0x302, 0x325, 0x009, 0x004,
+ 0x000, 0x00c, 0x044, 0x034, 0x003, 0x0c0, 0x303, 0x044,
+ 0x3d4, 0x009, 0x000, 0x3e2, 0x362, 0x291, 0x00d, 0x006,
+ 0x004, 0x0cd, 0x291, 0x397, 0x39b, 0x300, 0x28f, 0x327,
+ 0x28e, 0x04d, 0x397, 0x3ab, 0x300, 0x28f, 0x327, 0x291,
+ 0x04d, 0x141, 0x28f, 0x000, 0x36b, 0x044, 0x16d, 0x00a,
+ 0x347, 0x2c7, 0x2c2, 0x347, 0x2c8, 0x2c3, 0x0c4, 0x397,
+ 0x0c0, 0x399, 0x307, 0x399, 0x302, 0x2e0, 0x006, 0x00f,
+ 0x349, 0x2c7, 0x2c7, 0x34b, 0x2c8, 0x2c8, 0x349, 0x397,
+ 0x397, 0x141, 0x399, 0x000, 0x3ef, 0x044, 0x1bf, 0x00a,
+ 0x347, 0x31e, 0x28f, 0x307, 0x28f, 0x302, 0x324, 0x007,
+ 0x055, 0x0c3, 0x2ce, 0x397, 0x39b, 0x300, 0x28f, 0x04c,
+ 0x2c7, 0x28e, 0x397, 0x3ab, 0x300, 0x28f, 0x04c, 0x2c7,
+ 0x291, 0x347, 0x317, 0x290, 0x044, 0x3ad, 0x002, 0x0c0,
+ 0x32e, 0x362, 0x32e, 0x010, 0x00f, 0x023, 0x0c1, 0x30a,
+ 0x0c0, 0x330, 0x362, 0x330, 0x004, 0x00f, 0x013, 0x364,
+ 0x026, 0x010, 0x009, 0x00a, 0x362, 0x32e, 0x002, 0x006,
+ 0x005, 0x044, 0x1e6, 0x00a, 0x141, 0x330, 0x000, 0x3ec,
+ 0x3c4, 0x3ef, 0x026, 0x141, 0x32e, 0x000, 0x3dc, 0x0c3,
+ 0x2ce, 0x044, 0x1bf, 0x003, 0x044, 0x30e, 0x002, 0x044,
+ 0x015, 0x00a, 0x307, 0x314, 0x300, 0x315, 0x247, 0x316,
+ 0x141, 0x28f, 0x000, 0x3a9, 0x044, 0x3b3, 0x002, 0x362,
+ 0x316, 0x006, 0x00f, 0x004, 0x0c0, 0x396, 0x044, 0x177,
+ 0x00a, 0x347, 0x316, 0x2ec, 0x347, 0x2c2, 0x2c7, 0x347,
+ 0x2c3, 0x2c8, 0x045, 0x3a7, 0x060, 0x044, 0x192, 0x000,
+ 0x207, 0x384, 0x008, 0x247, 0x304, 0x09c, 0x189, 0x395,
+ 0x106, 0x0a0, 0x044, 0x18a, 0x002, 0x3c4, 0x300, 0x01a,
+ 0x140, 0x303, 0x009, 0x010, 0x140, 0x304, 0x009, 0x007,
+ 0x3c5, 0x004, 0x01a, 0x000, 0x013, 0x3c5, 0x020, 0x01a,
+ 0x000, 0x00e, 0x140, 0x304, 0x009, 0x007, 0x3c5, 0x080,
+ 0x01a, 0x000, 0x005, 0x3c5, 0x001, 0x01a, 0x044, 0x1ba,
+ 0x003, 0x3d7, 0x200, 0x013, 0x0c0, 0x014, 0x397, 0x2a5,
+ 0x044, 0x1e9, 0x003, 0x045, 0x0c0, 0x303, 0x0c0, 0x330,
+ 0x0c0, 0x331, 0x0c0, 0x32e, 0x0c0, 0x32f, 0x044, 0x3ad,
+ 0x002, 0x0c0, 0x303, 0x307, 0x303, 0x302, 0x2d1, 0x006,
+ 0x0bb, 0x364, 0x303, 0x001, 0x001, 0x016, 0x307, 0x2ae,
+ 0x302, 0x313, 0x114, 0x302, 0x315, 0x247, 0x2a5, 0x307,
+ 0x2af, 0x302, 0x313, 0x104, 0x300, 0x315, 0x247, 0x2a6,
+ 0x000, 0x014, 0x307, 0x2ae, 0x300, 0x2a9, 0x114, 0x302,
+ 0x314, 0x247, 0x2a5, 0x307, 0x2af, 0x300, 0x2a9, 0x104,
+ 0x300, 0x314, 0x247, 0x2a6, 0x362, 0x2a5, 0x000, 0x00f,
+ 0x003, 0x045, 0x362, 0x2a6, 0x0ff, 0x006, 0x003, 0x045,
+ 0x044, 0x3f0, 0x002, 0x362, 0x303, 0x002, 0x00f, 0x02c,
+ 0x083, 0x18a, 0x385, 0x101, 0x0a0, 0x044, 0x18a, 0x002,
+ 0x044, 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x334,
+ 0x347, 0x335, 0x336, 0x302, 0x046, 0x343, 0x047, 0x336,
+ 0x00a, 0x00e, 0x307, 0x046, 0x347, 0x047, 0x336, 0x302,
+ 0x332, 0x343, 0x333, 0x336, 0x002, 0x006, 0x347, 0x323,
+ 0x290, 0x045, 0x307, 0x303, 0x384, 0x001, 0x104, 0x18a,
+ 0x385, 0x101, 0x207, 0x0a0, 0x044, 0x18a, 0x002, 0x044,
+ 0x1cc, 0x003, 0x044, 0x30e, 0x002, 0x307, 0x046, 0x347,
+ 0x047, 0x336, 0x302, 0x332, 0x343, 0x333, 0x336, 0x002,
+ 0x006, 0x347, 0x323, 0x290, 0x045, 0x364, 0x303, 0x001,
+ 0x001, 0x00a, 0x340, 0x00e, 0x330, 0x341, 0x00f, 0x331,
+ 0x000, 0x008, 0x340, 0x00e, 0x32e, 0x341, 0x00f, 0x32f,
+ 0x307, 0x2fd, 0x347, 0x2fe, 0x336, 0x327, 0x32e, 0x347,
+ 0x32f, 0x337, 0x320, 0x330, 0x341, 0x331, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x003, 0x045, 0x141, 0x303,
+ 0x000, 0x343, 0x347, 0x290, 0x317, 0x347, 0x28f, 0x318,
+ 0x347, 0x28e, 0x319, 0x347, 0x291, 0x31a, 0x347, 0x2a9,
+ 0x31b, 0x347, 0x313, 0x31c, 0x347, 0x293, 0x396, 0x307,
+ 0x32e, 0x347, 0x32f, 0x336, 0x300, 0x330, 0x341, 0x331,
+ 0x336, 0x247, 0x2fd, 0x347, 0x336, 0x2fe, 0x307, 0x2fd,
+ 0x347, 0x2fe, 0x336, 0x327, 0x397, 0x0c0, 0x337, 0x282,
+ 0x343, 0x337, 0x336, 0x002, 0x019, 0x30a, 0x397, 0x327,
+ 0x330, 0x282, 0x00a, 0x004, 0x141, 0x315, 0x30a, 0x397,
+ 0x327, 0x32e, 0x282, 0x00a, 0x004, 0x141, 0x314, 0x3ce,
+ 0x3ff, 0x2fe, 0x000, 0x2ea, 0x045, 0x364, 0x2d4, 0x002,
+ 0x001, 0x00c, 0x347, 0x317, 0x31d, 0x347, 0x317, 0x323,
+ 0x0c1, 0x329, 0x000, 0x031, 0x362, 0x316, 0x000, 0x00e,
+ 0x007, 0x364, 0x2d4, 0x020, 0x001, 0x01f, 0x307, 0x317,
+ 0x114, 0x247, 0x31d, 0x307, 0x317, 0x104, 0x247, 0x323,
+ 0x0c4, 0x329, 0x362, 0x31d, 0x000, 0x00f, 0x004, 0x0c0,
+ 0x31d, 0x362, 0x323, 0x0b8, 0x006, 0x00f, 0x3c7, 0x0b8,
+ 0x323, 0x000, 0x00a, 0x3c7, 0x038, 0x31d, 0x3c7, 0x0b8,
+ 0x323, 0x0c4, 0x329, 0x045, 0x0c0, 0x314, 0x0c0, 0x315,
+ 0x0c0, 0x316, 0x3ce, 0x3ff, 0x2fe, 0x045, 0x347, 0x319,
+ 0x28e, 0x347, 0x318, 0x28f, 0x347, 0x317, 0x290, 0x347,
+ 0x31a, 0x291, 0x347, 0x396, 0x293, 0x347, 0x31b, 0x2bd,
+ 0x347, 0x2bd, 0x2bc, 0x347, 0x2bc, 0x2bb, 0x347, 0x2bb,
+ 0x2ba, 0x347, 0x2ba, 0x2b5, 0x347, 0x2b5, 0x2b4, 0x347,
+ 0x2b4, 0x2b3, 0x347, 0x2b3, 0x2b2, 0x080, 0x302, 0x31c,
+ 0x247, 0x2c1, 0x347, 0x2c1, 0x2c0, 0x347, 0x2c0, 0x2bf,
+ 0x347, 0x2bf, 0x2be, 0x347, 0x2be, 0x2b9, 0x347, 0x2b9,
+ 0x2b8, 0x347, 0x2b8, 0x2b7, 0x347, 0x2b7, 0x2b6, 0x044,
+ 0x3ad, 0x002, 0x044, 0x022, 0x009, 0x045, 0x309, 0x2c7,
+ 0x34b, 0x2c8, 0x336, 0x209, 0x34b, 0x336, 0x336, 0x209,
+ 0x34b, 0x336, 0x336, 0x300, 0x2c7, 0x341, 0x2c8, 0x336,
+ 0x247, 0x332, 0x347, 0x336, 0x333, 0x309, 0x2c7, 0x34b,
+ 0x2c8, 0x336, 0x300, 0x332, 0x341, 0x333, 0x336, 0x247,
+ 0x334, 0x347, 0x336, 0x335, 0x045, 0x3a7, 0x022, 0x090,
+ 0x364, 0x2d4, 0x001, 0x001, 0x00b, 0x364, 0x2cf, 0x001,
+ 0x001, 0x005, 0x044, 0x1a9, 0x000, 0x045, 0x044, 0x1b7,
+ 0x000, 0x362, 0x2f6, 0x002, 0x00f, 0x02b, 0x0c0, 0x304,
+ 0x044, 0x25b, 0x003, 0x347, 0x2a5, 0x2e2, 0x001, 0x007,
+ 0x347, 0x2a6, 0x2e1, 0x009, 0x005, 0x141, 0x2ee, 0x045,
+ 0x364, 0x2f6, 0x001, 0x001, 0x037, 0x0c1, 0x304, 0x044,
+ 0x25b, 0x003, 0x362, 0x2a5, 0x0ff, 0x001, 0x007, 0x362,
+ 0x2a6, 0x0ff, 0x009, 0x028, 0x141, 0x2ee, 0x045, 0x307,
+ 0x2bb, 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2bd, 0x300,
+ 0x2af, 0x247, 0x2a6, 0x347, 0x2a5, 0x2e2, 0x347, 0x2a6,
+ 0x2e1, 0x364, 0x2f6, 0x001, 0x001, 0x00e, 0x307, 0x2be,
+ 0x300, 0x2ae, 0x247, 0x2a5, 0x307, 0x2c0, 0x300, 0x2af,
+ 0x247, 0x2a6, 0x044, 0x1bf, 0x003, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x00f, 0x364, 0x2cf, 0x001, 0x001, 0x007, 0x3d7,
+ 0x200, 0x013, 0x0c0, 0x014, 0x3c5, 0x100, 0x00d, 0x0c6,
+ 0x2ce, 0x34e, 0x306, 0x306, 0x0c2, 0x308, 0x362, 0x308,
+ 0x00d, 0x00e, 0x019, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x04c, 0x2c7, 0x303, 0x34e, 0x306, 0x306, 0x364, 0x306,
+ 0x001, 0x009, 0x005, 0x044, 0x2fa, 0x00a, 0x141, 0x308,
+ 0x000, 0x3e6, 0x140, 0x30a, 0x001, 0x05c, 0x0c0, 0x303,
+ 0x0c0, 0x304, 0x362, 0x304, 0x00b, 0x00e, 0x048, 0x397,
+ 0x294, 0x300, 0x304, 0x04c, 0x2c7, 0x305, 0x362, 0x305,
+ 0x000, 0x00f, 0x007, 0x308, 0x305, 0x101, 0x247, 0x305,
+ 0x307, 0x305, 0x302, 0x303, 0x00f, 0x005, 0x347, 0x305,
+ 0x303, 0x362, 0x305, 0x00f, 0x009, 0x025, 0x141, 0x293,
+ 0x0c0, 0x30a, 0x0c4, 0x2f7, 0x0c0, 0x304, 0x362, 0x304,
+ 0x00b, 0x00e, 0x01c, 0x397, 0x294, 0x300, 0x304, 0x04c,
+ 0x2c7, 0x305, 0x397, 0x294, 0x300, 0x304, 0x329, 0x305,
+ 0x320, 0x305, 0x2aa, 0x2aa, 0x04d, 0x141, 0x304, 0x000,
+ 0x3e7, 0x141, 0x304, 0x000, 0x3b7, 0x362, 0x303, 0x00a,
+ 0x00f, 0x008, 0x151, 0x293, 0x0c0, 0x30a, 0x0c4, 0x2f7,
+ 0x044, 0x06e, 0x003, 0x044, 0x096, 0x003, 0x045, 0x307,
+ 0x2a5, 0x347, 0x2e2, 0x2a5, 0x247, 0x2e2, 0x307, 0x2a6,
+ 0x347, 0x2e1, 0x2a6, 0x247, 0x2e1, 0x044, 0x014, 0x003,
+ 0x045, 0x347, 0x303, 0x304, 0x347, 0x304, 0x30e, 0x3ce,
+ 0x3ff, 0x2fe, 0x152, 0x303, 0x0c0, 0x305, 0x362, 0x305,
+ 0x003, 0x00f, 0x0b9, 0x141, 0x303, 0x362, 0x303, 0x3f1,
+ 0x00f, 0x004, 0x000, 0x0ac, 0x362, 0x303, 0x00f, 0x006,
+ 0x007, 0x347, 0x30c, 0x30d, 0x000, 0x0a6, 0x307, 0x308,
+ 0x112, 0x390, 0x294, 0x327, 0x303, 0x04d, 0x044, 0x096,
+ 0x003, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002, 0x00f,
+ 0x00d, 0x08c, 0x189, 0x395, 0x104, 0x328, 0x308, 0x121,
+ 0x3a4, 0x0ff, 0x000, 0x007, 0x088, 0x189, 0x395, 0x101,
+ 0x0a0, 0x044, 0x18a, 0x002, 0x044, 0x30e, 0x002, 0x347,
+ 0x00e, 0x32e, 0x347, 0x00f, 0x32f, 0x364, 0x2f6, 0x001,
+ 0x001, 0x024, 0x044, 0x2e8, 0x00a, 0x362, 0x2f6, 0x002,
+ 0x00f, 0x010, 0x08e, 0x189, 0x395, 0x104, 0x328, 0x308,
+ 0x121, 0x3a4, 0x0ff, 0x044, 0x18a, 0x002, 0x000, 0x005,
+ 0x3c6, 0x0ff, 0x01a, 0x044, 0x30e, 0x002, 0x340, 0x00e,
+ 0x32e, 0x341, 0x00f, 0x32f, 0x140, 0x305, 0x001, 0x007,
+ 0x362, 0x303, 0x3f1, 0x009, 0x005, 0x347, 0x32f, 0x30b,
+ 0x362, 0x305, 0x001, 0x009, 0x005, 0x347, 0x32f, 0x30c,
+ 0x362, 0x305, 0x002, 0x009, 0x005, 0x347, 0x32f, 0x30d,
+ 0x362, 0x32f, 0x000, 0x00f, 0x00d, 0x348, 0x32e, 0x32e,
+ 0x348, 0x32f, 0x32f, 0x141, 0x32e, 0x3c1, 0x000, 0x32f,
+ 0x307, 0x32e, 0x347, 0x32f, 0x336, 0x302, 0x2fd, 0x343,
+ 0x2fe, 0x336, 0x002, 0x00b, 0x347, 0x303, 0x304, 0x347,
+ 0x32e, 0x2fd, 0x347, 0x32f, 0x2fe, 0x140, 0x305, 0x009,
+ 0x007, 0x141, 0x2fd, 0x3c1, 0x000, 0x2fe, 0x141, 0x305,
+ 0x000, 0x346, 0x307, 0x30b, 0x306, 0x30d, 0x003, 0x01d,
+ 0x307, 0x30e, 0x111, 0x247, 0x304, 0x362, 0x30c, 0x000,
+ 0x00f, 0x004, 0x142, 0x304, 0x362, 0x304, 0x3f1, 0x00f,
+ 0x005, 0x3c7, 0x3f1, 0x304, 0x362, 0x304, 0x00f, 0x006,
+ 0x004, 0x0cf, 0x304, 0x307, 0x308, 0x112, 0x390, 0x294,
+ 0x327, 0x304, 0x04d, 0x045, 0x044, 0x290, 0x001, 0x3c4,
+ 0x3fe, 0x200, 0x020, 0x2e0, 0x002, 0x045, 0x1f6, 0x185,
+ 0x1f5, 0x129, 0x1f4, 0x0d0, 0x1b3, 0x0fc, 0x122, 0x0ff,
+ 0x1f6, 0x189, 0x1f5, 0x11b, 0x1f5, 0x0aa, 0x194, 0x0ce,
+ 0x123, 0x0f3, 0x0d2, 0x0ff, 0x364, 0x233, 0x020, 0x009,
+ 0x051, 0x307, 0x272, 0x196, 0x384, 0x00f, 0x0c9, 0x286,
+ 0x242, 0x286, 0x397, 0x00a, 0x189, 0x395, 0x3f7, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x380, 0x00a, 0x051, 0x2c7,
+ 0x284, 0x101, 0x051, 0x2c7, 0x288, 0x3a4, 0x07f, 0x040,
+ 0x307, 0x272, 0x384, 0x07f, 0x222, 0x042, 0x00b, 0x005,
+ 0x101, 0x000, 0x3ed, 0x347, 0x284, 0x285, 0x3c4, 0x00f,
+ 0x284, 0x1d3, 0x285, 0x1d6, 0x288, 0x342, 0x288, 0x286,
+ 0x003, 0x008, 0x156, 0x285, 0x151, 0x286, 0x00b, 0x3fc,
+ 0x362, 0x285, 0x003, 0x003, 0x004, 0x0c3, 0x285, 0x307,
+ 0x285, 0x0a1, 0x184, 0x305, 0x284, 0x044, 0x1bd, 0x000,
+ 0x045, 0x3db, 0x071, 0x0b9, 0x0ff, 0x089, 0x0e2, 0x12c,
+ 0x176, 0x1bf, 0x00d, 0x007, 0x006, 0x003, 0x002, 0x005,
+ 0x0b3, 0x156, 0x1fa, 0x27f, 0x080, 0x09e, 0x14b, 0x1f0,
+ 0x27f, 0x080, 0x08b, 0x140, 0x1e9, 0x27f, 0x081, 0x138,
+ 0x1e3, 0x27f, 0x00f, 0x0b2, 0x14f, 0x1f0, 0x27f, 0x080,
+ 0x0a3, 0x145, 0x1e7, 0x27f, 0x080, 0x095, 0x13d, 0x1e0,
+ 0x27f, 0x08b, 0x136, 0x1db, 0x27f, 0x0c5, 0x28b, 0x347,
+ 0x271, 0x281, 0x3c4, 0x07f, 0x281, 0x0c5, 0x282, 0x044,
+ 0x2ea, 0x001, 0x009, 0x004, 0x0c2, 0x282, 0x044, 0x154,
+ 0x00b, 0x347, 0x283, 0x289, 0x307, 0x241, 0x364, 0x233,
+ 0x002, 0x001, 0x004, 0x307, 0x225, 0x384, 0x0ff, 0x242,
+ 0x283, 0x00b, 0x056, 0x0c3, 0x28b, 0x302, 0x289, 0x397,
+ 0x00b, 0x189, 0x395, 0x062, 0x044, 0x2ea, 0x001, 0x009,
+ 0x004, 0x380, 0x004, 0x051, 0x2c7, 0x28c, 0x2a9, 0x121,
+ 0x3a4, 0x07f, 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282,
+ 0x042, 0x003, 0x007, 0x001, 0x005, 0x101, 0x000, 0x3ed,
+ 0x1d5, 0x28c, 0x347, 0x241, 0x281, 0x364, 0x233, 0x002,
+ 0x001, 0x005, 0x347, 0x225, 0x281, 0x3c4, 0x0ff, 0x281,
+ 0x342, 0x289, 0x281, 0x347, 0x28c, 0x282, 0x366, 0x282,
+ 0x00f, 0x009, 0x006, 0x1c0, 0x281, 0x0c1, 0x282, 0x044,
+ 0x154, 0x00b, 0x001, 0x00d, 0x141, 0x28b, 0x362, 0x28b,
+ 0x03f, 0x001, 0x00d, 0x151, 0x283, 0x000, 0x3f5, 0x362,
+ 0x28b, 0x00a, 0x003, 0x004, 0x0ca, 0x28b, 0x397, 0x00b,
+ 0x189, 0x395, 0x070, 0x044, 0x2ea, 0x001, 0x009, 0x004,
+ 0x380, 0x013, 0x327, 0x24c, 0x131, 0x003, 0x005, 0x105,
+ 0x000, 0x3fc, 0x051, 0x2c7, 0x28a, 0x3a4, 0x07f, 0x121,
+ 0x040, 0x307, 0x271, 0x384, 0x07f, 0x282, 0x042, 0x003,
+ 0x005, 0x101, 0x000, 0x3f0, 0x1d6, 0x28a, 0x387, 0x00b,
+ 0x189, 0x395, 0x06b, 0x300, 0x28a, 0x050, 0x247, 0x28a,
+ 0x30a, 0x28b, 0x3b7, 0x201, 0x184, 0x305, 0x28a, 0x044,
+ 0x1bd, 0x000, 0x045, 0x0c1, 0x283, 0x342, 0x282, 0x281,
+ 0x003, 0x006, 0x141, 0x283, 0x000, 0x3f9, 0x045, 0x0c0,
+ 0x244, 0x044, 0x172, 0x000, 0x043, 0x047, 0x0a1, 0x1aa,
+ 0x2c5, 0x233, 0x0c0, 0x244, 0x044, 0x172, 0x000, 0x364,
+ 0x233, 0x001, 0x029, 0x0f4, 0x000, 0x047, 0x000, 0x3e9,
+ 0x364, 0x025, 0x004, 0x009, 0x0d4, 0x307, 0x2cf, 0x18a,
+ 0x003, 0x006, 0x002, 0x08c, 0x000, 0x0bd, 0x364, 0x200,
+ 0x100, 0x009, 0x0c0, 0x0c0, 0x2e6, 0x0c0, 0x2e7, 0x08d,
+ 0x18b, 0x385, 0x090, 0x040, 0x083, 0x040, 0x387, 0x080,
+ 0x044, 0x3e3, 0x001, 0x044, 0x346, 0x002, 0x3c5, 0x001,
+ 0x2cf, 0x0c0, 0x2d8, 0x0c0, 0x2f1, 0x3c4, 0x3ef, 0x027,
+ 0x387, 0x085, 0x187, 0x385, 0x080, 0x247, 0x041, 0x3c5,
+ 0x020, 0x00c, 0x0c0, 0x2f6, 0x044, 0x3b3, 0x002, 0x140,
+ 0x2d3, 0x009, 0x026, 0x364, 0x292, 0x008, 0x009, 0x021,
+ 0x044, 0x16d, 0x009, 0x0d0, 0x312, 0x0c0, 0x30a, 0x0c0,
+ 0x2ee, 0x0d0, 0x2f7, 0x044, 0x253, 0x00b, 0x342, 0x293,
+ 0x312, 0x001, 0x00e, 0x0c1, 0x30a, 0x347, 0x293, 0x312,
+ 0x0c8, 0x2f7, 0x044, 0x253, 0x00b, 0x000, 0x3f1, 0x3c4,
+ 0x1ff, 0x2cf, 0x0c0, 0x30a, 0x307, 0x290, 0x183, 0x305,
+ 0x28f, 0x183, 0x305, 0x28e, 0x247, 0x2f2, 0x0cf, 0x2ce,
+ 0x044, 0x113, 0x003, 0x044, 0x33f, 0x002, 0x055, 0x364,
+ 0x2cf, 0x020, 0x009, 0x005, 0x3c4, 0x3bf, 0x200, 0x3c4,
+ 0x3ee, 0x2cf, 0x364, 0x2df, 0x001, 0x009, 0x04a, 0x364,
+ 0x33c, 0x002, 0x009, 0x045, 0x141, 0x2cf, 0x044, 0x346,
+ 0x002, 0x3c5, 0x002, 0x2cf, 0x347, 0x2d5, 0x2f7, 0x0c4,
+ 0x303, 0x044, 0x08a, 0x009, 0x044, 0x333, 0x008, 0x151,
+ 0x2f7, 0x00b, 0x3f6, 0x347, 0x2d5, 0x2f7, 0x044, 0x253,
+ 0x00b, 0x055, 0x3c4, 0x3fc, 0x2cf, 0x364, 0x2cf, 0x040,
+ 0x009, 0x019, 0x3c4, 0x39d, 0x2cf, 0x0ce, 0x2ce, 0x364,
+ 0x2cf, 0x001, 0x009, 0x007, 0x0cf, 0x2ce, 0x3c4, 0x3fe,
+ 0x2cf, 0x364, 0x2cf, 0x010, 0x009, 0x005, 0x3c4, 0x3bf,
+ 0x200, 0x3c4, 0x1ff, 0x2cf, 0x044, 0x33f, 0x002, 0x020,
+ 0x0d3, 0x000, 0x347, 0x2cb, 0x306, 0x044, 0x1e6, 0x00a,
+ 0x044, 0x333, 0x008, 0x151, 0x2f7, 0x009, 0x3f5, 0x045,
+ 0x397, 0x201, 0x0a0, 0x044, 0x18a, 0x002, 0x020, 0x1bf,
+ 0x003, 0x397, 0x2ba, 0x04c, 0x118, 0x04d, 0x109, 0x272,
+ 0x2c1, 0x00b, 0x3fa, 0x045, 0x397, 0x2b2, 0x04c, 0x2c7,
+ 0x303, 0x101, 0x04c, 0x322, 0x303, 0x00b, 0x009, 0x04c,
+ 0x111, 0x04d, 0x101, 0x327, 0x303, 0x04d, 0x101, 0x272,
+ 0x2c1, 0x00b, 0x3ed, 0x045, 0x2f4, 0x200, 0x001, 0x008,
+ 0x364, 0x2cf, 0x007, 0x029, 0x1dc, 0x007, 0x287, 0x385,
+ 0x300, 0x208, 0x001, 0x04b, 0x364, 0x33b, 0x010, 0x029,
+ 0x1dc, 0x007, 0x3c5, 0x002, 0x00c, 0x287, 0x384, 0x0ff,
+ 0x305, 0x21d, 0x1b7, 0x2e4, 0x001, 0x001, 0x01d, 0x040,
+ 0x384, 0x007, 0x117, 0x001, 0x00c, 0x3c5, 0x040, 0x02d,
+ 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x3bf, 0x02d, 0x042,
+ 0x247, 0x029, 0x364, 0x018, 0x001, 0x009, 0x005, 0x3c5,
+ 0x010, 0x021, 0x2e4, 0x002, 0x001, 0x017, 0x264, 0x080,
+ 0x001, 0x005, 0x0a4, 0x1a7, 0x285, 0x040, 0x3c5, 0x080,
+ 0x02d, 0x081, 0x044, 0x189, 0x000, 0x3c4, 0x37f, 0x02d,
+ 0x042, 0x247, 0x02a, 0x000, 0x014, 0x38e, 0x3ff, 0x1b8,
+ 0x00a, 0x00a, 0x244, 0x029, 0x3c4, 0x3ef, 0x021, 0x3c4,
+ 0x3fd, 0x00c, 0x120, 0x001, 0x004, 0x244, 0x02a, 0x020,
+ 0x181, 0x006, 0x364, 0x23f, 0x003, 0x009, 0x007, 0x044,
+ 0x340, 0x001, 0x247, 0x249, 0x054, 0x347, 0x249, 0x248,
+ 0x2c7, 0x23f, 0x083, 0x18a, 0x385, 0x018, 0x364, 0x23f,
+ 0x001, 0x001, 0x006, 0x245, 0x057, 0x000, 0x005, 0x208,
+ 0x244, 0x057, 0x087, 0x18c, 0x385, 0x0e0, 0x364, 0x23f,
+ 0x002, 0x001, 0x009, 0x245, 0x057, 0x044, 0x27d, 0x001,
+ 0x000, 0x005, 0x208, 0x244, 0x057, 0x020, 0x181, 0x006,
+ 0x04e, 0x0ce, 0x045, 0x03c, 0x0f5, 0x04e, 0x0fd, 0x07e,
+ 0x00a, 0x0f9
+
+}; /* end fm10000_serdes_swap_code_prd2 */
diff --git a/drivers/net/fm10k/switch/fm10k_spico_code.h b/drivers/net/fm10k/switch/fm10k_spico_code.h
new file mode 100644
index 0000000..c172638
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_spico_code.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SPICO_CODE_H_
+#define _FM10K_SPICO_CODE_H_
+#include <stdint.h>
+
+extern const uint32_t fm10000_sbus_master_code_version_build_id_prd;
+extern const uint32_t fm10000_sbus_master_code_size_prd;
+extern const uint16_t fm10000_sbus_master_code_prd[];
+
+extern const uint32_t fm10000_serdes_spico_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_spico_code_size_prd2;
+extern const uint16_t fm10000_serdes_spico_code_prd2[];
+
+extern const uint32_t fm10000_serdes_swap_code_version_build_id_prd2;
+extern const uint32_t fm10000_serdes_swap_code_size_prd2;
+extern const uint16_t fm10000_serdes_swap_code_prd2[];
+
+#endif /* _FM10K_SPICO_CODE_H_ */
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 3/5] net/fm10k: add ffu and statistics and config file functions
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add ffu to support offload flow into HW.
It supports forward, mirror, push VLAN, pop VLAN.
It also supports flowset for a group flow definition.
The config file can configure debug log, port speed,
epl port mapping dpdk port, flowset. All these configuration
will be used by switch management.
Statistics includes epl port, ffu rule, dpdk port, and error.
All these statistics data are read from HW.
Modify switch header file to support getting logical port
and glort and device info.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 3 +
drivers/net/fm10k/switch/fm10k_config.c | 857 +++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_config.h | 178 +++++
drivers/net/fm10k/switch/fm10k_ffu.c | 1250 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_ffu.h | 31 +
drivers/net/fm10k/switch/fm10k_stats.c | 1105 +++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_stats.h | 257 +++++++
drivers/net/fm10k/switch/fm10k_switch.h | 158 +++-
8 files changed, 3830 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/fm10k/switch/fm10k_config.c
create mode 100644 drivers/net/fm10k/switch/fm10k_config.h
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.c
create mode 100644 drivers/net/fm10k/switch/fm10k_ffu.h
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.c
create mode 100644 drivers/net/fm10k/switch/fm10k_stats.h
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 6f5ae60..68a72d5 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -85,6 +85,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ext_port.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_serdes.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_sm.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/switch/fm10k_config.c b/drivers/net/fm10k/switch/fm10k_config.c
new file mode 100644
index 0000000..6fb9bab
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.c
@@ -0,0 +1,857 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <unistd.h>
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+
+
+#define FM10K_CONFIG_WORD_MAX 10
+#define FM10K_CONFIG_WORD_LEN 20
+#define FM10K_CONFIG_STR_MAX 100
+
+/* used for store the key strings */
+struct fm10k_cfg_key_item {
+ const char *key_str[FM10K_CONFIG_WORD_MAX];
+ uint8_t type;
+};
+
+/* configuration file path */
+static const char *fm10k_config_dpdk_conf_file = "/etc/dpdk_fm10k.conf";
+/* configuration structure */
+static struct fm10k_dpdk_cfg fm10k_config_dpdk_cfg;
+
+/* configuration key strings */
+static struct fm10k_cfg_key_item fm10k_config_key_items[] = {
+ /* enable debug print */
+ { {"debug", "print", "enable"},
+ FM10K_CONFIG_DEBUG_ENABLE},
+ { {"debug", "print", "config"},
+ FM10K_CONFIG_DEBUG_CONFIG},
+ { {"debug", "print", "ffu", "init"},
+ FM10K_CONFIG_DEBUG_FFU_INIT},
+ { {"debug", "print", "ffu", "register"},
+ FM10K_CONFIG_DEBUG_FFU_REG},
+ { {"debug", "print", "ffu", "rule"},
+ FM10K_CONFIG_DEBUG_FFU_RULE},
+ { {"debug", "print", "stats", "port"},
+ FM10K_CONFIG_DEBUG_STATS_PORT},
+ { {"debug", "print", "stats", "queue"},
+ FM10K_CONFIG_DEBUG_STATS_QUEUE},
+ { {"debug", "print", "stats", "rule"},
+ FM10K_CONFIG_DEBUG_STATS_FFU},
+ { {"debug", "print", "stats", "detail"},
+ FM10K_CONFIG_DEBUG_STATS_MORE},
+ { {"debug", "print", "stats", "interval"},
+ FM10K_CONFIG_DEBUG_STATS_INTERVAL},
+ /* general configuration */
+ { {"dpdk", "bind", "pf", "number"},
+ FM10K_CONFIG_BIND_PF_NUMBER},
+ { {"extern", "port", "speed"},
+ FM10K_CONFIG_EXT_PORT_SPEED},
+ /* internal redirect configuration */
+ { {"dpdk", "port", "*", "map", "pf"},
+ FM10K_CONFIG_DPDK_PORT_MAP_PF},
+ { {"extern", "port", "*", "map", "pf"},
+ FM10K_CONFIG_EXT_PORT_MAP_PF},
+ /* external redirect configuration */
+ { {"flowset", "start"},
+ FM10K_CONFIG_FLOWSET_START},
+ { {"flowset", "stop"},
+ FM10K_CONFIG_FLOWSET_STOP},
+ { {"flowset", "enable"},
+ FM10K_CONFIG_FLOWSET_ENABLE},
+ { {"flow", "*", "condition", "source", "extern", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT},
+ { {"flow", "*", "condition", "source", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT},
+ { {"flow", "*", "condition", "vlan"},
+ FM10K_CONFIG_FLOW_COND_VLAN},
+ { {"flow", "*", "action", "forward", "extern", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT},
+ { {"flow", "*", "action", "forward", "dpdk", "port"},
+ FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT},
+ { {"flow", "*", "action", "forward", "vlan"},
+ FM10K_CONFIG_FLOW_ACT_FW_VALN},
+};
+
+static char default_flowset[10] = "default";
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_2pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PF.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF. ",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+static struct fm10k_cfg_config_item fm10k_silc_nic_2ext_4pep[] = {
+ { FM10K_CONFIG_BIND_PF_NUMBER, FM10K_CONFIG_VALUE_INT, 0,
+ "# Tell how many PF ports are bound in DPDK.\n"
+ "# Driver will check the number when initializes.",
+ .val.int64 = 4
+ },
+
+ { FM10K_CONFIG_EXT_PORT_SPEED, FM10K_CONFIG_VALUE_INT, 0,
+ "# Set external port speed, 40 means 40G, 100 means 100G.",
+ .val.int64 = 100
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "# Map 1 or 2 PF ports to one DPDK port.\n"
+ "# If mapped PF number is 2, traffic will be\n"
+ "# load balance between the 2 PFs.\n"
+ "# And the DPDK port queue number will be configured\n"
+ "# more than 2(each PF need at least 1 DPDK port queue).",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 0,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_DPDK_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "# Map 1 or 2 PF to one external port. If mapped PF number is 2,\n"
+ "# traffic will be load balance between the 2 PF.",
+ .val.int64 = 0
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 1,
+ "",
+ .val.int64 = 2
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 1
+ },
+ { FM10K_CONFIG_EXT_PORT_MAP_PF, FM10K_CONFIG_VALUE_INT, 2,
+ "",
+ .val.int64 = 3
+ },
+ { FM10K_CONFIG_FLOWSET_START, FM10K_CONFIG_VALUE_STR, 0,
+ "# Define flow rule",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_STOP, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_FLOWSET_ENABLE, FM10K_CONFIG_VALUE_STR, 0,
+ "",
+ .val.str = default_flowset
+ },
+ { FM10K_CONFIG_TYPE_NULL, 0, 0,
+ "",
+ .val.int64 = 0
+ },
+};
+
+
+static int
+fm10k_config_conf_line_parser(char *buff, struct fm10k_cfg_config_item *item)
+{
+ char *p;
+ char *endptr;
+ const char *cfg_delims = { " " };
+ const char *key_delims = { "." };
+ struct fm10k_cfg_key_item *key;
+ char cfgs[3][FM10K_CONFIG_STR_MAX];
+ char cmp_keys[FM10K_CONFIG_WORD_MAX][FM10K_CONFIG_WORD_LEN];
+ long key_param = 0;
+ uint16_t i, j;
+ uint8_t val_type = FM10K_CONFIG_VALUE_NULL;
+
+ for (i = 0; i < 3; i++) {
+ if (i == 0)
+ p = strtok(buff, cfg_delims);
+ else
+ p = strtok(NULL, cfg_delims);
+ if (p == NULL)
+ return -1;
+ strncpy(cfgs[i], p, FM10K_CONFIG_STR_MAX);
+ }
+
+ p = strtok(NULL, cfg_delims);
+ if (p)
+ return -1;
+
+ memset(cmp_keys, 0, sizeof(cmp_keys));
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (i == 0)
+ p = strtok(cfgs[0], key_delims);
+ else
+ p = strtok(NULL, key_delims);
+ if (p == NULL)
+ break;
+ strncpy(cmp_keys[i], p, FM10K_CONFIG_WORD_LEN);
+ }
+
+ if (strcmp(cfgs[1], "int") == 0)
+ val_type = FM10K_CONFIG_VALUE_INT;
+ else if (strcmp(cfgs[1], "string") == 0)
+ val_type = FM10K_CONFIG_VALUE_STR;
+ else
+ return -1;
+
+ for (i = 0; i < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]); i++) {
+ key = &fm10k_config_key_items[i];
+ for (j = 0; j < FM10K_CONFIG_WORD_MAX; j++) {
+ if (key->key_str[j] &&
+ strlen(cmp_keys[j]) > 0) {
+ if (strcmp(key->key_str[j], "*") == 0) {
+ key_param =
+ strtol(cmp_keys[j], &endptr, 10);
+ if ((key_param == 0 &&
+ endptr == cmp_keys[j]) ||
+ (endptr != cmp_keys[j] &&
+ strlen(endptr) != 0))
+ break;
+ } else if (strcmp(key->key_str[j],
+ cmp_keys[j]) != 0) {
+ break;
+ }
+ } else if (key->key_str[j] == 0 &&
+ strlen(cmp_keys[j]) == 0) {
+ if (val_type == FM10K_CONFIG_VALUE_STR) {
+ item->val.str =
+ malloc(strlen(cfgs[2]) + 1);
+ if (item->val.str == NULL)
+ return -1;
+ strcpy(item->val.str, cfgs[2]);
+ } else {
+ item->val.int64 =
+ strtol(cfgs[2], NULL, 10);
+ }
+ item->type = key->type;
+ item->key_param = key_param;
+ item->val_type = val_type;
+ return 0;
+ }
+ }
+ }
+ return -1;
+}
+
+static bool
+fm10k_config_blank_line_check(char *line)
+{
+ uint16_t i;
+
+ for (i = 0; i < strlen(line); i++) {
+ if (line[i] == ' ' ||
+ line[i] == '\n' ||
+ line[i] == '\t' || line[i] == 0x0d) /* 0d: CR */
+ continue;
+ else
+ return false;
+ }
+ return true;
+}
+
+static int
+fm10k_config_conf_file_load(void)
+{
+ int i = 0;
+ FILE *fp;
+ char buff[255];
+ struct fm10k_cfg_config_item *item;
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "r");
+ if (fp == NULL)
+ return -1;
+
+ fm10k_config_dpdk_cfg.config_list = malloc
+ (sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+ if (fm10k_config_dpdk_cfg.config_list == NULL)
+ return -1;
+ memset(fm10k_config_dpdk_cfg.config_list, 0,
+ sizeof(struct fm10k_cfg_config_item) * FM10K_SW_CONFIG_MAX);
+
+ while (fgets(buff, sizeof(buff), fp)) {
+ if (buff[0] == '#' || buff[0] == 0)
+ continue;
+
+ if (fm10k_config_blank_line_check(buff))
+ continue;
+
+ item = &fm10k_config_dpdk_cfg.config_list[i++];
+ if (fm10k_config_conf_line_parser(buff, item) < 0) {
+ FM10K_SW_ERR("Unknown configuration: %s", buff);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+struct fm10k_hw*
+fm10k_config_hw_get(int port_no)
+{
+ int hw_port;
+
+ hw_port = fm10k_config_dpdk_cfg.dpdk_port_map[port_no].map_no[0];
+ return fm10k_config_dpdk_cfg.pf_hw[hw_port];
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_current_get(void)
+{
+ return fm10k_config_dpdk_cfg.current;
+}
+
+void
+fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new)
+{
+ fm10k_config_dpdk_cfg.current = new;
+}
+
+bool
+fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow)
+{
+ return (list == flow);
+}
+
+static void
+fm10k_config_flow_list_init(struct fm10k_cfg_flow *list)
+{
+ list->next = list;
+ list->prev = list;
+}
+
+static void
+fm10k_config_flow_list_add(struct fm10k_cfg_flow *new,
+ struct fm10k_cfg_flow *list)
+{
+ struct fm10k_cfg_flow *next = list->next;
+ next->prev = new;
+ new->next = next;
+ new->prev = list;
+ list->next = new;
+}
+
+void
+fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow)
+{
+ fm10k_config_flow_list_add(flow, flowset->flow_head.prev);
+}
+
+void
+fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow)
+{
+ flow->prev->next = flow->next;
+ flow->next->prev = flow->prev;
+ flow->prev = NULL;
+ flow->next = NULL;
+}
+
+struct fm10k_cfg_flowset *
+fm10k_config_flowset_get(const char *name)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ if (strcmp(flowset->name, name) == 0)
+ break;
+ flowset = flowset->next;
+ }
+
+ if (flowset == NULL) {
+ flowset = malloc(sizeof(struct fm10k_cfg_flowset));
+ if (flowset == NULL)
+ return NULL;
+ strcpy(flowset->name, name);
+ fm10k_config_flow_list_init(&flowset->flow_head);
+ flowset->next = fm10k_config_dpdk_cfg.flowset_head.next;
+ fm10k_config_dpdk_cfg.flowset_head.next = flowset;
+ }
+ return flowset;
+}
+
+
+static int
+fm10k_cfg_flow_item_set(struct fm10k_cfg_flowset *flowset,
+ uint8_t flow_no, uint8_t type, int64_t val)
+{
+ struct fm10k_cfg_flow *tmp;
+ struct fm10k_cfg_flow *flow = NULL;
+
+ if (flowset == NULL)
+ return -1;
+
+ tmp = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, tmp)) {
+ if (tmp->flow_no == flow_no) {
+ flow = tmp;
+ break;
+ }
+ tmp = tmp->next;
+ }
+ if (flow == NULL) {
+ flow = malloc(sizeof(struct fm10k_cfg_flow));
+ memset(flow, 0, sizeof(struct fm10k_cfg_flow));
+ flow->flow_no = flow_no;
+ fm10k_config_flow_list_add_tail(flowset, flow);
+ }
+ if (flow == NULL)
+ return -1;
+
+ switch (type) {
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ flow->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->src_port.port_no = val;
+ break;
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ flow->src_port.vlan_id = val;
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ if (flow->fw_port[0].port_type == FM10K_CONFIG_FLOW_NONE_PORT) {
+ flow->fw_port[0].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[0].port_no = val;
+ } else {
+ flow->fw_port[1].port_type =
+ FM10K_CONFIG_FLOW_DPDK_PORT;
+ flow->fw_port[1].port_no = val;
+ }
+ break;
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flow->fw_port[0].vlan_id == 0)
+ flow->fw_port[0].vlan_id = val;
+ else
+ flow->fw_port[1].vlan_id = val;
+ break;
+ default:
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_transfer(void)
+{
+ int i;
+ int offset;
+ uint32_t tmp, data;
+ struct fm10k_cfg_config_item *item;
+ struct fm10k_cfg_flowset *flowset = NULL;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+
+ switch (item->type) {
+ case FM10K_CONFIG_BIND_PF_NUMBER:
+ fm10k_config_dpdk_cfg.pf_num = item->val.int64;
+ break;
+ case FM10K_CONFIG_EXT_PORT_SPEED:
+ fm10k_config_dpdk_cfg.ext_port_speed = item->val.int64;
+ break;
+ case FM10K_CONFIG_DPDK_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.dpdk_port_map[item->key_param];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+ case FM10K_CONFIG_EXT_PORT_MAP_PF:
+ map =
+ &fm10k_config_dpdk_cfg.ext_port_map
+ [item->key_param - 1];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PFS;
+ map->map_no[1] = item->val.int64;
+ } else {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = item->val.int64;
+ }
+ break;
+
+ case FM10K_CONFIG_DEBUG_ENABLE:
+ case FM10K_CONFIG_DEBUG_CONFIG:
+ case FM10K_CONFIG_DEBUG_FFU_INIT:
+ case FM10K_CONFIG_DEBUG_FFU_REG:
+ case FM10K_CONFIG_DEBUG_FFU_RULE:
+ case FM10K_CONFIG_DEBUG_STATS_PORT:
+ case FM10K_CONFIG_DEBUG_STATS_QUEUE:
+ case FM10K_CONFIG_DEBUG_STATS_FFU:
+ case FM10K_CONFIG_DEBUG_STATS_MORE:
+ offset = item->type - FM10K_CONFIG_DEBUG_START;
+ tmp = fm10k_config_dpdk_cfg.debug_cfg;
+ data = 1;
+ if (item->val.int64 != 1)
+ fm10k_config_dpdk_cfg.debug_cfg =
+ tmp & ~(data << offset);
+ else
+ fm10k_config_dpdk_cfg.debug_cfg |=
+ (data << offset);
+ break;
+ case FM10K_CONFIG_DEBUG_STATS_INTERVAL:
+ fm10k_config_dpdk_cfg.stats_interval = item->val.int64;
+ break;
+
+ case FM10K_CONFIG_FLOWSET_START:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ flowset = fm10k_config_flowset_get(item->val.str);
+ if (flowset == NULL)
+ return -1;
+ break;
+ case FM10K_CONFIG_FLOWSET_STOP:
+ flowset = NULL;
+ break;
+ case FM10K_CONFIG_FLOWSET_ENABLE:
+ /* skip /n */
+ item->val.str[strlen(item->val.str) - 1] = 0;
+ fm10k_config_dpdk_cfg.current =
+ fm10k_config_flowset_get(item->val.str);
+ break;
+ case FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT:
+ case FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_COND_VLAN:
+ case FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT:
+ case FM10K_CONFIG_FLOW_ACT_FW_VALN:
+ if (flowset == NULL ||
+ fm10k_cfg_flow_item_set(flowset,
+ item->key_param, item->type,
+ item->val.int64) != 0)
+ return -1;
+ break;
+ default:
+ return -1;
+ }
+ }
+ return 0;
+}
+
+static bool
+fm10k_config_conf_file_exist(void)
+{
+ if (access(fm10k_config_dpdk_conf_file, 0) == 0)
+ return true;
+ return false;
+}
+
+static int
+fm10k_config_conf_file_item_create(FILE *fp, char *buff,
+ struct fm10k_cfg_key_item *key,
+ struct fm10k_cfg_config_item *item)
+{
+ int i;
+
+ if (item->type != key->type)
+ return 0;
+
+ for (i = 0; i < FM10K_CONFIG_WORD_MAX; i++) {
+ if (key->key_str[i] == 0)
+ break;
+ if (i == 0) {
+ sprintf(buff + strlen(buff),
+ "%s", key->key_str[i]);
+ } else {
+ if (strcmp(key->key_str[i], "*") == 0)
+ sprintf(buff + strlen(buff),
+ ".%u", item->key_param);
+ else
+ sprintf(buff + strlen(buff),
+ ".%s", key->key_str[i]);
+ }
+ }
+ if (item->val_type == FM10K_CONFIG_VALUE_INT)
+ sprintf(buff + strlen(buff),
+ " int %lld\n", (long long)item->val.int64);
+ else if (item->val_type == FM10K_CONFIG_VALUE_STR)
+ sprintf(buff + strlen(buff), " string %s\n", item->val.str);
+ else
+ return -1;
+ fwrite(buff, strlen(buff), 1, fp);
+ FM10K_SW_TRACE("[write] %s", buff);
+
+ return 0;
+}
+
+static int
+fm10k_config_conf_file_create(void)
+{
+ uint16_t i, j;
+ struct fm10k_cfg_key_item *key;
+ struct fm10k_cfg_config_item *item;
+ FILE *fp;
+ char buff[255] = "";
+
+ fp = fopen(fm10k_config_dpdk_conf_file, "w");
+ if (fp == NULL)
+ return -1;
+
+ for (i = 0; i < FM10K_SW_CONFIG_MAX; i++) {
+ item = &fm10k_config_dpdk_cfg.config_list[i];
+ if (item->type == FM10K_CONFIG_TYPE_NULL)
+ break;
+ buff[0] = 0;
+ if (strlen(item->describe) > 0)
+ sprintf(buff, "\n\n%s\n", item->describe);
+ for (j = 0; j < sizeof(fm10k_config_key_items) /
+ sizeof(fm10k_config_key_items[0]); j++) {
+ key = &fm10k_config_key_items[j];
+ if (fm10k_config_conf_file_item_create
+ (fp, buff, key, item) != 0) {
+ fclose(fp);
+ return -1;
+ }
+ }
+ }
+ fclose(fp);
+ return 0;
+}
+
+void
+fm10k_config_cfg_flowset_show(void)
+{
+ struct fm10k_cfg_flowset *flowset;
+
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d & %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d --> "
+ "%4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+}
+
+static void
+fm10k_config_cfg_describe(struct fm10k_switch *sw,
+ struct fm10k_device_info *info)
+{
+ uint16_t i;
+ struct fm10k_cfg_flowset *flowset;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K STATIC CONFIG ---\n");
+ FM10K_SW_INFO(" Card Type: %s\n", info->desc);
+ FM10K_SW_INFO(" PF Max : %d\n", fm10k_config_dpdk_cfg.pf_max);
+ FM10K_SW_INFO(" PF Bind : %d\n", fm10k_config_dpdk_cfg.pf_num);
+ FM10K_SW_INFO(" DEBUG : %#x\n", fm10k_config_dpdk_cfg.debug_cfg);
+ FM10K_SW_INFO(" STATS GAP: %d sec\n",
+ fm10k_config_dpdk_cfg.stats_interval);
+ FM10K_SW_INFO(" EXT PORT speed: %d Gbps\n",
+ fm10k_config_dpdk_cfg.ext_port_speed);
+ FM10K_SW_INFO(" FLOWSET ENABLE: %s\n",
+ fm10k_config_dpdk_cfg.current->name);
+
+ for (i = 0; i < fm10k_config_dpdk_cfg.ext_port_num; i++) {
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" EXT PORT[%d] MAP: PF%d PF%d\n", i + 1,
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.ext_port_map[i].map_no[1]);
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_config_dpdk_cfg.dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+ if (fm10k_config_dpdk_cfg.ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0]);
+ else
+ FM10K_SW_INFO(" DPDK PORT[%d] MAP: PF%d PF%d\n", i,
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[0],
+ fm10k_config_dpdk_cfg.dpdk_port_map[i].map_no[1]);
+ }
+
+ flowset = fm10k_config_dpdk_cfg.flowset_head.next;
+ while (flowset) {
+ FM10K_SW_INFO("\n FLOWSET : %s\n", flowset->name);
+ struct fm10k_cfg_flow *flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ const char *port_type[3] = { "NON", "EXT", "DPDK" };
+ if (flow->fw_port[1].port_type !=
+ FM10K_CONFIG_FLOW_NONE_PORT) {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d & %4s PORT %d"
+ " VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id,
+ port_type[flow->fw_port[1].port_type],
+ flow->fw_port[1].port_no,
+ flow->fw_port[1].vlan_id);
+ } else {
+ FM10K_SW_INFO(" FLOW %d : %4s PORT %d VLAN %4d"
+ " --> %4s PORT %d VLAN %4d\n",
+ flow->flow_no,
+ port_type[flow->src_port.port_type],
+ flow->src_port.port_no,
+ flow->src_port.vlan_id,
+ port_type[flow->fw_port[0].port_type],
+ flow->fw_port[0].port_no,
+ flow->fw_port[0].vlan_id);
+ }
+ flow = flow->next;
+ }
+ flowset = flowset->next;
+ }
+ FM10K_SW_INFO("\n");
+}
+
+int
+fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw)
+{
+ struct fm10k_device_info *info = fm10k_get_device_info(hw);
+
+ fm10k_config_dpdk_cfg.stats_interval = 2;
+ fm10k_config_dpdk_cfg.pf_max = info->num_peps;
+ fm10k_config_dpdk_cfg.ext_port_num = info->num_ext_ports;
+
+ if (!fm10k_config_conf_file_exist()) {
+ if (info->num_epls == 2 && info->num_peps == 2)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_2pep;
+ else if (info->num_epls == 2 && info->num_peps == 4)
+ fm10k_config_dpdk_cfg.config_list =
+ fm10k_silc_nic_2ext_4pep;
+ else
+ return -1;
+
+ fm10k_config_conf_file_create();
+ } else {
+ if (fm10k_config_conf_file_load() < 0)
+ return -1;
+ }
+
+ if (fm10k_config_conf_file_transfer() < 0)
+ return -1;
+ sw->dpdk_cfg = &fm10k_config_dpdk_cfg;
+
+ fm10k_config_cfg_describe(sw, info);
+
+ return 0;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_config.h b/drivers/net/fm10k/switch/fm10k_config.h
new file mode 100644
index 0000000..f1c6ee4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_config.h
@@ -0,0 +1,178 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_CONFIG_H_
+#define _FM10K_CONFIG_H_
+
+
+#include <stdint.h>
+#include "fm10k_switch.h"
+
+/* General configuration */
+#define FM10K_CONFIG_TYPE_NULL 0
+#define FM10K_CONFIG_BIND_PF_NUMBER 1
+#define FM10K_CONFIG_EXT_PORT_SPEED 2
+
+/* Internal redirect configuration */
+#define FM10K_CONFIG_DPDK_PORT_MAP_PF 10
+#define FM10K_CONFIG_EXT_PORT_MAP_PF 11
+
+/* Debug configuration */
+#define FM10K_CONFIG_DEBUG_START 50
+#define FM10K_CONFIG_DEBUG_ENABLE (FM10K_CONFIG_DEBUG_START + 0)
+#define FM10K_CONFIG_DEBUG_CONFIG (FM10K_CONFIG_DEBUG_START + 1)
+#define FM10K_CONFIG_DEBUG_FFU_INIT (FM10K_CONFIG_DEBUG_START + 2)
+#define FM10K_CONFIG_DEBUG_FFU_REG (FM10K_CONFIG_DEBUG_START + 3)
+#define FM10K_CONFIG_DEBUG_FFU_RULE (FM10K_CONFIG_DEBUG_START + 4)
+#define FM10K_CONFIG_DEBUG_STATS_PORT (FM10K_CONFIG_DEBUG_START + 5)
+#define FM10K_CONFIG_DEBUG_STATS_QUEUE (FM10K_CONFIG_DEBUG_START + 6)
+#define FM10K_CONFIG_DEBUG_STATS_FFU (FM10K_CONFIG_DEBUG_START + 7)
+#define FM10K_CONFIG_DEBUG_STATS_MORE (FM10K_CONFIG_DEBUG_START + 8)
+#define FM10K_CONFIG_DEBUG_STATS_INTERVAL (FM10K_CONFIG_DEBUG_START + 9)
+
+/* external redirect configuration */
+#define FM10K_CONFIG_FLOWSET_START 80
+#define FM10K_CONFIG_FLOWSET_STOP 81
+#define FM10K_CONFIG_FLOWSET_ENABLE 82
+
+#define FM10K_CONFIG_FLOW_COND_START 100
+#define FM10K_CONFIG_FLOW_COND_SRC_EXT_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 0)
+#define FM10K_CONFIG_FLOW_COND_SRC_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_COND_START + 1)
+#define FM10K_CONFIG_FLOW_COND_VLAN \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+#define FM10K_CONFIG_FLOW_COND_END \
+ (FM10K_CONFIG_FLOW_COND_START + 2)
+
+#define FM10K_CONFIG_FLOW_ACT_START 200
+#define FM10K_CONFIG_FLOW_ACT_FW_EXT_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 0)
+#define FM10K_CONFIG_FLOW_ACT_FW_DPDK_PORT \
+ (FM10K_CONFIG_FLOW_ACT_START + 1)
+#define FM10K_CONFIG_FLOW_ACT_FW_VALN \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+#define FM10K_CONFIG_FLOW_ACT_END \
+ (FM10K_CONFIG_FLOW_ACT_START + 2)
+
+#define FM10K_CONFIG_FLOW_NONE_PORT 0
+#define FM10K_CONFIG_FLOW_EXT_PORT 1
+#define FM10K_CONFIG_FLOW_DPDK_PORT 2
+
+#define FM10K_CONFIG_VALUE_NULL 0
+#define FM10K_CONFIG_VALUE_INT 1
+#define FM10K_CONFIG_VALUE_STR 2
+
+/* SWITCH config */
+#define FM10K_CONFIG_PORT_MAP_NULL 0
+#define FM10K_CONFIG_PORT_MAP_PF 1
+#define FM10K_CONFIG_PORT_MAP_PFS 2
+#define FM10K_CONFIG_PORT_MAP_PFSS 3
+
+/* DPDK port */
+#define FM10K_CONFIG_DPDK_NULL 0
+#define FM10K_CONFIG_DPDK_PF 1
+#define FM10K_CONFIG_DPDK_VF 2
+#define FM10K_CONFIG_DPDK_MAX 3
+
+struct fm10k_cfg_port_pf_map {
+ uint8_t type;
+ uint8_t map_no[2];
+};
+
+/* Configuration read from file */
+struct fm10k_cfg_config_item {
+ uint8_t type;
+ uint8_t val_type;
+ uint16_t key_param;
+ const char *describe;
+ union {
+ int64_t int64;
+ char *str;
+ } val;
+};
+
+struct fm10k_hw;
+
+struct fm10k_dpdk_port {
+ uint8_t type;
+ struct fm10k_hw *hw;
+ void *rte_dev;
+ void *flow_list;
+ uint16_t pf_no;
+ uint8_t tx_queue_num;
+ uint8_t rx_queue_num;
+};
+
+
+/* Flow configuration */
+struct fm10k_cfg_port {
+ uint8_t port_type;
+ uint8_t port_no;
+ uint16_t vlan_id;
+};
+
+struct fm10k_cfg_flow {
+ /* set by configuration */
+ struct fm10k_cfg_flow *prev;
+ struct fm10k_cfg_flow *next;
+ uint8_t flow_no; /* only configured flow has this NO. */
+ struct fm10k_cfg_port src_port;
+ struct fm10k_cfg_port fw_port[2];
+ /* set by ffu rule add */
+ uint16_t rule_id;
+};
+
+#define FM10K_CONFIG_FLOWSET_NAME_MAX 256
+struct fm10k_cfg_flowset {
+ char name[FM10K_CONFIG_FLOWSET_NAME_MAX];
+ struct fm10k_cfg_flow flow_head;
+ struct fm10k_cfg_flowset *next;
+};
+
+/* Configuration */
+struct fm10k_dpdk_cfg {
+ uint8_t pf_num; /* configure by conf */
+ uint8_t pf_bind; /* initialize by dpdk */
+ uint8_t pf_max; /* set by card type */
+ uint8_t ext_port_num; /* configure by conf */
+ uint8_t ext_port_speed; /* configure by conf */
+ uint32_t debug_cfg; /* configure by conf */
+ uint32_t stats_interval;/* configure by conf */
+
+ struct fm10k_hw *master_hw; /* initialize by dpdk */
+ struct fm10k_hw *pf_hw[FM10K_SW_PEP_PORTS_MAX]; /* initialize by dpdk */
+
+ /* initialize by dpdk */
+ struct fm10k_dpdk_port ports[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf or default*/
+ struct fm10k_cfg_port_pf_map dpdk_port_map[FM10K_SW_LOGICAL_PORTS_MAX];
+ /* configure by conf */
+ struct fm10k_cfg_port_pf_map ext_port_map[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_cfg_config_item *config_list; /* configure by conf */
+ struct fm10k_cfg_flowset flowset_head; /* transfer from conf file */
+ struct fm10k_cfg_flowset *current;
+};
+
+static inline bool
+fm10k_config_check_debug(struct fm10k_dpdk_cfg *cfg, uint16_t dbg)
+{
+ return cfg->debug_cfg & 1 << (dbg - FM10K_CONFIG_DEBUG_START) &&
+ cfg->debug_cfg & 1;
+}
+
+struct fm10k_cfg_flowset *fm10k_config_flowset_get(const char *name);
+struct fm10k_cfg_flowset *fm10k_config_flowset_current_get(void);
+void fm10k_config_flowset_current_set(struct fm10k_cfg_flowset *new);
+void fm10k_config_flow_list_add_tail(struct fm10k_cfg_flowset *flowset,
+ struct fm10k_cfg_flow *flow);
+void fm10k_config_flow_list_delete(struct fm10k_cfg_flow *flow);
+bool fm10k_config_flow_list_end(struct fm10k_cfg_flow *list,
+ struct fm10k_cfg_flow *flow);
+
+void fm10k_config_cfg_flowset_show(void);
+struct fm10k_hw *fm10k_config_hw_get(int port_no);
+int fm10k_config_init(struct fm10k_switch *sw, struct fm10k_hw *hw);
+
+#endif /* _FM10K_CONFIG_H */
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.c b/drivers/net/fm10k/switch/fm10k_ffu.c
new file mode 100644
index 0000000..37be806
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.c
@@ -0,0 +1,1250 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+
+/*
+ * one SLICE for 40bits SEL
+ * SLICE 28 FOR SEL SGLORT(16bits) and VLAN(16bits)
+ * FOR ACT ROUTE
+ * SLICE 29 FOR SEL ETHER_TYPE(16bits)
+ * FOR ACT MIRROR | SET_VLAN
+ * SLICE 30 FOR SEL MPLS_LABEL0(32bits)
+ * SLICE 31 FOR SEL MPLS_LABEL1(32bits)
+ */
+#define FM10K_FFU_SLICE_START 28
+#define FM10K_FFU_SLICE_SGLORT_VID 28
+#define FM10K_FFU_SLICE_ETYPE_VID2 29
+#define FM10K_FFU_SLICE_MPLS_LABEL0 30
+#define FM10K_FFU_SLICE_MPLS_LABEL1 31
+
+#define FM10K_FFU_SLICE_SGLORT_VLAN_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_SGLORT) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI_VID) | \
+ FM10K_SW_FFU_SLICE_CFG_START_ACTION | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_ETHER_TYPE_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L2_TYPE) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_VPRI2_VID2) | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL0_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_127_96) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+#define FM10K_FFU_SLICE_MPLS_LABEL1_CFG \
+ (FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_0, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_1, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_2, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_CFG_SELECT_3, \
+ FM10K_SW_FFU_MUX_SEL_L3_SIP_95_64) | \
+ FM10K_SW_FFU_SLICE_CFG_START_COMPARE | \
+ FM10K_SW_FFU_SLICE_CFG_VALID_LOW)
+
+static uint64_t fm10k_ffu_slice_cfgs[4] = {
+ FM10K_FFU_SLICE_SGLORT_VLAN_CFG,
+ FM10K_FFU_SLICE_ETHER_TYPE_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL0_CFG,
+ FM10K_FFU_SLICE_MPLS_LABEL1_CFG
+};
+
+#define FM10K_FFU_INIT_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_INIT)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_RULE_PRINT(cfg, ...) \
+do { \
+ if (fm10k_config_check_debug(cfg, FM10K_CONFIG_DEBUG_FFU_RULE)) \
+ FM10K_SW_INFO(__VA_ARGS__); \
+} while (0)
+
+#define FM10K_FFU_CNT_INDEX(x) (FM10K_SW_FFU_CNT_START + (x))
+#define FM10K_FFU_FLOW_START 10
+#define FM10K_FFU_MIRROR_PROFILE 1
+#define FM10K_FFU_SGLORT_TYPE_NULL 0
+#define FM10K_FFU_SGLORT_TYPE_EPL 1
+#define FM10K_FFU_SGLORT_TYPE_PF 2
+#define FM10K_FFU_SGLORT_TYPE_PFS 3
+#define FM10K_FFU_SGLORT_TYPE_DPDK 4
+
+struct fm10k_ffu_rule_data {
+ uint16_t sglort_type;
+ uint16_t cond_sglort;
+ uint16_t cond_vlan;
+ uint16_t act_dglort;
+ uint16_t act_vlan;
+ uint16_t bypass_dglort;
+ uint16_t bypass_vlan;
+};
+
+
+static uint8_t fm10k_ffu_bitmap[FM10K_FFU_RULE_MAX / 8];
+static uint8_t
+fm10k_ffu_rule_get_bit(int id)
+{
+ int num = id / 8;
+ int offset = id % 8;
+
+ return (fm10k_ffu_bitmap[num] >> offset) & 0x1;
+}
+
+static void
+fm10k_ffu_rule_set_bit(int id, uint8_t bit)
+{
+ int num = id / 8;
+ int offset = id % 8;
+ uint8_t tmp = fm10k_ffu_bitmap[num];
+ uint8_t data = 1;
+
+ if (bit == 0)
+ fm10k_ffu_bitmap[num] = tmp & ~(data << offset);
+ else
+ fm10k_ffu_bitmap[num] |= (data << offset);
+}
+
+static int
+fm10k_ffu_rule_alloc(void)
+{
+ int i;
+
+ for (i = FM10K_FFU_FLOW_START; i < FM10K_FFU_RULE_MAX; i++) {
+ if (fm10k_ffu_rule_get_bit(i) != 0)
+ continue;
+ fm10k_ffu_rule_set_bit(i, 1);
+ return i;
+ }
+ return -1;
+}
+
+static void
+fm10k_ffu_rule_free(int id)
+{
+ fm10k_ffu_rule_set_bit(id, 0);
+}
+
+static void
+fm10k_ffu_always_mismatch(struct fm10k_switch *sw,
+ unsigned int slice, unsigned int idx)
+{
+ uint64_t temp64;
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~0ULL) |
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY_TOP, ~0ULL);
+ fm10k_write_switch_reg128(sw, FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64, temp64);
+}
+
+static void
+fm10k_ffu_route_dglort(struct fm10k_switch *sw, unsigned int slice,
+ unsigned int idx, uint16_t sglort, uint16_t dglort)
+{
+ uint64_t temp64, temp64_2;
+
+ /*
+ * Set the key to exact match on the 16 SGLORT bits and
+ * always match everywhere else.
+ */
+ temp64 = FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, sglort);
+ temp64_2 =
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_TCAM_KEY, ~sglort & 0xffff);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(slice, idx),
+ temp64_2, temp64);
+
+ /*
+ * Set the corresponding SRAM entry to ROUTE_GLORT to the
+ * corresponding DGLORT.
+ */
+ temp64 = 0x40;
+ temp64 = temp64 << 32 |
+ FM10K_SW_MAKE_REG_FIELD(FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ /*
+ * 11.5.4.2 FFU_SLICE_SRAM[0..31][0..1023]
+ */
+ temp64_2 =
+ FM10K_SW_FFU_CNT_BANK << (35 - 23) |
+ (FM10K_SW_FFU_CNT_START + idx);
+ temp64 |= temp64_2 << 23;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(slice, idx), temp64);
+}
+
+
+static void
+fm10k_ffu_set_dest_glort_mask(struct fm10k_switch *sw,
+ unsigned int idx, uint16_t dglort, uint64_t dport_mask)
+{
+ uint64_t temp64;
+ unsigned int multiple_dports;
+ unsigned int num_dports;
+ unsigned int amplification_factor;
+ unsigned int hashed_entries;
+ unsigned int i, j;
+
+ multiple_dports = (dport_mask & (dport_mask - 1));
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to port ", __func__, dglort);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ if (multiple_dports) {
+ /*
+ * Create a pair of entries and use the hash value to select
+ * among them.
+ */
+ num_dports = 0;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i)) {
+ num_dports++;
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d", i);
+ }
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, "\n");
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ amplification_factor = 16 / num_dports;
+ hashed_entries = num_dports * amplification_factor;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_HASHED);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_COUNT,
+ hashed_entries);
+ fm10k_write_switch_reg64(sw, FM10K_SW_GLORT_RAM(idx), temp64);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ for (j = 0; j < amplification_factor; j++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ (1ULL << i)));
+ sw->glort_dest_table_idx++;
+ }
+ } else {
+ /*
+ * Set the corresponding entry in the DGLORT map RAM to use
+ * strict indexing straight into the DEST_TABLE, then write
+ * the corresponding destination port in the DEST_TABLE.
+ */
+
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(sw->glort_dest_table_idx),
+ FM10K_SW_MAKE_REG_FIELD64
+ (GLORT_DEST_TABLE_MASK,
+ dport_mask));
+ sw->glort_dest_table_idx++;
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++)
+ if (dport_mask & (1ULL << i))
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg, " %d\n", i);
+ }
+}
+
+static struct fm10k_multi_glort {
+ uint8_t lport1;
+ uint8_t lport2;
+ uint16_t vlan1;
+ uint16_t vlan2;
+} fm10k_multi_glorts[FM10K_SW_FFU_RULE_MAX];
+
+static uint32_t
+fm10k_ffu_multi_glort_get(uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2, bool *p_new)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (lport1 == fm10k_multi_glorts[i].lport1 &&
+ lport2 == fm10k_multi_glorts[i].lport2 &&
+ vlan1 == fm10k_multi_glorts[i].vlan1 &&
+ vlan2 == fm10k_multi_glorts[i].vlan2) {
+ if (p_new != NULL)
+ *p_new = false;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ for (i = 0; i < FM10K_SW_FFU_RULE_MAX; i++) {
+ if (fm10k_multi_glorts[i].lport1 == 0 &&
+ fm10k_multi_glorts[i].lport2 == 0 &&
+ fm10k_multi_glorts[i].vlan1 == 0 &&
+ fm10k_multi_glorts[i].vlan2 == 0) {
+ fm10k_multi_glorts[i].lport1 = lport1;
+ fm10k_multi_glorts[i].lport2 = lport2;
+ fm10k_multi_glorts[i].vlan1 = vlan1;
+ fm10k_multi_glorts[i].vlan2 = vlan2;
+ if (p_new != NULL)
+ *p_new = true;
+ return FM10K_SW_MULTI_GLORT_START + i;
+ }
+ }
+
+ return 0;
+}
+
+static uint32_t
+fm10k_ffu_set_dest_glort_multi_cast(struct fm10k_switch *sw,
+ uint8_t lport1, uint8_t lport2,
+ uint16_t vlan1, uint16_t vlan2)
+{
+ uint32_t dglort;
+ bool is_new = false;
+ uint32_t idx;
+ uint64_t temp64;
+
+ dglort =
+ fm10k_ffu_multi_glort_get(lport1, lport2,
+ vlan1, vlan2, &is_new);
+
+ if (!is_new)
+ return dglort;
+
+ FM10K_FFU_INIT_PRINT(sw->dpdk_cfg,
+ "%s set glort %#x to (%d:%d) (%d:%d)\n",
+ __func__, dglort, lport1, vlan1, lport2, vlan2);
+ /*
+ * Create an exact-match key for the given DGLORT in the DGLORT CAM.
+ */
+ idx = sw->glort_cam_ram_idx++;
+ fm10k_write_switch_reg(sw, FM10K_SW_GLORT_CAM(idx),
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY, dglort) |
+ FM10K_SW_MAKE_REG_FIELD(GLORT_CAM_KEY_INVERT, ~dglort));
+
+ /*
+ * Create multiple entries for each dport to increase the
+ * hash modulus and capture more hash entropy. The maximum
+ * number of hashed entries is 16.
+ */
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_STRICT,
+ FM10K_SW_GLORT_RAM_STRICT_STRICT);
+ temp64 |=
+ FM10K_SW_MAKE_REG_FIELD64(GLORT_RAM_DEST_INDEX,
+ sw->glort_dest_table_idx);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_RAM(idx), temp64);
+
+ /* GLORT_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * DestMask 47:0 RW 0x0
+ * IP_MulticastIndex 59:48 RW 0x0
+ * Reserved 63:60 RSV 0x0
+ */
+ temp64 = sw->mcast_dest_table_idx;
+ temp64 = temp64 << 48 | 1 << lport1 | 1 << lport2;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE
+ (sw->glort_dest_table_idx), temp64);
+ sw->glort_dest_table_idx++;
+
+ /* MCAST_DEST_TABLE
+ * Field Name Bit(s) Type Default
+ * PortMask 47:0 RW 0x0
+ * LenTableIdx 61:48 RW 0x0
+ * Reserved 63:62 RSV 00b
+ */
+ temp64 = sw->mcast_len_table_idx;
+ temp64 = 1 << lport1 | 1 << lport2 | temp64 << 48;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE
+ (sw->mcast_dest_table_idx++), temp64);
+
+ /* MCAST_LEN_TABLE
+ * Field Name Bit(s) Type Default
+ * L3_McastIdx 14:0 RW 0x0
+ * L3_Repcnt 26:15 RW 0x0
+ * Reserved 31:27 RSV 0x0
+ */
+ temp64 =
+ sw->mcast_vlan_table_idx | 1 << 15;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE
+ (sw->mcast_len_table_idx++), temp64);
+
+ /* MCAST_VLAN_TABLE
+ * Field Name Bit(s) Type Default
+ * VID 11:0 RW 0x0
+ * DGLORT 27:12 RW 0x0
+ * ReplaceVID 28 RW 0b
+ * ReplaceDGLORT 29 RW 0b
+ * Reserved 31:30 RSV 00b
+ */
+ temp64 = vlan1 |
+ fm10k_switch_pf_glort_get
+ (lport1) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+ temp64 = vlan2 |
+ fm10k_switch_pf_glort_get
+ (lport2) << 12 | 1 << 28 | 1 << 29;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE
+ (sw->mcast_vlan_table_idx++), temp64);
+
+ return dglort;
+}
+
+static uint64_t
+fm10k_data64_field64_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+
+ if (start == end) {
+ tmp64 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp64 = tmp64 >> (64 - end + start);
+ }
+ return tmp64;
+}
+
+static uint32_t
+fm10k_data64_field32_get(uint64_t data, int start, int end)
+{
+ uint64_t tmp64 = data;
+ uint32_t tmp32;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp64 = tmp64 << (64 - end);
+ tmp32 = tmp64 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+static uint32_t
+fm10k_data32_field_get(uint32_t data, int start, int end)
+{
+ uint32_t tmp32 = data;
+
+ if (start == end) {
+ tmp32 = (data >> start) & 1;
+ } else {
+ tmp32 = tmp32 << (64 - end);
+ tmp32 = tmp32 >> (64 - end + start);
+ }
+ return tmp32;
+}
+
+
+static void
+fm10k_glort_register_dump(struct fm10k_switch *sw)
+{
+ uint32_t i;
+ uint64_t data64;
+ uint32_t data32;
+ uint32_t tmp32;
+
+ if (!fm10k_config_check_debug
+ (sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("----- GLORT -----\n");
+
+ for (i = 0; i < sw->glort_cam_ram_idx; i++) {
+ data32 = fm10k_read_switch_reg(sw, FM10K_SW_GLORT_CAM(i));
+ FM10K_SW_INFO("[%02u]GLORT_CAM %#x Key %#x\n", i, data32,
+ fm10k_data32_field_get(data32, 0, 15));
+ data64 = fm10k_read_switch_reg64(sw, FM10K_SW_GLORT_RAM(i));
+ FM10K_SW_INFO(" GLORT_RAM %#llx Strict %u DestIndex %u\n",
+ (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 0, 1),
+ fm10k_data64_field32_get(data64, 2, 13));
+ tmp32 = fm10k_data64_field32_get(data64, 2, 13);
+ data64 =
+ fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLORT_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" GLORT_DEST_TABLE[%u] %#llx "
+ "IP_MulticastIndex %u\n",
+ tmp32, (unsigned long long)data64,
+ fm10k_data64_field32_get(data64, 48, 59));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 59);
+ if (tmp32 == 0)
+ continue;
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_SCHED_MCAST_DEST_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_DEST_TABLE[%u] "
+ "%#llx PortMask %#llx"
+ " LenTableIdx %u\n", tmp32,
+ (unsigned long long)data64,
+ (unsigned long long)
+ fm10k_data64_field64_get(data64, 0, 47),
+ fm10k_data64_field32_get(data64, 48, 61));
+ tmp32 = fm10k_data64_field32_get(data64, 48, 61);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_SCHED_MCAST_LEN_TABLE(tmp32));
+ FM10K_SW_INFO(" SCHED_MCAST_LEN_TABLE[%u] %#x "
+ "L3_McastIdx %u L3_Repcnt %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 14),
+ fm10k_data32_field_get(data32, 15, 26));
+ tmp32 = fm10k_data32_field_get(data32, 0, 14);
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ data32 = fm10k_read_switch_reg(sw,
+ FM10K_SW_MOD_MCAST_VLAN_TABLE(tmp32 + 1));
+ FM10K_SW_INFO(" MOD_MCAST_VLAN_TABLE[%u] %#x VID %u "
+ "DGLORT %#x ReplaceVID %u ReplaceDGLORT %u\n",
+ tmp32 + 1, data32,
+ fm10k_data32_field_get(data32, 0, 11),
+ fm10k_data32_field_get(data32, 12, 27),
+ fm10k_data32_field_get(data32, 28, 28),
+ fm10k_data32_field_get(data32, 29, 29));
+ }
+}
+
+static void
+fm10k_ffu_register_dump(struct fm10k_switch *sw)
+{
+ int i, j;
+ uint32_t data[8];
+ uint32_t ffu_valid;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_FFU_REG))
+ return;
+
+ FM10K_SW_INFO("--------------- FFU REGISTERS DUMP -----------------\n");
+
+ fm10k_read_switch_array(sw, FM10K_SW_FFU_MASTER_VALID, data, 2);
+ FM10K_SW_INFO("FFU_MASTER_VALID: %#x %#x\n", data[0], data[1]);
+ ffu_valid = data[0];
+
+ for (i = 0; i < 32; i++) {
+ if ((ffu_valid & (1 << i)) == 0)
+ continue;
+
+ FM10K_SW_INFO("------ SLICE%d ------\n", i);
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_VALID(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_VALID[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CASCADE_ACTION[%d]: %#x %#x\n",
+ i, data[0], data[1]);
+
+ for (j = 0; j < 1; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_CFG(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_CFG[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ for (j = 0; j < 32; j++) {
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_TCAM(i, j), data, 4);
+ if ((data[0] != 0 || data[1] != 0 ||
+ data[2] != 0 || data[3] != 0))
+ FM10K_SW_INFO("FFU_SLICE_TCAM[%d][%d]: "
+ "%#x %#x %#x %#x\n",
+ i, j, data[0], data[1],
+ data[2], data[3]);
+
+ fm10k_read_switch_array(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, j), data, 2);
+ if (data[0] != 0 || data[1] != 0)
+ FM10K_SW_INFO("FFU_SLICE_SRAM[%d][%d]: %#x %#x\n",
+ i, j, data[0], data[1]);
+ }
+ }
+}
+
+
+static void
+fm10k_ffu_mirror_set_action(struct fm10k_switch *sw,
+ uint16_t ffu_slice, uint16_t table_idx)
+{
+ uint64_t data64;
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice, table_idx), 0xffffffffff, 0x1);
+ /* SET FFU RX_MIRROR */
+ data64 = 0x60; /* higher than route, not necessary */
+ data64 = data64 << 32 | 0x2 << 21 | 1 << (8 + 4) | 1 << 4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), data64);
+}
+
+static void
+fm10k_ffu_mirror_set_forward(struct fm10k_switch *sw,
+ int profile,
+ uint16_t vlan,
+ uint16_t dest_lport,
+ uint32_t dest_sglort)
+{
+ uint64_t data64;
+
+ /* RX_MIRROR_CFG */
+ fm10k_write_switch_reg(sw, 0xd50000 + 0xd, profile);
+
+ /* FH_MIRROR_PROFILE_TABLE */
+ fm10k_write_switch_reg(sw,
+ 0xd50000 + 0x1 * profile + 0x40, dest_lport);
+
+ /* MOD_MIRROR_PROFILE_TABLE don't work */
+ data64 = vlan;
+ data64 = (dest_sglort & 0xffff) | data64 << 17;
+ fm10k_write_switch_reg64(sw,
+ 0xe80000 + 0x2 * profile + 0x24000, data64);
+}
+
+
+int
+fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_ext_port,
+ uint16_t dest_ext_port,
+ uint16_t vlan)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+ uint16_t mirror_profile = FM10K_FFU_MIRROR_PROFILE;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_ffu_mirror_set_action(sw, ffu_slice, table_idx);
+
+ fm10k_ffu_mirror_set_forward(sw, mirror_profile,
+ vlan, sw->epl_map[dest_ext_port - 1].logical_port,
+ sw->epl_map[dest_ext_port - 1].glort);
+ return 0;
+}
+
+int
+fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_ext_port)
+{
+ uint16_t table_idx;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_SGLORT_VID + 1;
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(src_ext_port - 1);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(ffu_slice, table_idx), 0);
+ return 0;
+}
+
+static void
+fm10k_ffu_logical_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t lport, uint16_t vlan_id)
+{
+ uint64_t data;
+
+ /* 11.21.3.9 INGRESS_VID_TABLE[0..4095] */
+ data = fm10k_read_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000);
+ data |= 1 << lport;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * vlan_id + 0x20000, data);
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "%s lport:%d, vlan_id:%d, reg:%#llx\n",
+ __func__, lport, vlan_id, (unsigned long long)data);
+}
+
+static void
+fm10k_ffu_glort_port_vlan_set(struct fm10k_switch *sw,
+ uint16_t glort, uint16_t vlan_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ if (sw->pep_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->pep_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == glort) {
+ fm10k_ffu_logical_port_vlan_set(sw,
+ sw->epl_map[i].logical_port, vlan_id);
+ break;
+ }
+ }
+}
+
+
+static uint16_t
+fm10k_ffu_dpdk_port_glort_get(struct fm10k_switch *sw, int port)
+{
+ int pf;
+ uint16_t glort = 0;
+
+ if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFS ||
+ sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ glort = fm10k_switch_pfs_glort_get
+ (sw->dpdk_cfg->dpdk_port_map[port].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[port].map_no[1]);
+ } else if (sw->dpdk_cfg->dpdk_port_map[port].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf = sw->dpdk_cfg->dpdk_port_map[port].map_no[0];
+ glort = fm10k_switch_pf_glort_get(pf);
+ }
+ return glort;
+}
+
+static void
+fm10k_ffu_rule_enable_single_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint16_t dglort,
+ uint16_t dvlan)
+{
+ uint64_t temp64;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (dvlan) {
+ /* Force updating VLAN tag if present.
+ * Add if absent. 11.5.3.4
+ */
+ temp64 = 3;
+ sram[sram_idx] =
+ temp64 << 16 | dvlan |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 2) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_FIELD_SET);
+ fm10k_ffu_glort_port_vlan_set(sw, dglort, dvlan);
+ sram_idx++;
+ }
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+static void
+fm10k_ffu_rule_enable_multi_cast(struct fm10k_switch *sw,
+ int rule_id,
+ uint16_t sglort,
+ uint16_t svlan,
+ uint8_t lport1,
+ uint8_t lport2,
+ uint16_t vlan1,
+ uint16_t vlan2)
+{
+ uint64_t temp64;
+ uint32_t dglort;
+ uint64_t sglort_vid_tcam = 0, sglort_vid_tcam_mask = 0;
+ uint64_t sram[4] = { 0, 0, 0, 0 };
+ uint16_t sram_idx = 0, tcam_slice, sram_slice, i;
+
+ dglort =
+ fm10k_ffu_set_dest_glort_multi_cast(sw,
+ lport1, lport2, vlan1, vlan2);
+
+ sglort_vid_tcam |= sglort;
+ sglort_vid_tcam_mask |= 0xffff;
+ if (svlan) {
+ temp64 = svlan;
+ sglort_vid_tcam |= temp64 << 16;
+ sglort_vid_tcam_mask |= 0xfff0000;
+ fm10k_ffu_glort_port_vlan_set(sw, sglort, svlan);
+ }
+
+ fm10k_ffu_logical_port_vlan_set(sw, lport1, vlan1);
+ fm10k_ffu_logical_port_vlan_set(sw, lport2, vlan2);
+
+ /* set counter */
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_BANK,
+ FM10K_SW_FFU_CNT_BANK) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COUNTER_INDEX,
+ FM10K_FFU_CNT_INDEX(rule_id));
+
+ sram[sram_idx] |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_PRECEDENCE, 3) |
+ FM10K_SW_MAKE_REG_FIELD
+ (FFU_SLICE_SRAM_ROUTE_GLORT_DGLORT, dglort) |
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_SRAM_COMMAND,
+ FM10K_SW_FFU_SLICE_SRAM_COMMAND_ROUTE_GLORT);
+ sram_idx++;
+
+ if (sglort_vid_tcam) {
+ tcam_slice = FM10K_FFU_SLICE_SGLORT_VID;
+ temp64 =
+ FM10K_SW_MAKE_REG_FIELD64
+ (FFU_SLICE_TCAM_KEY,
+ ~sglort_vid_tcam & sglort_vid_tcam_mask);
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM(tcam_slice, rule_id),
+ temp64, sglort_vid_tcam);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "TCAM slice:%d, rule:%d, data0:%#llx, data1:%#llx\n",
+ tcam_slice, rule_id,
+ (unsigned long long)sglort_vid_tcam,
+ (unsigned long long)temp64);
+ }
+
+ /* blank the next SLICE TCAM */
+ fm10k_ffu_always_mismatch(sw, tcam_slice + 1, rule_id);
+
+ for (i = 0; i < sram_idx; i++) {
+ sram_slice = FM10K_FFU_SLICE_START + i;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice, rule_id), sram[i]);
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "SRAM slice:%d, rule:%d, data:%#llx\n",
+ sram_slice, rule_id, (unsigned long long)sram[i]);
+ }
+ /* disable the next SLICE SRAM */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(sram_slice + 1, rule_id), 0);
+
+ fm10k_stats_rule_count_reg(rule_id);
+}
+
+
+void
+fm10k_ffu_flow_enable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ uint16_t sglort = 0;
+ uint16_t svlan = 0;
+ uint16_t dglort = 0;
+ uint16_t dvlan = 0;
+ uint16_t lport1 = 0, lport2 = 0;
+ uint16_t dvlan2 = 0;
+
+ switch (flow->src_port.port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ sglort = fm10k_switch_epl_glort_get(flow->src_port.port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ sglort =
+ fm10k_ffu_dpdk_port_glort_get(sw,
+ flow->src_port.port_no);
+ break;
+ }
+ switch (flow->fw_port[0].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[0].port_no - 1);
+ lport1 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[0].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[0].port_no);
+ lport1 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[0].port_no);
+ break;
+ }
+ svlan = flow->src_port.vlan_id;
+ dvlan = flow->fw_port[0].vlan_id;
+
+ flow->rule_id = fm10k_ffu_rule_alloc();
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Set rule cond sglort:%#x, vlan:%d ==> "
+ "act dglort:%#x, vlan:%d",
+ sglort, svlan, dglort, dvlan);
+
+ if (flow->fw_port[1].port_type) {
+ switch (flow->fw_port[1].port_type) {
+ case FM10K_CONFIG_FLOW_EXT_PORT:
+ dglort =
+ fm10k_switch_epl_glort_get
+ (flow->fw_port[1].port_no - 1);
+ lport2 =
+ fm10k_switch_epl_logical_get
+ (flow->fw_port[1].port_no - 1);
+ break;
+ case FM10K_CONFIG_FLOW_DPDK_PORT:
+ dglort =
+ fm10k_ffu_dpdk_port_glort_get
+ (sw, flow->fw_port[1].port_no);
+ lport2 =
+ fm10k_switch_pf_logical_get
+ (flow->fw_port[1].port_no);
+ break;
+ }
+ dvlan2 = flow->fw_port[1].vlan_id;
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ " (bypass glort:%#x, vlan:%d)\n",
+ dglort, dvlan2);
+
+ fm10k_ffu_rule_enable_multi_cast(sw,
+ flow->rule_id, sglort, svlan,
+ lport1, lport2, dvlan, dvlan2);
+ } else {
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg, "\n");
+ fm10k_ffu_rule_enable_single_cast(sw,
+ flow->rule_id, sglort, svlan, dglort, dvlan);
+ }
+}
+
+
+void
+fm10k_ffu_flow_disable(struct fm10k_switch *sw, struct fm10k_cfg_flow *flow)
+{
+ int i;
+
+ if (flow->rule_id == 0)
+ return;
+
+ FM10K_FFU_RULE_PRINT(sw->dpdk_cfg,
+ "Remove flow %d rule %d\n",
+ flow->flow_no, flow->rule_id);
+
+ for (i = FM10K_FFU_SLICE_START; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_ffu_always_mismatch(sw, i, flow->rule_id);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_SRAM(i, flow->rule_id), 0);
+ }
+ fm10k_ffu_rule_free(flow->rule_id);
+}
+
+
+static int
+fm10k_ffu_configured_flowset_enable(struct fm10k_switch *sw)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flow *flow;
+
+ flowset = fm10k_config_flowset_current_get();
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+
+ return 0;
+}
+
+
+void
+fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name)
+{
+ struct fm10k_cfg_flowset *flowset;
+ struct fm10k_cfg_flowset *cur_fs;
+ struct fm10k_cfg_flow *flow;
+
+ cur_fs = fm10k_config_flowset_current_get();
+ if (strcmp(cur_fs->name, new_name) == 0)
+ return;
+
+ flowset = fm10k_config_flowset_get(new_name);
+ if (flowset == NULL) {
+ FM10K_SW_ERR("Can not find flowset %s!!\n", new_name);
+ return;
+ }
+
+ /* disable current flowset */
+ flow = cur_fs->flow_head.next;
+ while (!fm10k_config_flow_list_end(&cur_fs->flow_head, flow)) {
+ fm10k_ffu_flow_disable(sw, flow);
+ flow = flow->next;
+ }
+
+ /* enable new flowset */
+ fm10k_config_flowset_current_set(flowset);
+ flow = flowset->flow_head.next;
+ while (!fm10k_config_flow_list_end(&flowset->flow_head, flow)) {
+ fm10k_ffu_flow_enable(sw, flow);
+ flow = flow->next;
+ }
+}
+
+
+int
+fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg)
+{
+ int ret = 0;
+ uint64_t data64;
+ uint16_t i, j;
+ uint32_t sglort = 0, dglort = 0;
+ uint16_t table_idx = 0;
+ uint16_t ffu_slice = FM10K_FFU_SLICE_START;
+
+ sw->mcast_dest_table_idx = 1;
+ sw->mcast_len_table_idx = 1;
+ sw->mcast_vlan_table_idx = 1;
+
+ for (i = 0; i < sizeof(fm10k_ffu_slice_cfgs) / sizeof(uint64_t); i++) {
+ for (j = 0; j < FM10K_SW_FFU_NUM_SCENARIOS; j++) {
+ data64 = fm10k_ffu_slice_cfgs[i];
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CFG
+ (FM10K_FFU_SLICE_START + i, j),
+ data64);
+ }
+ FM10K_FFU_INIT_PRINT(cfg, "SET slice %d cfg = %#llx\n",
+ FM10K_FFU_SLICE_START + i,
+ (unsigned long long)data64);
+ }
+
+ for (table_idx = 0; table_idx < FM10K_SW_FFU_SLICE_TCAM_ENTRIES / 2;
+ table_idx++)
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ fm10k_ffu_always_mismatch(sw, i, table_idx);
+
+ /*
+ * Create a TCAM entry to match each SGLORT that might be used, and
+ * set the corresponding SRAM action entries to ROUTE_GLORT to the
+ * corresponding DGLORT (see above table).
+ * SGLORT ROUTE is always the first slice
+ */
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ if (cfg->ext_port_map[i].type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pf_glort_get
+ (cfg->ext_port_map[i].map_no[0]);
+ } else if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ sglort = fm10k_switch_epl_glort_get(i);
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ }
+
+ table_idx = FM10K_FFU_EXT_PORT_RULE_INGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, sglort, dglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ table_idx = FM10K_FFU_EXT_PORT_RULE_EGRESS(i);
+ fm10k_ffu_route_dglort(sw,
+ ffu_slice, table_idx, dglort, sglort);
+
+ /* blank the next ffu slice */
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_FFU_SLICE_TCAM
+ (ffu_slice + 1, table_idx),
+ 0xffffffffff, 0x1);
+ }
+
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_CASCADE_ACTION(i),
+ 0xf0f000f);
+ /* Set slice 0 to valid for all scenarios */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_FFU_SLICE_VALID(i),
+ FM10K_SW_FFU_SLICE_VALID_ALL_SCENARIOS);
+ }
+
+ /* Mark slice 0 as valid and all chunks as invalid */
+ data64 = 0;
+ for (i = ffu_slice; i < FM10K_SW_FFU_NUM_SLICES; i++)
+ data64 |= FM10K_SW_FFU_MASTER_VALID_SLICE_VALID(i);
+ fm10k_write_switch_reg64(sw, FM10K_SW_FFU_MASTER_VALID, data64);
+
+ /*
+ * Set up the DGLORT map according to the desired
+ * SGLORT -> { DGLORT, logical_port } map.
+ */
+ for (i = 0; i < sw->info->num_peps; i++) {
+ if (sw->pep_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->pep_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ if (sw->epl_map[i].glort == 0)
+ continue;
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++,
+ sw->epl_map[i].glort,
+ FM10K_SW_DPORT_MASK
+ (sw->epl_map[i].logical_port));
+ }
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ uint64_t dport_mask = 0;
+
+ if (cfg->ext_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ dglort =
+ fm10k_switch_pfs_glort_get
+ (cfg->ext_port_map[i].map_no[0],
+ cfg->ext_port_map[i].map_no[1]);
+ dport_mask =
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[0]].logical_port) |
+ FM10K_SW_DPORT_MASK
+ (sw->pep_map[cfg->ext_port_map[i].map_no[1]].logical_port);
+ fm10k_ffu_set_dest_glort_mask(sw,
+ sw->glort_cam_ram_idx++, dglort, dport_mask);
+ }
+ }
+
+ /* Ensure the rest of the DGLORT TCAM won't match */
+ for (table_idx = sw->glort_cam_ram_idx;
+ table_idx < FM10K_SW_GLORT_CAM_ENTRIES;
+ table_idx++)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_GLORT_CAM(table_idx),
+ FM10K_SW_GLORT_CAM_MATCH_NONE);
+
+ ret = fm10k_ffu_configured_flowset_enable(sw);
+
+ fm10k_ffu_register_dump(sw);
+ fm10k_glort_register_dump(sw);
+ return ret;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_ffu.h b/drivers/net/fm10k/switch/fm10k_ffu.h
new file mode 100644
index 0000000..bcd9329
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_ffu.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_FFU_H_
+#define _FM10K_FFU_H_
+
+
+#include <stdint.h>
+
+#define FM10K_FFU_EXT_PORT_RULE_INGRESS(i) ((i) * 2)
+#define FM10K_FFU_EXT_PORT_RULE_EGRESS(i) ((i) * 2 + 1)
+#define FM10K_FFU_RULE_MAX FM10K_SW_FFU_RULE_MAX
+
+struct fm10k_switch;
+struct fm10k_dpdk_cfg;
+struct fm10k_cfg_flow;
+
+int fm10k_ffu_init(struct fm10k_switch *sw, struct fm10k_dpdk_cfg *cfg);
+
+int fm10k_ffu_mirror_set(struct fm10k_switch *sw,
+ uint16_t src_dpdk_port, u16 dest_dpdk_port, uint16_t vlan);
+int fm10k_ffu_mirror_reset(struct fm10k_switch *sw, int src_dpdk_port);
+
+void fm10k_ffu_flow_enable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flow_disable(struct fm10k_switch *sw,
+ struct fm10k_cfg_flow *flow);
+void fm10k_ffu_flowset_switch(struct fm10k_switch *sw, const char *new_name);
+
+#endif /* _FM10K_FFU_H */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
new file mode 100644
index 0000000..451813f
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -0,0 +1,1105 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_stats.h"
+#include "fm10k_switch.h"
+#include "fm10k_config.h"
+#include "fm10k_ffu.h"
+
+
+#define FM10K_MAX_VLAN_COUNTER 63
+#define FM10K_NB_RX_STATS_BANKS 6
+#define FM10K_BINS_PER_RX_STATS_BANK 16
+#define FM10K_WORDS_PER_RX_STATS_COUNTER 4
+
+/* Max expected entries in read stats scatter gather list */
+#define MAX_STATS_SGLIST 128
+#define FM10K_RX_STATS_BASE (0xE00000)
+#define FM10K_MOD_BASE (0xE80000)
+#define FM10K_CM_APPLY_BASE (0xD40000)
+#define FM10K_EPL_BASE (0x0E0000)
+
+#define FM10K_RX_STATS_BANK(index1, index0, word) \
+ ((0x001000) * ((index1) - 0) + \
+ (0x000004) * ((index0) - 0) + \
+ (word) + (0x000000) + \
+ (FM10K_RX_STATS_BASE))
+#define FM10K_MOD_STATS_BANK_FRAME(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x025000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_MOD_STATS_BANK_BYTE(index1, index0, word) \
+ ((0x000800) * ((index1) - 0) + \
+ (0x000002) * ((index0) - 0) + \
+ (word) + (0x026000) + \
+ (FM10K_MOD_BASE))
+#define FM10K_CM_APPLY_DROP_COUNT(index, word) \
+ ((0x000002) * ((index) - 0) + \
+ (word) + (0x000880) + \
+ (FM10K_CM_APPLY_BASE))
+#define FM10K_MAC_OVERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000021) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_JABBER_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000022) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERSIZE_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000023) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_RUNT_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000024) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_OVERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000025) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_UNDERRUN_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000026) + \
+ (FM10K_EPL_BASE))
+#define FM10K_MAC_CODE_ERROR_COUNTER(index1, index0) \
+ ((0x000400) * ((index1) - 0) + \
+ (0x000080) * ((index0) - 0) + \
+ (0x000027) + \
+ (FM10K_EPL_BASE))
+
+/* Rx Bank Definitions */
+#define FM10K_RX_STAT_BANK_TYPE 0
+#define FM10K_RX_STAT_BANK_SIZE 1
+#define FM10K_RX_STAT_BANK_PRI 2
+#define FM10K_RX_STAT_BANK_FWD_1 3
+#define FM10K_RX_STAT_BANK_FWD_2 4
+#define FM10K_RX_STAT_BANK_VLAN 5
+
+/* FM10K_RX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_RX_STAT_NONIP_L2UCAST 0
+#define FM10K_RX_STAT_NONIP_L2MCAST 1
+#define FM10K_RX_STAT_NONIP_L2BCAST 2
+#define FM10K_RX_STAT_IPV4_L2UCAST 3
+#define FM10K_RX_STAT_IPV4_L2MCAST 4
+#define FM10K_RX_STAT_IPV4_L2BCAST 5
+#define FM10K_RX_STAT_IPV6_L2UCAST 6
+#define FM10K_RX_STAT_IPV6_L2MCAST 7
+#define FM10K_RX_STAT_IPV6_L2BCAST 8
+#define FM10K_RX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_RX_STAT_CLASS_BASED_PAUSE 10
+#define FM10K_RX_STAT_FRAMING_ERR 11
+#define FM10K_RX_STAT_FCS_ERR 12
+
+/* FM10K_RX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_RX_STAT_LEN_LT_64 0
+#define FM10K_RX_STAT_LEN_EQ_64 1
+#define FM10K_RX_STAT_LEN_65_127 2
+#define FM10K_RX_STAT_LEN_128_255 3
+#define FM10K_RX_STAT_LEN_256_511 4
+#define FM10K_RX_STAT_LEN_512_1023 5
+#define FM10K_RX_STAT_LEN_1024_1522 6
+#define FM10K_RX_STAT_LEN_1523_2047 7
+#define FM10K_RX_STAT_LEN_2048_4095 8
+#define FM10K_RX_STAT_LEN_4096_8191 9
+#define FM10K_RX_STAT_LEN_8192_10239 10
+#define FM10K_RX_STAT_LEN_GE_10240 11
+
+/* FM10K_RX_STAT_BANK_PRI Bin Definitions */
+#define FM10K_RX_STAT_PRI_0 0
+#define FM10K_RX_STAT_PRI_1 1
+#define FM10K_RX_STAT_PRI_2 2
+#define FM10K_RX_STAT_PRI_3 3
+#define FM10K_RX_STAT_PRI_4 4
+#define FM10K_RX_STAT_PRI_5 5
+#define FM10K_RX_STAT_PRI_6 6
+#define FM10K_RX_STAT_PRI_7 7
+#define FM10K_RX_STAT_PRI_8 8
+#define FM10K_RX_STAT_PRI_9 9
+#define FM10K_RX_STAT_PRI_10 10
+#define FM10K_RX_STAT_PRI_11 11
+#define FM10K_RX_STAT_PRI_12 12
+#define FM10K_RX_STAT_PRI_13 13
+#define FM10K_RX_STAT_PRI_14 14
+#define FM10K_RX_STAT_PRI_15 15
+
+/* FM10K_RX_STAT_BANK_FWD_1 Bin Definitions */
+#define FM10K_RX_STAT_FID_FORWARDED 0
+#define FM10K_RX_STAT_FLOOD_FORWARDED 1
+#define FM10K_RX_STAT_SPECIALLY_HANDLED 2
+#define FM10K_RX_STAT_PARSER_ERROR_DROP 3
+#define FM10K_RX_STAT_ECC_ERROR_DROP 4
+#define FM10K_RX_STAT_TRAPPED 5
+#define FM10K_RX_STAT_PAUSE_DROPS 6
+#define FM10K_RX_STAT_STP_DROPS 7
+#define FM10K_RX_STAT_SECURITY_VIOLATIONS 8
+#define FM10K_RX_STAT_VLAN_TAG_DROPS 9
+#define FM10K_RX_STAT_VLAN_INGRESS_DROPS 10
+#define FM10K_RX_STAT_VLAN_EGRESS_DROPS 11
+#define FM10K_RX_STAT_GLORT_MISS_DROPS 12
+#define FM10K_RX_STAT_FFU_DROPS 13
+#define FM10K_RX_STAT_TRIGGER_DROPS 14
+#define FM10K_RX_STAT_OVERFLOW4 15
+
+/* FM10K_RX_STAT_BANK_FWD_2 Bin Definitions */
+#define FM10K_RX_STAT_POLICER_DROPS 0
+#define FM10K_RX_STAT_TTL_DROPS 1
+#define FM10K_RX_STAT_CM_PRIV_DROPS 2
+#define FM10K_RX_STAT_CM_SMP0_DROPS 3
+#define FM10K_RX_STAT_CM_SMP1_DROPS 4
+#define FM10K_RX_STAT_CM_RX_HOG_0_DROPS 5
+#define FM10K_RX_STAT_CM_RX_HOG_1_DROPS 6
+#define FM10K_RX_STAT_CM_TX_HOG_0_DROPS 7
+#define FM10K_RX_STAT_CM_TX_HOG_1_DROPS 8
+/* Bin 9 reserved */
+#define FM10K_RX_STAT_TRIGGER_REDIRECTS 10
+#define FM10K_RX_STAT_FLOOD_CONTROL_DROPS 11
+#define FM10K_RX_STAT_GLORT_FORWARDED 12
+#define FM10K_RX_STAT_LOOPBACK_SUPPRESS 13
+#define FM10K_RX_STAT_OTHERS 14
+#define FM10K_RX_STAT_OVERFLOW5 15
+
+/* FM10K_RX_STAT_BANK_VLAN Bin definitions */
+#define FM10K_RX_STAT_VLAN_UCAST 0
+#define FM10K_RX_STAT_VLAN_MCAST 1
+#define FM10K_RX_STAT_VLAN_BCAST 2
+
+/* Tx Bank Definitions */
+#define FM10K_TX_STAT_BANK_TYPE 0
+#define FM10K_TX_STAT_BANK_SIZE 1
+
+/* FM10K_TX_STAT_BANK_TYPE Bin Definitions */
+#define FM10K_TX_STAT_L2UCAST 0
+#define FM10K_TX_STAT_L2MCAST 1
+#define FM10K_TX_STAT_L2BCAST 2
+#define FM10K_TX_STAT_ERR_SENT 3
+#define FM10K_TX_STAT_TIMEOUT_DROP 4
+#define FM10K_TX_STAT_ERR_DROP 5
+#define FM10K_TX_STAT_ECC_DROP 6
+#define FM10K_TX_STAT_LOOPBACK_DROP 7
+#define FM10K_TX_STAT_TTL1_DROP 8
+#define FM10K_TX_STAT_IEEE802_3_PAUSE 9
+#define FM10K_TX_STAT_CLASS_BASED_PAUSE 10
+
+/* FM10K_TX_STAT_BANK_SIZE Bin Definitions */
+#define FM10K_TX_STAT_LEN_LT_64 0
+#define FM10K_TX_STAT_LEN_EQ_64 1
+#define FM10K_TX_STAT_LEN_65_127 2
+#define FM10K_TX_STAT_LEN_128_255 3
+#define FM10K_TX_STAT_LEN_256_511 4
+#define FM10K_TX_STAT_LEN_512_1023 5
+#define FM10K_TX_STAT_LEN_1024_1522 6
+#define FM10K_TX_STAT_LEN_1523_2047 7
+#define FM10K_TX_STAT_LEN_2048_4095 8
+#define FM10K_TX_STAT_LEN_4096_8191 9
+#define FM10K_TX_STAT_LEN_8192_10239 10
+#define FM10K_TX_STAT_LEN_GE_10240 11
+
+
+/* This structure is used to build a table of all rx / tx counters */
+struct fm10k_port_count_mapping {
+ /* Bank in which the counter is located */
+ uint32_t bank;
+
+ /* Bin in which the counter is located */
+ uint32_t bin;
+
+ /*
+ * Offset (in bytes) of a the frame counter in
+ * the fm10k_counters structure
+ */
+ uint32_t frame_offset;
+
+ /*
+ * Offset (in bytes) of a the byte counter in
+ * the fm10k_counters structure
+ */
+ uint32_t byte_offset;
+};
+
+
+/*
+ * Table of all RX port counters in the RX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping rx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * frame_offset
+ * byte_offset
+ */
+ /* Type Bank */
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_NONIP_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_nonip),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_nonip)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV4_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv4),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv4)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_ucst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_mcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IPV6_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_pkts_ipv6),
+ offsetof(struct fm10k_port_counters, cnt_rx_bcst_octets_ipv6)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_pause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_cbpause_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FRAMING_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_framing_error_octets)},
+{ FM10K_RX_STAT_BANK_TYPE, FM10K_RX_STAT_FCS_ERR,
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors),
+ offsetof(struct fm10k_port_counters, cnt_rx_fcs_errors_octets)},
+
+ /* Size Bank */
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_minto63_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_rx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_64_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_65to127_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_128to255_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_256to511_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_512to1023_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1024to1522_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_1523to2047_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_2048to4095_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_4096to8191_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_8192to10239_octets)},
+{ FM10K_RX_STAT_BANK_SIZE, FM10K_RX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_10240tomax_octets)},
+
+ /* priority Bank */
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_0,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[0]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[0])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_1,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[1]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[1])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_2,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[2]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[2])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_3,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[3]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[3])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_4,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[4]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[4])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_5,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[5]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[5])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_6,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[6]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[6])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_7,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[7]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[7])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_8,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[8]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[8])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_9,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[9]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[9])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_10,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[10]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[10])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_11,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[11]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[11])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_12,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[12]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[12])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_13,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[13]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[13])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_14,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[14]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[14])},
+{ FM10K_RX_STAT_BANK_PRI, FM10K_RX_STAT_PRI_15,
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_pkts[15]),
+ offsetof(struct fm10k_port_counters, cnt_rx_priority_octets[15])},
+
+ /* Forwarding Bank */
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FID_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_fid_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FLOOD_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SPECIALLY_HANDLED,
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_pkts),
+ offsetof(struct fm10k_port_counters, cnt_specially_handled_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PARSER_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parse_err_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_ECC_ERROR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_parity_error_pkts),
+ offsetof(struct fm10k_port_counters, cnt_parity_error_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRAPPED,
+ offsetof(struct fm10k_port_counters, cnt_trapped_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trapped_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_PAUSE_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_pause_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_STP_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_stp_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_SECURITY_VIOLATIONS,
+ offsetof(struct fm10k_port_counters, cnt_security_violation_pkts),
+ offsetof(struct fm10k_port_counters, cnt_security_violation_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_TAG_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_tag_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_INGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_ingressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_VLAN_EGRESS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_pkts),
+ offsetof(struct fm10k_port_counters, cnt_vlan_egressbv_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_GLORT_MISS_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_miss_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_FFU_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ffu_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_1, FM10K_RX_STAT_TRIGGER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_POLICER_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_policer_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TTL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_ttl_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_PRIV_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_cmpriv_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_SMP1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_smp1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_RX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_rx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_0_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog0_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_CM_TX_HOG_1_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_hog1_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_TRIGGER_REDIRECTS,
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_pkts),
+ offsetof(struct fm10k_port_counters, cnt_trigger_redir_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_FLOOD_CONTROL_DROPS,
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_flood_control_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_GLORT_FORWARDED,
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_pkts),
+ offsetof(struct fm10k_port_counters, cnt_glort_forwarded_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_LOOPBACK_SUPPRESS,
+ offsetof(struct fm10k_port_counters, cnt_loopback_drops_pkts),
+ offsetof(struct fm10k_port_counters, cnt_loopback_drop_octets)},
+{ FM10K_RX_STAT_BANK_FWD_2, FM10K_RX_STAT_OTHERS,
+ offsetof(struct fm10k_port_counters, cnt_other_pkts),
+ offsetof(struct fm10k_port_counters, cnt_other_octets)},
+
+}; /* end rxPortCntMapTable */
+
+
+/*
+ * Table of all TX port counters in the TX banks
+ * This table is used for retrieving counters as
+ * well as resetting them
+ */
+static struct fm10k_port_count_mapping tx_port_cnt_map_table[] = {
+ /* Bank Bin
+ * frame_offset
+ * byte_offset
+ */
+ /* Type Bank */
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2UCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ucst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2MCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_mcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_L2BCAST,
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_bcst_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_SENT,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_sent_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TIMEOUT_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_timeout_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ERR_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_error_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_error_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_ECC_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_unrepair_ecc_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_LOOPBACK_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_loopback_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_TTL1_DROP,
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_ttl_drop_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_IEEE802_3_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_pause_octets)},
+{ FM10K_TX_STAT_BANK_TYPE, FM10K_TX_STAT_CLASS_BASED_PAUSE,
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_cbpause_octets)},
+
+ /* Size Bank */
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_LT_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_minto63_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_EQ_64,
+ offsetof(struct fm10k_port_counters, cnt_tx_64_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_64_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_65_127,
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_65to127_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_128_255,
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_128to255_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_256_511,
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_256to511_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_512_1023,
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_512to1023_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1024_1522,
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1024to1522_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_1523_2047,
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_1523to2047_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_2048_4095,
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_2048to4095_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_4096_8191,
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_4096to8191_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_8192_10239,
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_8192to10239_octets)},
+{ FM10K_TX_STAT_BANK_SIZE, FM10K_TX_STAT_LEN_GE_10240,
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_pkts),
+ offsetof(struct fm10k_port_counters, cnt_tx_10240tomax_octets)},
+
+}; /* end tx_port_cnt_map_table */
+
+
+/*
+ * Get a 32bit EPL counter by adding it to the scatter gather
+ * list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+static inline void
+fm10k_get_epl_port_stat32(struct fm10k_scatter_gather_entry *sg_list,
+ struct fm10k_port_counters *counters,
+ int *sg_list_cnt, int epl, int lane)
+{
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_oversized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_JABBER_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_jabber_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_UNDERSIZE_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_undersized_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_RUNT_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_fragment_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_OVERRUN_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_over_run_pkts;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+
+ sg_list[*sg_list_cnt].addr = FM10K_MAC_CODE_ERROR_COUNTER(epl, lane);
+ sg_list[*sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_code_errors;
+ sg_list[*sg_list_cnt].count = 1;
+ (*sg_list_cnt)++;
+}
+
+
+static void
+fm10k_read_scatter_gather(struct fm10k_switch *sw,
+ int n_entries,
+ struct fm10k_scatter_gather_entry *sg_list)
+{
+ int i;
+
+ for (i = 0; i < n_entries; i++) {
+ uint32_t addr = sg_list[i].addr;
+ uint32_t count = sg_list[i].count;
+ uint32_t *data = sg_list[i].data;
+
+ fm10k_read_switch_array(sw, addr, data, count);
+ }
+}
+
+
+static int
+fm10k_get_port_counters(struct fm10k_switch *sw,
+ struct fm10k_ext_port *ext_port,
+ struct fm10k_port_counters *counters)
+{
+ int phys_port;
+ int epl;
+ int lane;
+ struct timeval ts;
+ bool valid_ip_stats = true;
+ uint32_t i;
+ struct fm10k_scatter_gather_entry sg_list[MAX_STATS_SGLIST];
+ int sg_list_cnt = 0;
+ /* Temporary bin array to retrieve 128bit
+ * port counters (frame + bytes).
+ */
+ uint32_t cnt_rx_port_stats_bank[FM10K_NB_RX_STATS_BANKS]
+ [FM10K_BINS_PER_RX_STATS_BANK]
+ [FM10K_WORDS_PER_RX_STATS_COUNTER];
+ /*
+ * Fill the counters structure with 0 to assure
+ * all fields not explicitly set below, which will
+ * be the fields not supported by the FM10000,
+ * will be 0 on return.
+ */
+ memset(counters, 0, sizeof(*counters));
+
+ phys_port = ext_port->portno;
+ epl = ext_port->eplno;
+ lane = ext_port->first_lane;
+
+ counters->cnt_version = FM10K_STATS_VERSION;
+
+ /*
+ * Reading counters for each RX bank
+ *
+ * Because the RX_STATS_BANK register contains
+ * both, frame count and byte count in a 128bit
+ * register, we first store the data in a temporary
+ * array.
+ * 1. Fill Scatter Gather to retrieve each bin
+ * counter from the rx bank and store in a temp
+ * array.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 128bit read of an rx counter to the scatter gather
+ * list.
+ * NOTE: Read values will be stored in the temporary
+ * array switchExt->cnt_rx_port_stats_bank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [out] - Array to store the
+ * 128bit values sgList
+ * [out] - scatter gather array to hold the reads sgListCnt
+ * [out] - number of entry in the sgList
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_RX_STATS_BANK(bank, (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data = cnt_rx_port_stats_bank[bank][bin];
+ sg_list[sg_list_cnt].count = 4;
+ sg_list_cnt++;
+}
+
+ /*
+ * 2. Fill in the scatter gather for tx bank
+ * counters. They will directly be stored
+ * in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Add a 64bit read of a tx counter to the scatter gather list.
+ * Depends on the existence of the variables:
+ * sgList [out] - scatter gather array to hold the reads
+ * sgListCnt [out] - number of entry in the sgList
+ */
+ uint32_t bank = tx_port_cnt_map_table[i].bank;
+ uint32_t bin = tx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = tx_port_cnt_map_table[i].frame_offset;
+
+ sg_list[sg_list_cnt].addr =
+ FM10K_MOD_STATS_BANK_FRAME(bank,
+ (phys_port << 4 | (bin)), 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)(((uint8_t *)counters) + (frame_offset));
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ }
+
+ /*
+ * 3. Fill in the scatter gather for TX CM DROP
+ * and for EPL counters. They will directly be
+ * stored in the fm10k_counters structure upon
+ * scatter gather read.
+ */
+ sg_list[sg_list_cnt].addr =
+ FM10K_CM_APPLY_DROP_COUNT(phys_port, 0);
+ sg_list[sg_list_cnt].data =
+ (uint32_t *)&counters->cnt_rx_cm_drop_pkts;
+ sg_list[sg_list_cnt].count = 2;
+ sg_list_cnt++;
+ /* EPL Counters */
+ fm10k_get_epl_port_stat32(sg_list, counters, &sg_list_cnt, epl, lane);
+
+ /*
+ * 4. Execute scatter gather read.
+ */
+ if (sg_list_cnt >= MAX_STATS_SGLIST) {
+ /*
+ * Pretty static. Mainly to warn if something new added,
+ * but the array size is not adjust accordingly
+ */
+ FM10K_SW_ERR("Scatter list array %d for port %d overflow.\n",
+ sg_list_cnt, phys_port);
+ return -1;
+ }
+
+ /*
+ * Taking lock to protect temporary structures used to
+ * store 128b counters
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ /* now get the stats in one shot, optimized for fibm */
+ fm10k_read_scatter_gather(sw, sg_list_cnt, sg_list);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ counters->timestamp = ts.tv_sec * 1000000 + ts.tv_usec;
+
+ /*
+ * 5. Retrieve the frame/byte counts from 128bit
+ * registers stored in temporary array and set
+ * the proper fm_portCounter structure members.
+ */
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ /*
+ * Update the counter variable related to a 128bit HW counter
+ * NOTE: Read values will be retrieved from the temporary
+ * array switchExt->cnt_rx_PortStatsBank;
+ * Depends on the existence of the variables:
+ * switchExt->cnt_rx_PortStatsBank [in] - Array to read the
+ * 128bit values from
+ */
+ uint32_t bank = rx_port_cnt_map_table[i].bank;
+ uint32_t bin = rx_port_cnt_map_table[i].bin;
+ uint32_t frame_offset = rx_port_cnt_map_table[i].frame_offset;
+ uint32_t byte_offset = rx_port_cnt_map_table[i].byte_offset;
+
+ *((uint64_t *)(((uint8_t *)counters) + frame_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][1]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][0])));
+
+ *((uint64_t *)(((uint8_t *)counters) + byte_offset)) =
+ (((uint64_t)(cnt_rx_port_stats_bank[bank][bin][3]) << 32) |
+ ((uint64_t)(cnt_rx_port_stats_bank[bank][bin][2])));
+ }
+
+ /*
+ * 6. Set some counters that are not available
+ * in HW but can be computed from two or more
+ * HW counters.
+ */
+ /*
+ * If IP parsing is enabled then the IP stats will be read and will be
+ * valid. Else all traffic, whether IP or not, is counted as _nonip.
+ */
+ if (valid_ip_stats) {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip +
+ counters->cnt_rx_ucst_pkts_ipv4 +
+ counters->cnt_rx_ucst_pkts_ipv6;
+
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip +
+ counters->cnt_rx_mcst_pkts_ipv4 +
+ counters->cnt_rx_mcst_pkts_ipv6;
+
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip +
+ counters->cnt_rx_bcst_pkts_ipv4 +
+ counters->cnt_rx_bcst_pkts_ipv6;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_octets_ipv4 =
+ counters->cnt_rx_ucst_octets_ipv4 +
+ counters->cnt_rx_mcst_octets_ipv4 +
+ counters->cnt_rx_bcst_octets_ipv4;
+
+ counters->cnt_rx_octets_ipv6 =
+ counters->cnt_rx_ucst_octets_ipv6 +
+ counters->cnt_rx_mcst_octets_ipv6 +
+ counters->cnt_rx_bcst_octets_ipv6;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip +
+ counters->cnt_rx_octets_ipv4 +
+ counters->cnt_rx_octets_ipv6;
+ } else {
+ /*
+ * RX counters.
+ */
+ counters->cnt_rx_ucst_pkts =
+ counters->cnt_rx_ucst_pkts_nonip;
+ counters->cnt_rx_mcst_pkts =
+ counters->cnt_rx_mcst_pkts_nonip;
+ counters->cnt_rx_bcst_pkts =
+ counters->cnt_rx_bcst_pkts_nonip;
+
+ /*
+ * Misc. counters.
+ */
+ counters->cnt_rx_octets_nonip =
+ counters->cnt_rx_ucst_octets_nonip +
+ counters->cnt_rx_mcst_octets_nonip +
+ counters->cnt_rx_bcst_octets_nonip;
+
+ counters->cnt_rx_good_octets =
+ counters->cnt_rx_octets_nonip;
+ }
+
+ counters->cnt_trigger_drop_redir_pkts =
+ counters->cnt_trigger_redir_pkts +
+ counters->cnt_trigger_drop_pkts;
+
+ /* Emulate Tx _octets counter using the sum of all size bins. */
+ counters->cnt_tx_octets += counters->cnt_tx_minto63_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_64_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_65to127_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_128to255_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_256to511_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_512to1023_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1024to1522_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_1523to2047_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_2048to4095_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_4096to8191_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_8192to10239_octets;
+ counters->cnt_tx_octets += counters->cnt_tx_10240tomax_octets;
+
+ return 0;
+} /* end fm10k_get_port_counters */
+
+
+struct fm10k_stats_data {
+ uint64_t rx_pkts;
+ uint64_t tx_pkts;
+ uint64_t rx_drop;
+};
+
+
+static uint64_t
+fm10k_stats_ffu_count_get(struct fm10k_switch *sw, int table_id)
+{
+ uint64_t data;
+
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x4000 * FM10K_SW_FFU_CNT_BANK +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x8000);
+ } else {
+ data = fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x800 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x4 * (table_id + FM10K_SW_FFU_CNT_START) +
+ 0x10000);
+ }
+ return data;
+}
+
+static void
+fm10k_stats_epl_ffu_print(struct fm10k_switch *sw, int epl)
+{
+ int table_id;
+ uint64_t data;
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_INGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] ingress %-12llu\n",
+ table_id, (unsigned long long)(data));
+
+ table_id = FM10K_FFU_EXT_PORT_RULE_EGRESS(epl);
+ data = fm10k_stats_ffu_count_get(sw, table_id);
+ FM10K_SW_INFO(" FFU[%d] egress %-12llu\n",
+ table_id, (unsigned long long)(data));
+}
+
+#define FM10K_STATS_RULE_NUM_MAX 100
+static uint16_t fm10k_stats_rule_list[FM10K_STATS_RULE_NUM_MAX];
+void
+fm10k_stats_rule_count_reg(uint16_t rule_id)
+{
+ int i;
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0) {
+ fm10k_stats_rule_list[i] = rule_id;
+ break;
+ }
+ }
+}
+
+void
+fm10k_stats_ffu_count_print(struct fm10k_switch *sw)
+{
+ int i, rule_id;
+ uint64_t data;
+ static uint64_t ffu_count[FM10K_STATS_RULE_NUM_MAX];
+
+ for (i = 0; i < FM10K_STATS_RULE_NUM_MAX; i++) {
+ if (fm10k_stats_rule_list[i] == 0)
+ continue;
+
+ rule_id = fm10k_stats_rule_list[i];
+ data = fm10k_stats_ffu_count_get(sw, rule_id);
+ FM10K_SW_INFO("FFU[%d] count %-12llu\n",
+ rule_id,
+ (unsigned long long)
+ (data - ffu_count[rule_id]));
+ ffu_count[rule_id] = data;
+ }
+}
+
+void
+fm10k_stats_epl_port_print(struct fm10k_switch *sw)
+{
+ int i, lport;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_EXT_PORTS_MAX];
+ struct fm10k_stats_data data;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ lport = sw->epl_map[i].logical_port;
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ data.rx_pkts =
+ counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts =
+ counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ FM10K_SW_INFO("EPL port %-2d lport %-5d tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, lport,
+ (unsigned long long)
+ (data.tx_pkts - last_data[i].tx_pkts),
+ (unsigned long long)
+ (data.rx_pkts - last_data[i].rx_pkts),
+ (unsigned long long)
+ (data.rx_drop - last_data[i].rx_drop));
+ last_data[i] = data;
+
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU))
+ fm10k_stats_epl_ffu_print(sw, i);
+ }
+}
+
+
+static void
+fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
+{
+ unsigned int i;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ uint64_t *pdata;
+
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+
+ for (i = 0;
+ i < sizeof(rx_port_cnt_map_table) /
+ sizeof(rx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ rx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d rx bank %d idx %d pkt %llu\n",
+ lport,
+ rx_port_cnt_map_table[i].bank,
+ rx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+ for (i = 0;
+ i < sizeof(tx_port_cnt_map_table) /
+ sizeof(tx_port_cnt_map_table[0]);
+ i++) {
+ pdata =
+ (uint64_t *)((uint8_t *)(&counters) +
+ tx_port_cnt_map_table[i].frame_offset);
+ if (*pdata != 0) {
+ FM10K_SW_INFO("port %d tx bank %d idx %d pkt %llu\n",
+ lport,
+ tx_port_cnt_map_table[i].bank,
+ tx_port_cnt_map_table[i].bin,
+ (unsigned long long)*pdata);
+ }
+ }
+}
+
+void
+fm10k_stats_port_bank_print(struct fm10k_switch *sw)
+{
+ int i;
+
+ for (i = 0;
+ i < FM10K_SW_EPLS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->epl_map[i].logical_port);
+ }
+ for (i = 0;
+ i < FM10K_SW_PEPS_SUPPORTED;
+ i++) {
+ fm10k_stats_port_counter_print(sw,
+ sw->pep_map[i].logical_port);
+ }
+}
+
+
+void *
+fm10k_switch_process_stats(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ usec_delay(5000000);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_ENABLE))
+ return NULL;
+
+ if (!sw->dpdk_cfg->stats_interval)
+ return NULL;
+
+ while (sw->detaching == 0) {
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_PORT)) {
+ FM10K_SW_INFO("--- port statistic ---\n");
+ fm10k_stats_epl_port_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_FFU)) {
+ FM10K_SW_INFO("--- ffu statistic ---\n");
+ fm10k_stats_ffu_count_print(sw);
+ }
+ if (fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_MORE)) {
+ FM10K_SW_INFO("--- detail statistic ---\n");
+ fm10k_stats_port_bank_print(sw);
+ }
+ usec_delay(1000000 * sw->dpdk_cfg->stats_interval);
+ }
+ return NULL;
+}
diff --git a/drivers/net/fm10k/switch/fm10k_stats.h b/drivers/net/fm10k/switch/fm10k_stats.h
new file mode 100644
index 0000000..0bc84e4
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_stats.h
@@ -0,0 +1,257 @@
+/* spdx-license-identifier: bsd-3-clause
+ * copyright 2019 silicom ltd. connectivity solutions
+ */
+
+#ifndef _fm10k_stats_h_
+#define _fm10k_stats_h_
+
+
+#include <stdint.h>
+
+
+#define FM10K_STATS_VERSION (1 << 4)
+
+
+struct fm10k_scatter_gather_entry {
+ uint32_t addr;
+ uint32_t count;
+ uint32_t *data;
+
+};
+
+
+struct fm10k_port_counters {
+ uint64_t cnt_version;
+ uint64_t cnt_rx_ucst_pkts;
+ uint64_t cnt_rx_ucst_pkts_nonip;
+ uint64_t cnt_rx_ucst_pkts_ipv4;
+ uint64_t cnt_rx_ucst_pkts_ipv6;
+ uint64_t cnt_rx_bcst_pkts;
+ uint64_t cnt_rx_bcst_pkts_nonip;
+ uint64_t cnt_rx_bcst_pkts_ipv4;
+ uint64_t cnt_rx_bcst_pkts_ipv6;
+ uint64_t cnt_rx_mcst_pkts;
+ uint64_t cnt_rx_mcst_pkts_nonip;
+ uint64_t cnt_rx_mcst_pkts_ipv4;
+ uint64_t cnt_rx_mcst_pkts_ipv6;
+ uint64_t cnt_rx_pause_pkts;
+ uint64_t cnt_rx_cbpause_pkts;
+ uint64_t cnt_rx_fcs_errors;
+ uint64_t cnt_rx_symbol_errors;
+ uint64_t cnt_rx_framesize_errors;
+ uint64_t cnt_rx_framing_error_pkts;
+ uint64_t cnt_rx_minto63_pkts;
+ uint64_t cnt_rx_64_pkts;
+ uint64_t cnt_rx_65to127_pkts;
+ uint64_t cnt_rx_128to255_pkts;
+ uint64_t cnt_rx_256to511_pkts;
+ uint64_t cnt_rx_512to1023_pkts;
+ uint64_t cnt_rx_1024to1522_pkts;
+ uint64_t cnt_rx_1523to2047_pkts;
+ uint64_t cnt_rx_2048to4095_pkts;
+ uint64_t cnt_rx_4096to8191_pkts;
+ uint64_t cnt_rx_8192to10239_pkts;
+ uint64_t cnt_rx_10240tomax_pkts;
+ uint64_t cnt_rx_minto63_octets;
+ uint64_t cnt_rx_64_octets;
+ uint64_t cnt_rx_65to127_octets;
+ uint64_t cnt_rx_128to255_octets;
+ uint64_t cnt_rx_256to511_octets;
+ uint64_t cnt_rx_512to1023_octets;
+ uint64_t cnt_rx_1024to1522_octets;
+ uint64_t cnt_rx_1523to2047_octets;
+ uint64_t cnt_rx_2048to4095_octets;
+ uint64_t cnt_rx_4096to8191_octets;
+ uint64_t cnt_rx_8192to10239_octets;
+ uint64_t cnt_rx_10240tomax_octets;
+ uint64_t cnt_rx_octets_nonip;
+ uint64_t cnt_rx_octets_ipv4;
+ uint64_t cnt_rx_octets_ipv6;
+ uint64_t cnt_rx_ucst_octets_nonip;
+ uint64_t cnt_rx_ucst_octets_ipv4;
+ uint64_t cnt_rx_ucst_octets_ipv6;
+ uint64_t cnt_rx_bcst_octets_nonip;
+ uint64_t cnt_rx_bcst_octets_ipv4;
+ uint64_t cnt_rx_bcst_octets_ipv6;
+ uint64_t cnt_rx_mcst_octets_nonip;
+ uint64_t cnt_rx_mcst_octets_ipv4;
+ uint64_t cnt_rx_mcst_octets_ipv6;
+ uint64_t cnt_rx_pause_octets;
+ uint64_t cnt_rx_cbpause_octets;
+ uint64_t cnt_rx_fcs_errors_octets;
+ uint64_t cnt_rx_framing_error_octets;
+ uint64_t cnt_rx_good_octets;
+ uint64_t cnt_rx_bad_octets;
+ uint64_t cnt_rx_priority_pkts[16];
+ uint64_t cnt_rx_invalid_priority_pkts;
+ uint64_t cnt_rx_priority_octets[16];
+ uint64_t cnt_rx_invalid_priority_octets;
+ uint64_t cnt_fid_forwarded_pkts;
+ uint64_t cnt_flood_forwarded_pkts;
+ uint64_t cnt_glort_switched_pkts;
+ uint64_t cnt_glort_routed_pkts;
+ uint64_t cnt_specially_handled_pkts;
+ uint64_t cnt_parse_err_drop_pkts;
+ uint64_t cnt_parity_error_pkts;
+ uint64_t cnt_trapped_pkts;
+ uint64_t cnt_pause_drop_pkts;
+ uint64_t cnt_stp_drop_pkts;
+ uint64_t cnt_stp_ingress_drops_pkts;
+ uint64_t cnt_stp_egress_drops_pkts;
+ uint64_t cnt_reserved_trap_pkts;
+ uint64_t cnt_security_violation_pkts;
+ uint64_t cnt_vlan_tag_drop_pkts;
+ uint64_t cnt_vlan_ingressbv_pkts;
+ uint64_t cnt_vlan_egressbv_pkts;
+ uint64_t cnt_loopback_drops_pkts;
+ uint64_t cnt_glort_miss_drop_pkts;
+ uint64_t cnt_ffu_drop_pkts;
+ uint64_t cnt_invalid_drop_pkts;
+ uint64_t cnt_policer_drop_pkts;
+ uint64_t cnt_ttl_drop_pkts;
+ uint64_t cnt_global_wmdrop_pkts;
+ uint64_t cnt_rx_mpdrop_pkts;
+ uint64_t cnt_rx_hogdrop_pkts;
+ uint64_t cnt_tx_hogdrop_pkts;
+ uint64_t cnt_other_pkts;
+ uint64_t cnt_flood_control_drop_pkts;
+ uint64_t cnt_cmpriv_drop_pkts;
+ uint64_t cnt_smp0_drop_pkts;
+ uint64_t cnt_smp1_drop_pkts;
+ uint64_t cnt_rx_hog0_drop_pkts;
+ uint64_t cnt_rx_hog1_drop_pkts;
+ uint64_t cnt_tx_hog0_drop_pkts;
+ uint64_t cnt_tx_hog1_drop_pkts;
+ uint64_t cnt_rate_limit0_drop_pkts;
+ uint64_t cnt_rate_limit1_drop_pkts;
+ uint64_t cnt_bad_smp_drop_pkts;
+ uint64_t cnt_trigger_drop_redir_pkts;
+ uint64_t cnt_trigger_drop_pkts;
+ uint64_t cnt_trigger_redir_pkts;
+ uint64_t cnt_glort_forwarded_pkts;
+ uint64_t cnt_trigger_mirrored_pkts;
+ uint64_t cnt_broadcast_drop_pkts;
+ uint64_t cnt_dlf_drop_pkts;
+ uint64_t cnt_rx_cm_drop_pkts;
+ uint64_t cnt_fid_forwarded_octets;
+ uint64_t cnt_flood_forwarded_octets;
+ uint64_t cnt_specially_handled_octets;
+ uint64_t cnt_parse_err_drop_octets;
+ uint64_t cnt_parity_error_octets;
+ uint64_t cnt_trapped_octets;
+ uint64_t cnt_pause_drop_octets;
+ uint64_t cnt_stp_drop_octets;
+ uint64_t cnt_security_violation_octets;
+ uint64_t cnt_vlan_tag_drop_octets;
+ uint64_t cnt_vlan_ingressbv_octets;
+ uint64_t cnt_vlan_egressbv_octets;
+ uint64_t cnt_loopback_drop_octets;
+ uint64_t cnt_glort_miss_drop_octets;
+ uint64_t cnt_ffu_drop_octets;
+ uint64_t cnt_policer_drop_octets;
+ uint64_t cnt_ttl_drop_octets;
+ uint64_t cnt_other_octets;
+ uint64_t cnt_flood_control_drop_octets;
+ uint64_t cnt_cmpriv_drop_octets;
+ uint64_t cnt_smp0_drop_octets;
+ uint64_t cnt_smp1_drop_octets;
+ uint64_t cnt_rx_hog0_drop_octets;
+ uint64_t cnt_rx_hog1_drop_octets;
+ uint64_t cnt_tx_hog0_drop_octets;
+ uint64_t cnt_tx_hog1_drop_octets;
+ uint64_t cnt_trigger_drop_octets;
+ uint64_t cnt_trigger_redir_octets;
+ uint64_t cnt_glort_forwarded_octets;
+ uint64_t cnt_tx_ucst_pkts;
+ uint64_t cnt_tx_bcst_pkts;
+ uint64_t cnt_tx_mcst_pkts;
+ uint64_t cnt_tx_ucst_pkts_nonip;
+ uint64_t cnt_tx_bcst_pkts_nonip;
+ uint64_t cnt_tx_mcst_pkts_nonip;
+ uint64_t cnt_tx_ucst_pkts_ip;
+ uint64_t cnt_tx_bcst_pkts_ip;
+ uint64_t cnt_tx_mcst_pkts_ip;
+ uint64_t cnt_tx_pause_pkts;
+ uint64_t cnt_tx_cbpause_pkts;
+ uint64_t cnt_tx_fcs_err_drop_pkts;
+ uint64_t cnt_tx_framing_error_pkts;
+ uint64_t cnt_tx_error_sent_pkts;
+ uint64_t cnt_tx_error_drop_pkts;
+ uint64_t cnt_tx_timeout_pkts;
+ uint64_t cnt_tx_outofmem_err_pkts;
+ uint64_t cnt_tx_unrepair_ecc_pkts;
+ uint64_t cnt_tx_loopback_pkts;
+ uint64_t cnt_tx_ttl_drop_pkts;
+ uint64_t cnt_tx_minto63_pkts;
+ uint64_t cnt_tx_64_pkts;
+ uint64_t cnt_tx_65to127_pkts;
+ uint64_t cnt_tx_128to255_pkts;
+ uint64_t cnt_tx_256to511_pkts;
+ uint64_t cnt_tx_512to1023_pkts;
+ uint64_t cnt_tx_1024to1522_pkts;
+ uint64_t cnt_tx_1523to2047_pkts;
+ uint64_t cnt_tx_2048to4095_pkts;
+ uint64_t cnt_tx_4096to8191_pkts;
+ uint64_t cnt_tx_8192to10239_pkts;
+ uint64_t cnt_tx_10240tomax_pkts;
+ uint64_t cnt_tx_minto63_octets;
+ uint64_t cnt_tx_64_octets;
+ uint64_t cnt_tx_65to127_octets;
+ uint64_t cnt_tx_128to255_octets;
+ uint64_t cnt_tx_256to511_octets;
+ uint64_t cnt_tx_512to1023_octets;
+ uint64_t cnt_tx_1024to1522_octets;
+ uint64_t cnt_tx_1523to2047_octets;
+ uint64_t cnt_tx_2048to4095_octets;
+ uint64_t cnt_tx_4096to8191_octets;
+ uint64_t cnt_tx_8192to10239_octets;
+ uint64_t cnt_tx_10240tomax_octets;
+ uint64_t cnt_tx_ucst_octets_nonip;
+ uint64_t cnt_tx_bcst_octets_nonip;
+ uint64_t cnt_tx_mcst_octets_nonip;
+ uint64_t cnt_tx_ucst_octets_ip;
+ uint64_t cnt_tx_bcst_octets_ip;
+ uint64_t cnt_tx_mcst_octets_ip;
+ uint64_t cnt_tx_ucst_octets;
+ uint64_t cnt_tx_mcst_octets;
+ uint64_t cnt_tx_bcst_octets;
+ uint64_t cnt_tx_fcs_err_drop_octets;
+ uint64_t cnt_tx_octets;
+ uint64_t cnt_tx_error_octets;
+ uint64_t cnt_tx_framing_error_octets;
+ uint64_t cnt_tx_pause_octets;
+ uint64_t cnt_tx_cbpause_octets;
+ uint64_t cnt_tx_fcs_errored_octets;
+ uint64_t cnt_tx_error_sent_octets;
+ uint64_t cnt_tx_timeout_octets;
+ uint64_t cnt_tx_outofmem_err_octets;
+ uint64_t cnt_tx_unrepair_ecc_octets;
+ uint64_t cnt_tx_loopback_octets;
+ uint64_t cnt_tx_ttl_drop_octets;
+ uint64_t cnt_tx_priority_octets[16];
+ uint64_t cnt_under_run_pkts;
+ uint64_t cnt_over_run_pkts;
+ uint64_t cnt_rx_fragment_pkts;
+ uint64_t cnt_rx_undersized_pkts;
+ uint64_t cnt_rx_jabber_pkts;
+ uint64_t cnt_corrupted_pkts;
+ uint64_t cnt_code_errors;
+ uint64_t cnt_rx_oversized_pkts;
+ uint64_t cnt_tx_fcs_errored_pkts;
+ uint64_t cnt_stats_drop_count_tx;
+ uint64_t cnt_stats_drop_count_rx;
+ uint64_t cnt_tx_mirror_pkts;
+ uint64_t cnt_tx_mirror_octets;
+ uint64_t cnt_tx_cmdrop_pkts;
+ uint64_t timestamp;
+};
+
+void fm10k_stats_rule_count_reg(uint16_t rule_id);
+void *fm10k_switch_process_stats(void *ctx_);
+
+void fm10k_stats_epl_port_print(struct fm10k_switch *sw);
+void fm10k_stats_dpdk_port_print(struct fm10k_switch *sw);
+void fm10k_stats_ffu_count_print(struct fm10k_switch *sw);
+void fm10k_stats_port_bank_print(struct fm10k_switch *sw);
+
+#endif /* _fm10k_stats_h */
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
index 2423dbf..b4b363c 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.h
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -76,6 +76,37 @@
#define FM10K_SW_MEM_POOL_SEGS_MAX 24576
#define FM10K_SW_MEM_POOL_SEGS_RSVD 256
+/*
+ * GLORT MAP
+ */
+#define FM10K_SW_EPLA_GLORT 0x10
+#define FM10K_SW_EPLB_GLORT 0x20
+#define FM10K_SW_PEP01_GLORT 0x30
+#define FM10K_SW_PEP23_GLORT 0x40
+#define FM10K_SW_PEP45_GLORT 0x50
+#define FM10K_SW_PEP67_GLORT 0x60
+
+/*
+ * logical port number
+ */
+#define FM10K_SW_EPLA_LOGICAL_PORT 1
+#define FM10K_SW_EPLB_LOGICAL_PORT 2
+
+#define FM10K_SW_PEP01_LOGICAL_PORT 3
+#define FM10K_SW_PEP23_LOGICAL_PORT 4
+#define FM10K_SW_PEP45_LOGICAL_PORT 5
+#define FM10K_SW_PEP67_LOGICAL_PORT 6
+
+/*
+ * physical port number
+ */
+#define FM10K_SW_EPLA_PHYSICAL_PORT 0
+#define FM10K_SW_EPLB_PHYSICAL_PORT 4
+
+#define FM10K_SW_PEP01_PHYSICAL_PORT 36
+#define FM10K_SW_PEP23_PHYSICAL_PORT 40
+#define FM10K_SW_PEP45_PHYSICAL_PORT 44
+#define FM10K_SW_PEP67_PHYSICAL_PORT 48
#define FM10K_SW_CARD_ID(v_, d_) (((v_) << 16) | (d_))
@@ -111,6 +142,7 @@
#define FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4 0x01C2
#define FM10K_SW_DEV_ID_PE325G2DSIR 0x01C8
+
/*
* SWITCH
*/
@@ -125,6 +157,15 @@ struct fm10k_device_info {
uint8_t num_peps;
};
+static struct fm10k_device_info fm10k_device_table[] = {
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIR_QXSL4,
+ "Silicom PE3100G2DQiR-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRL_QXSL4,
+ "Silicom PE3100G2DQiRL-QX4/QS4/QL4", 2, 100, 2, 2 },
+ { FM10K_SW_VENDOR_ID_SILICOM, FM10K_SW_DEV_ID_PE3100G2DQIRM_QXSL4,
+ "Silicom PE3100G2DQiRM-QX4/QS4/QL4", 2, 100, 2, 4 },
+};
+
struct fm10k_i2c;
struct fm10k_sbus;
@@ -287,6 +328,114 @@ struct fm10k_switch {
fm10k_write_switch_reg(sw, FM10K_SW_GPIO_DATA, data);
}
+
+static struct fm10k_sw_port_map fm10k_pep_port_map[FM10K_SW_PEPS_SUPPORTED] = {
+ {
+ FM10K_SW_PEP01_GLORT,
+ FM10K_SW_PEP01_LOGICAL_PORT,
+ FM10K_SW_PEP01_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP23_GLORT,
+ FM10K_SW_PEP23_LOGICAL_PORT,
+ FM10K_SW_PEP23_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP45_GLORT,
+ FM10K_SW_PEP45_LOGICAL_PORT,
+ FM10K_SW_PEP45_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_PEP67_GLORT,
+ FM10K_SW_PEP67_LOGICAL_PORT,
+ FM10K_SW_PEP67_PHYSICAL_PORT
+ },
+};
+
+static struct fm10k_sw_port_map fm10k_epl_port_map[FM10K_SW_EPLS_SUPPORTED] = {
+ {
+ FM10K_SW_EPLA_GLORT,
+ FM10K_SW_EPLA_LOGICAL_PORT,
+ FM10K_SW_EPLA_PHYSICAL_PORT
+ },
+ {
+ FM10K_SW_EPLB_GLORT,
+ FM10K_SW_EPLB_LOGICAL_PORT,
+ FM10K_SW_EPLB_PHYSICAL_PORT
+ },
+};
+
+static inline uint32_t
+fm10k_switch_pf_logical_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_epl_logical_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].logical_port;
+}
+
+static inline uint32_t
+fm10k_switch_vf_glort_get(uint8_t vf_no)
+{
+ return FM10K_SW_VF_GLORT_START + vf_no;
+}
+
+static inline uint32_t
+fm10k_switch_pf_glort_get(uint8_t pf_no)
+{
+ return fm10k_pep_port_map[pf_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_epl_glort_get(uint8_t epl_no)
+{
+ return fm10k_epl_port_map[epl_no].glort;
+}
+
+static inline uint32_t
+fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2)
+{
+ uint8_t idx;
+ if (pf1 > pf2)
+ idx = (pf2 & 0xf) | (pf1 << 4 & 0xf0);
+ else
+ idx = (pf1 & 0xf) | (pf2 << 4 & 0xf0);
+ return FM10K_SW_PFS_GLORT_START + idx;
+}
+
+
+static inline struct fm10k_device_info*
+fm10k_get_device_info(struct fm10k_hw *hw)
+{
+ unsigned int i;
+ struct fm10k_device_info *info;
+ uint16_t pci_vendor = hw->vendor_id;
+ uint16_t pci_device = hw->device_id;
+ uint16_t pci_subvendor = hw->subsystem_vendor_id;
+ uint16_t pci_subdevice = hw->subsystem_device_id;
+
+ if (pci_vendor != FM10K_SW_VENDOR_ID_INTEL ||
+ pci_device != FM10K_SW_DEV_ID_FM10K)
+ return (NULL);
+
+ for (i = 0;
+ i < sizeof(fm10k_device_table) /
+ sizeof(fm10k_device_table[0]);
+ i++) {
+ info = &fm10k_device_table[i];
+ if (pci_subvendor == info->subvendor &&
+ pci_subdevice == info->subdevice) {
+ return info;
+ }
+ }
+
+ return NULL;
+}
+
+
#define fm10k_udelay usec_delay
typedef int eth_fm10k_dev_init_half_func(struct fm10k_hw *hw);
@@ -312,15 +461,6 @@ struct fm10k_flow_list *
void fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
void fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num);
-uint32_t fm10k_switch_pf_logical_get(uint8_t pf_no);
-uint32_t fm10k_switch_epl_logical_get(uint8_t epl_no);
-uint32_t fm10k_switch_vf_glort_get(uint8_t vf_no);
-uint32_t fm10k_switch_pf_glort_get(uint8_t pf_no);
-uint32_t fm10k_switch_pfs_glort_get(uint8_t pf1, uint8_t pf2);
-uint32_t fm10k_switch_epl_glort_get(uint8_t epl_no);
-uint32_t fm10k_switch_multi_glort_get(uint8_t pf1, uint8_t pf2,
- uint16_t vlan1, uint16_t vlan2, bool *p_new);
-
int fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan);
int fm10k_switch_mirror_reset(struct fm10k_hw *hw);
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
` (2 preceding siblings ...)
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-04-02 8:55 ` Wang, Xiao W
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support Xiaojun Liu
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Add flow interface to support offload flow into HW.
It supports parse vlan and parse mpls, all these
data will be transferred to ffu data.
Add switch management, includes initialization,
port mapping, epl port link, LED controller, interrupt handler.
It create 3 threads. One for interrupt handler, one for
LED controller, one for statistics.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
drivers/net/fm10k/Makefile | 2 +
drivers/net/fm10k/base/fm10k_type.h | 1 +
drivers/net/fm10k/switch/fm10k_flow.c | 872 +++++++++++
drivers/net/fm10k/switch/fm10k_flow.h | 26 +
drivers/net/fm10k/switch/fm10k_stats.c | 128 ++
drivers/net/fm10k/switch/fm10k_switch.c | 2382 +++++++++++++++++++++++++++++++
drivers/net/fm10k/switch/fm10k_switch.h | 1 +
7 files changed, 3412 insertions(+)
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.c
create mode 100644 drivers/net/fm10k/switch/fm10k_flow.h
create mode 100644 drivers/net/fm10k/switch/fm10k_switch.c
diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile
index 68a72d5..5a8d227 100644
--- a/drivers/net/fm10k/Makefile
+++ b/drivers/net/fm10k/Makefile
@@ -88,6 +88,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_spico_code.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_config.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_ffu.c
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_stats.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_flow.c
+SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_switch.c
endif
ifeq ($(CONFIG_RTE_ARCH_X86), y)
SRCS-$(CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR) += fm10k_rxtx_vec.c
diff --git a/drivers/net/fm10k/base/fm10k_type.h b/drivers/net/fm10k/base/fm10k_type.h
index 84781ba..810e5ee 100644
--- a/drivers/net/fm10k/base/fm10k_type.h
+++ b/drivers/net/fm10k/base/fm10k_type.h
@@ -717,6 +717,7 @@ struct fm10k_hw {
u16 subsystem_vendor_id;
u8 revision_id;
u32 flags;
+ bool switch_ready;
#define FM10K_HW_FLAG_CLOCK_OWNER BIT(0)
};
diff --git a/drivers/net/fm10k/switch/fm10k_flow.c b/drivers/net/fm10k/switch/fm10k_flow.c
new file mode 100644
index 0000000..353f021
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.c
@@ -0,0 +1,872 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <sys/queue.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdarg.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_eth_ctrl.h>
+#include <rte_tailq.h>
+#include <rte_flow_driver.h>
+
+#include "fm10k_flow.h"
+#include "fm10k_switch.h"
+#include "fm10k_ffu.h"
+#include "fm10k_config.h"
+
+
+static int fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static struct rte_flow *fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error);
+static int fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error);
+static int fm10k_flow_flush(struct rte_eth_dev *dev,
+ struct rte_flow_error *error);
+static int fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error);
+
+const struct rte_flow_ops fm10k_flow_ops = {
+ .validate = fm10k_flow_validate,
+ .create = fm10k_flow_create,
+ .destroy = fm10k_flow_destroy,
+ .flush = fm10k_flow_flush,
+};
+
+union fm10k_filter_t cons_filter;
+enum rte_filter_type fm10k_cons_filter_type = RTE_ETH_FILTER_NONE;
+
+/**
+ * MPLS filter configuration.
+ */
+enum fm10k_mpls_type {
+ FM10K_MPLS_TYPE_UNI,
+ FM10K_MPLS_TYPE_MULTI,
+};
+
+enum fm10k_mpls_action {
+ FM10K_MPLS_ACTION_DROP,
+ FM10K_MPLS_ACTION_QUEUE,
+};
+
+struct fm10k_mpls_filter_conf {
+ enum fm10k_mpls_type mpls_type; /**< mandatory for MPLS */
+ uint32_t mpls_header; /**< MPLS header */
+ uint32_t mpls_header_mask; /**< MPLS header mask */
+ enum fm10k_mpls_action mpls_action;
+ uint16_t queue;
+ uint8_t ffu_id;
+ uint8_t ffu_prio;
+};
+
+/**
+ * VLAN filter configuration.
+ */
+struct fm10k_vlan_filter_conf {
+ int ffu_id;
+ uint8_t ffu_prio;
+ uint8_t is_ingress;
+ uint8_t port;
+ uint8_t in_ext_port;
+ uint8_t out_ext_port;
+ uint16_t in_vlan;
+ uint16_t out_vlan;
+};
+
+
+union fm10k_filter_t {
+ struct fm10k_mpls_filter_conf mpls_filter;
+ struct fm10k_vlan_filter_conf vlan_filter;
+};
+
+typedef int (*parse_filter_t)(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+struct fm10k_valid_pattern {
+ enum rte_flow_item_type *items;
+ parse_filter_t parse_filter;
+};
+
+static enum rte_flow_item_type pattern_mpls_1[] = {
+ RTE_FLOW_ITEM_TYPE_ETH,
+ RTE_FLOW_ITEM_TYPE_MPLS,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_1[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static enum rte_flow_item_type pattern_vlan_2[] = {
+ RTE_FLOW_ITEM_TYPE_VLAN,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_PHY_PORT,
+ RTE_FLOW_ITEM_TYPE_END,
+};
+
+static int fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter);
+
+static struct fm10k_valid_pattern fm10k_supported_patterns[] = {
+ /* MPLS */
+ { pattern_mpls_1, fm10k_flow_parse_mpls_filter },
+ /* VLAN */
+ { pattern_vlan_1, fm10k_flow_parse_vlan_filter },
+ /* VLAN */
+ { pattern_vlan_2, fm10k_flow_parse_vlan_filter },
+};
+
+static const struct rte_flow_action *
+fm10k_next_item_of_action(const struct rte_flow_action *actions,
+ uint32_t *index)
+{
+ static const struct rte_flow_action *act;
+
+ act = actions + *index;
+ while (act->type == RTE_FLOW_ACTION_TYPE_VOID) {
+ (*index)++;
+ act = actions + *index;
+ }
+ return act;
+}
+
+/* Find the first VOID or non-VOID item pointer */
+static const struct rte_flow_item *
+fm10k_find_first_item(const struct rte_flow_item *item, bool is_void)
+{
+ bool is_find;
+
+ while (item->type != RTE_FLOW_ITEM_TYPE_END) {
+ if (is_void)
+ is_find = item->type == RTE_FLOW_ITEM_TYPE_VOID;
+ else
+ is_find = item->type != RTE_FLOW_ITEM_TYPE_VOID;
+ if (is_find)
+ break;
+ item++;
+ }
+ return item;
+}
+
+/* Skip all VOID items of the pattern */
+static void
+fm10k_pattern_skip_void_item(struct rte_flow_item *items,
+ const struct rte_flow_item *pattern)
+{
+ uint32_t cpy_count = 0;
+ const struct rte_flow_item *pb = pattern, *pe = pattern;
+
+ for (;;) {
+ /* Find a non-void item first */
+ pb = fm10k_find_first_item(pb, false);
+ if (pb->type == RTE_FLOW_ITEM_TYPE_END) {
+ pe = pb;
+ break;
+ }
+
+ /* Find a void item */
+ pe = fm10k_find_first_item(pb + 1, true);
+
+ cpy_count = pe - pb;
+ rte_memcpy(items, pb, sizeof(struct rte_flow_item) * cpy_count);
+
+ items += cpy_count;
+
+ if (pe->type == RTE_FLOW_ITEM_TYPE_END) {
+ pb = pe;
+ break;
+ }
+
+ pb = pe + 1;
+ }
+ /* Copy the END item. */
+ rte_memcpy(items, pe, sizeof(struct rte_flow_item));
+}
+
+/* Check if the pattern matches a supported item type array */
+static bool
+fm10k_match_pattern(enum rte_flow_item_type *item_array,
+ struct rte_flow_item *pattern)
+{
+ struct rte_flow_item *item = pattern;
+
+ while ((*item_array == item->type) &&
+ (*item_array != RTE_FLOW_ITEM_TYPE_END)) {
+ item_array++;
+ item++;
+ }
+
+ return (*item_array == RTE_FLOW_ITEM_TYPE_END &&
+ item->type == RTE_FLOW_ITEM_TYPE_END);
+}
+
+/* Find if there's parse filter function matched */
+static parse_filter_t
+fm10k_find_parse_filter_func(struct rte_flow_item *pattern, uint32_t *idx)
+{
+ parse_filter_t parse_filter = NULL;
+ uint8_t i = *idx;
+
+ for (; i < RTE_DIM(fm10k_supported_patterns); i++) {
+ if (fm10k_match_pattern(fm10k_supported_patterns[i].items,
+ pattern)) {
+ parse_filter = fm10k_supported_patterns[i].parse_filter;
+ break;
+ }
+ }
+
+ *idx = ++i;
+
+ return parse_filter;
+}
+
+/* Parse attributes */
+static int
+fm10k_flow_parse_attr(const struct rte_flow_attr *attr,
+ struct rte_flow_error *error)
+{
+ /* Not supported */
+ if (attr->group) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+ attr, "Not support group.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+/*
+ * MPLS
+ */
+/* 1. Last in item should be NULL as range is not supported.
+ * 2. Supported filter types: MPLS label.
+ * 3. Mask of fields which need to be matched should be
+ * filled with 1.
+ * 4. Mask of fields which needn't to be matched should be
+ * filled with 0.
+ */
+static int
+fm10k_flow_parse_mpls_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_mpls *mpls_spec;
+ const struct rte_flow_item_mpls *mpls_mask;
+ const struct rte_flow_item_eth *eth_spec;
+ enum rte_flow_item_type item_type;
+ const uint8_t label_mask[3] = {0xFF, 0xFF, 0xF0};
+ uint32_t label_be = 0;
+ uint32_t be_mask = 0;
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_ETH:
+ if (!item->spec || item->mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid ETH item");
+ return -rte_errno;
+ }
+ eth_spec =
+ (const struct rte_flow_item_eth *)item->spec;
+
+ if (rte_be_to_cpu_16(eth_spec->type) == 0x8847) {
+ filter->mpls_type = FM10K_MPLS_TYPE_UNI;
+ } else if (rte_be_to_cpu_16(eth_spec->type) == 0x8848) {
+ filter->mpls_type = FM10K_MPLS_TYPE_MULTI;
+ } else {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item, "Invalid ETH item");
+ return -rte_errno;
+ }
+ break;
+ case RTE_FLOW_ITEM_TYPE_MPLS:
+ mpls_spec =
+ (const struct rte_flow_item_mpls *)item->spec;
+ mpls_mask =
+ (const struct rte_flow_item_mpls *)item->mask;
+
+ if (!mpls_spec || !mpls_mask) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS item");
+ return -rte_errno;
+ }
+
+ if (memcmp(mpls_mask->label_tc_s, label_mask, 3)) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid MPLS label mask");
+ return -rte_errno;
+ }
+ rte_memcpy(((uint8_t *)&label_be + 1),
+ mpls_spec->label_tc_s, 3);
+ rte_memcpy(((uint8_t *)&be_mask + 1),
+ mpls_mask->label_tc_s, 3);
+ filter->mpls_header =
+ rte_be_to_cpu_32(label_be) >> 4;
+ filter->mpls_header_mask =
+ rte_be_to_cpu_32(be_mask) >> 4;
+
+ fm10k_cons_filter_type = RTE_ETH_FILTER_TUNNEL;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* MPLS action only supports QUEUE or DROP. */
+static int
+fm10k_flow_parse_mpls_action(const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_mpls_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ const struct rte_flow_action_queue *act_q;
+ uint32_t index = 0;
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_QUEUE &&
+ act->type != RTE_FLOW_ACTION_TYPE_DROP) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ if (act->type == RTE_FLOW_ACTION_TYPE_QUEUE) {
+ act_q = (const struct rte_flow_action_queue *)act->conf;
+ filter->mpls_action = FM10K_MPLS_ACTION_QUEUE;
+ filter->queue = act_q->index;
+ } else {
+ filter->mpls_action = FM10K_MPLS_ACTION_DROP;
+ }
+
+ /* Check if the next non-void item is END */
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_mpls_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ struct fm10k_mpls_filter_conf *mpls_filter =
+ &filter->mpls_filter;
+ int ret;
+
+ ret = fm10k_flow_parse_mpls_pattern(dev, pattern,
+ error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_mpls_action(actions, error, mpls_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+
+/*
+ * VLAN
+ */
+static int
+fm10k_flow_parse_vlan_pattern(__rte_unused struct rte_eth_dev *dev,
+ const struct rte_flow_item *pattern,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_item *item = pattern;
+ const struct rte_flow_item_vlan *vlan_spec;
+ const struct rte_flow_item_phy_port *pp_spec;
+ enum rte_flow_item_type item_type;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan pattern ffu id %d", filter->ffu_id);
+
+ for (; item->type != RTE_FLOW_ITEM_TYPE_END; item++) {
+ if (item->last) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Not support range");
+ return -rte_errno;
+ }
+ item_type = item->type;
+ switch (item_type) {
+ case RTE_FLOW_ITEM_TYPE_VLAN:
+ vlan_spec =
+ (const struct rte_flow_item_vlan *)item->spec;
+
+ if (!vlan_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid VLAN item");
+ return -rte_errno;
+ }
+ break;
+
+ case RTE_FLOW_ITEM_TYPE_PHY_PORT:
+ pp_spec =
+ (const struct rte_flow_item_phy_port *)item->spec;
+ if (!pp_spec) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ item,
+ "Invalid PHY PORT item");
+ return -rte_errno;
+ }
+ break;
+
+ default:
+ break;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_flow_parse_vlan_action(struct rte_eth_dev *dev,
+ const struct rte_flow_action *actions,
+ struct rte_flow_error *error,
+ struct fm10k_vlan_filter_conf *filter)
+{
+ const struct rte_flow_action *act;
+ uint32_t index = 0;
+
+ PMD_INIT_LOG(DEBUG, "Parse vlan action name %s ffu id %d",
+ dev->device->name, filter->ffu_id);
+
+ /* Check if the first non-void action is QUEUE or DROP. */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ index++;
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_MARK &&
+ act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+ if (act->type == RTE_FLOW_ACTION_TYPE_END)
+ return 0;
+
+ index++;
+ /* Check if the next non-void item is END */
+ act = fm10k_next_item_of_action(actions, &index);
+ if (act->type != RTE_FLOW_ACTION_TYPE_END) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+ act, "Not supported action.");
+ return -rte_errno;
+ }
+
+ return 0;
+}
+
+static int
+fm10k_flow_parse_vlan_filter(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error,
+ union fm10k_filter_t *filter)
+{
+ int ret;
+ struct fm10k_vlan_filter_conf *vlan_filter =
+ &filter->vlan_filter;
+
+ ret = fm10k_flow_parse_vlan_pattern(dev, pattern,
+ error, vlan_filter);
+ if (ret)
+ return ret;
+
+ ret = fm10k_flow_parse_vlan_action(dev, actions, error, vlan_filter);
+ if (ret)
+ return ret;
+
+ if (attr->ingress)
+ vlan_filter->is_ingress = 1;
+ else if (attr->egress)
+ vlan_filter->is_ingress = 0;
+ vlan_filter->ffu_prio = attr->priority;
+
+ ret = fm10k_flow_parse_attr(attr, error);
+ return ret;
+}
+
+/*
+ *
+ */
+static int
+fm10k_flow_validate(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct rte_flow_item *items; /* internal pattern w/o VOID items */
+ parse_filter_t parse_filter;
+ uint32_t item_num = 0; /* non-void item number of pattern*/
+ uint32_t i = 0;
+ bool flag = false;
+ int ret = -1;
+
+ if (!pattern) {
+ rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "NULL pattern.");
+ return -rte_errno;
+ }
+
+ if (!actions) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ACTION_NUM,
+ NULL, "NULL action.");
+ return -rte_errno;
+ }
+
+ if (!attr) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ATTR,
+ NULL, "NULL attribute.");
+ return -rte_errno;
+ }
+
+ /* Get the non-void item number of pattern */
+ while ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_END) {
+ if ((pattern + i)->type != RTE_FLOW_ITEM_TYPE_VOID)
+ item_num++;
+ i++;
+ }
+ item_num++;
+
+ items = rte_zmalloc("fm10k_pattern",
+ item_num * sizeof(struct rte_flow_item), 0);
+ if (!items) {
+ rte_flow_error_set(error, ENOMEM, RTE_FLOW_ERROR_TYPE_ITEM_NUM,
+ NULL, "No memory for PMD internal items.");
+ return -ENOMEM;
+ }
+
+ fm10k_pattern_skip_void_item(items, pattern);
+
+ i = 0;
+ do {
+ parse_filter = fm10k_find_parse_filter_func(items, &i);
+ if (!parse_filter && !flag) {
+ rte_flow_error_set(error, EINVAL,
+ RTE_FLOW_ERROR_TYPE_ITEM,
+ pattern, "Unsupported pattern");
+ rte_free(items);
+ return -rte_errno;
+ }
+ if (parse_filter)
+ ret = parse_filter(dev, attr, items, actions,
+ error, &cons_filter);
+ flag = true;
+ } while ((ret < 0) && (i < RTE_DIM(fm10k_supported_patterns)));
+
+ rte_free(items);
+
+ return ret;
+}
+
+static struct fm10k_cfg_flow *
+fm10k_flow_cfg_transfer(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ int i;
+ u8 port_id;
+ int set_port_num = 0, set_vlan_num = 0;
+ u16 fw_port_id = 0, bp_port_id = 0;
+ u16 filter_vlan_id = 0, fw_vlan_id = 0, bp_vlan_id = 0;
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct fm10k_cfg_flow *cf;
+
+ cf = rte_zmalloc("fm10k_rule", sizeof(struct fm10k_cfg_flow), 0);
+ if (!cf) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return NULL;
+ }
+ memset(cf, 0, sizeof(struct fm10k_cfg_flow));
+
+ port_id = fm10k_switch_dpdk_port_no_get(hw);
+ for (i = 0; i < 4; i++) {
+ if (pattern[i].type == RTE_FLOW_ITEM_TYPE_VLAN) {
+ filter_vlan_id =
+ rte_be_to_cpu_16
+ (((const struct rte_flow_item_vlan *)
+ pattern[i].spec)->tci);
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_PHY_PORT) {
+ if (set_port_num)
+ bp_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ else
+ fw_port_id =
+ ((const struct rte_flow_item_phy_port *)
+ pattern[i].spec)->index;
+ set_port_num++;
+ } else if (pattern[i].type == RTE_FLOW_ITEM_TYPE_END) {
+ break;
+ }
+ }
+
+ for (i = 0; i < 3; i++) {
+ if (actions[i].type == RTE_FLOW_ACTION_TYPE_MARK) {
+ if (set_vlan_num)
+ bp_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ else
+ fw_vlan_id =
+ ((const struct rte_flow_action_mark *)
+ actions[i].conf)->id;
+ set_vlan_num++;
+ } else if (actions[i].type == RTE_FLOW_ACTION_TYPE_END) {
+ break;
+ }
+ }
+
+ if (attr->ingress && !attr->egress) {
+ /* this port is DPDK port and it is destination port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = fw_port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && attr->egress) {
+ /* this port is DPDK port and it is source port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else if (!attr->ingress && !attr->egress) {
+ /* two ports are external port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ } else {
+ /* two ports are DPDK port */
+ cf->src_port.port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->src_port.port_no = port_id;
+ cf->src_port.vlan_id = filter_vlan_id;
+ cf->fw_port[0].port_type = FM10K_CONFIG_FLOW_DPDK_PORT;
+ cf->fw_port[0].port_no = fw_port_id;
+ cf->fw_port[0].vlan_id = fw_vlan_id;
+ }
+
+ if (set_port_num == 2 && set_vlan_num == 2) {
+ cf->fw_port[1].port_type = FM10K_CONFIG_FLOW_EXT_PORT;
+ cf->fw_port[1].port_no = bp_port_id;
+ cf->fw_port[1].vlan_id = bp_vlan_id;
+ }
+
+ return cf;
+}
+
+
+static struct rte_flow *
+fm10k_flow_create(struct rte_eth_dev *dev,
+ const struct rte_flow_attr *attr,
+ const struct rte_flow_item pattern[],
+ const struct rte_flow_action actions[],
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ struct fm10k_switch *sw = fm10k_switch_get();
+ struct fm10k_cfg_flow *cf;
+ int ret;
+
+ flow = rte_zmalloc("fm10k_flow", sizeof(struct rte_flow), 0);
+ if (!flow) {
+ rte_flow_error_set(error, ENOMEM,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to allocate memory");
+ return flow;
+ }
+
+ ret = fm10k_flow_validate(dev, attr, pattern, actions, error);
+ if (ret < 0)
+ return NULL;
+
+ cf = fm10k_flow_cfg_transfer(dev, attr, pattern, actions, error);
+ if (!cf)
+ goto free_flow;
+
+ flow->rule = cf;
+ fm10k_ffu_flow_enable(sw, cf);
+ fm10k_config_flow_list_add_tail(fm10k_config_flowset_current_get(), cf);
+
+ TAILQ_INSERT_TAIL((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw), flow, node);
+ return flow;
+
+free_flow:
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to create flow.");
+ rte_free(flow);
+ return NULL;
+}
+
+static int
+fm10k_flow_destroy(struct rte_eth_dev *dev,
+ struct rte_flow *flow,
+ struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ int ret = 0;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (flow->rule) {
+ fm10k_config_flow_list_delete
+ ((struct fm10k_cfg_flow *)flow->rule);
+ fm10k_ffu_flow_disable(sw,
+ (struct fm10k_cfg_flow *)flow->rule);
+ }
+
+ if (!ret) {
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow->rule);
+ rte_free(flow);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "Failed to destroy flow.");
+ }
+ return ret;
+}
+
+
+static int
+fm10k_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+ struct rte_flow *flow;
+ void *temp;
+ int ret = 0;
+ struct fm10k_cfg_flow *cf;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ /* Delete flows in flow list. */
+ TAILQ_FOREACH_SAFE(flow,
+ (struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ node, temp) {
+ cf = flow->rule;
+ if (cf) {
+ fm10k_config_flow_list_delete(cf);
+ fm10k_ffu_flow_disable(sw, cf);
+ } else {
+ rte_flow_error_set(error, -ret,
+ RTE_FLOW_ERROR_TYPE_HANDLE, NULL,
+ "No such rule in flow.");
+ }
+ TAILQ_REMOVE((struct fm10k_flow_list *)
+ fm10k_switch_dpdk_port_flow_list_get(hw),
+ flow, node);
+ rte_free(flow);
+ }
+
+ return ret;
+}
+
+void
+fm10k_flow_list_init(void *flow_list)
+{
+ TAILQ_INIT((struct fm10k_flow_list *)flow_list);
+}
+
+/* Flow operations */
+const struct rte_flow_ops *
+fm10k_flow_ops_get(void)
+{
+ return &fm10k_flow_ops;
+}
+
+
diff --git a/drivers/net/fm10k/switch/fm10k_flow.h b/drivers/net/fm10k/switch/fm10k_flow.h
new file mode 100644
index 0000000..c538544
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_flow.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#ifndef _FM10K_SW_FLOW_H_
+#define _FM10K_SW_FLOW_H_
+
+#include <rte_time.h>
+#include <rte_kvargs.h>
+#include <rte_hash.h>
+#include <rte_flow.h>
+#include <rte_flow_driver.h>
+#include <rte_tm_driver.h>
+
+/*
+ * Struct to store flow created.
+ */
+struct rte_flow {
+ TAILQ_ENTRY(rte_flow) node;
+ enum rte_filter_type filter_type;
+ void *rule;
+};
+
+TAILQ_HEAD(fm10k_flow_list, rte_flow);
+
+#endif /* _FM10K_SW_FLOW_H_ */
diff --git a/drivers/net/fm10k/switch/fm10k_stats.c b/drivers/net/fm10k/switch/fm10k_stats.c
index 451813f..ce491ba 100644
--- a/drivers/net/fm10k/switch/fm10k_stats.c
+++ b/drivers/net/fm10k/switch/fm10k_stats.c
@@ -1004,6 +1004,133 @@ struct fm10k_stats_data {
}
}
+void
+fm10k_stats_dpdk_port_print(struct fm10k_switch *sw)
+{
+ int i, j, lport, pf_no;
+ struct fm10k_port_counters counters;
+ struct fm10k_ext_port ext_port;
+ static struct fm10k_stats_data last_data[FM10K_SW_PEPS_MAX];
+ static struct fm10k_stats_data last_queue_data[FM10K_SW_PEPS_MAX][4];
+ struct fm10k_stats_data data, mydata;
+ char pf_ports[10];
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].hw == NULL)
+ continue;
+ if (sw->dpdk_cfg->ports[i].type != FM10K_CONFIG_DPDK_PF)
+ continue;
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_NULL)
+ continue;
+
+ memset(&mydata, 0, sizeof(mydata));
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ for (j = 0; j < 2; j++) {
+ pf_no =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[j];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters
+ (sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ }
+ } else if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ lport = sw->pep_map[pf_no].logical_port;
+ memset(&ext_port, 0, sizeof(ext_port));
+ ext_port.portno = lport;
+ fm10k_get_port_counters(sw, &ext_port, &counters);
+ data.rx_pkts = counters.cnt_rx_bcst_pkts +
+ counters.cnt_rx_ucst_pkts;
+ data.tx_pkts = counters.cnt_tx_bcst_pkts +
+ counters.cnt_tx_ucst_pkts;
+ data.rx_drop = counters.cnt_cmpriv_drop_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_data[pf_no].rx_pkts;
+ mydata.tx_pkts += data.tx_pkts -
+ last_data[pf_no].tx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_data[pf_no].rx_drop;
+ last_data[pf_no] = data;
+ } else {
+ continue;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[i].type ==
+ FM10K_CONFIG_PORT_MAP_PF)
+ sprintf(pf_ports, "%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0]);
+ else
+ sprintf(pf_ports, "%d/%d",
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0],
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1]);
+
+ FM10K_SW_INFO("DPDK port %-2d pf %-5s tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ i, pf_ports,
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg,
+ FM10K_CONFIG_DEBUG_STATS_QUEUE))
+ continue;
+ memset(&mydata, 0, sizeof(mydata));
+ for (j = 0;
+ j < sw->dpdk_cfg->ports[i].tx_queue_num;
+ j++) {
+ struct fm10k_hw *hw = sw->dpdk_cfg->ports[i].hw;
+ uint16_t queue_id = j, pf_no;
+
+ fm10k_switch_dpdk_hw_queue_map(hw, queue_id,
+ sw->dpdk_cfg->ports[i].tx_queue_num,
+ &hw, &queue_id);
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ data.tx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPTC(queue_id));
+ data.rx_pkts =
+ FM10K_READ_REG(hw, FM10K_QPRC(queue_id));
+ data.rx_drop =
+ FM10K_READ_REG(hw, FM10K_QPRDC(queue_id));
+ mydata.tx_pkts += data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts;
+ mydata.rx_pkts += data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts;
+ mydata.rx_drop += data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop;
+ FM10K_SW_INFO(" queue %d(%d:%d) tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n", j,
+ pf_no, queue_id,
+ (unsigned long long)(data.tx_pkts -
+ last_queue_data[pf_no][queue_id].tx_pkts),
+ (unsigned long long)(data.rx_pkts -
+ last_queue_data[pf_no][queue_id].rx_pkts),
+ (unsigned long long)(data.rx_drop -
+ last_queue_data[pf_no][queue_id].rx_drop));
+ last_queue_data[pf_no][queue_id] = data;
+ }
+ FM10K_SW_INFO(" total tx_pkt %-12llu "
+ "rx_pkt %-12llu drop_pkt %-12llu\n",
+ (unsigned long long)mydata.tx_pkts,
+ (unsigned long long)mydata.rx_pkts,
+ (unsigned long long)mydata.rx_drop);
+ }
+}
static void
fm10k_stats_port_counter_print(struct fm10k_switch *sw, int lport)
@@ -1088,6 +1215,7 @@ struct fm10k_stats_data {
FM10K_CONFIG_DEBUG_STATS_PORT)) {
FM10K_SW_INFO("--- port statistic ---\n");
fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
}
if (fm10k_config_check_debug(sw->dpdk_cfg,
FM10K_CONFIG_DEBUG_STATS_FFU)) {
diff --git a/drivers/net/fm10k/switch/fm10k_switch.c b/drivers/net/fm10k/switch/fm10k_switch.c
new file mode 100644
index 0000000..a3f2bbd
--- /dev/null
+++ b/drivers/net/fm10k/switch/fm10k_switch.c
@@ -0,0 +1,2382 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 Silicom Ltd. Connectivity Solutions
+ */
+
+#include <rte_ethdev.h>
+#include <rte_ethdev_pci.h>
+#include <rte_malloc.h>
+
+#include "../base/fm10k_type.h"
+#include "../base/fm10k_osdep.h"
+#include "../fm10k.h"
+#include "../fm10k_logs.h"
+#include "fm10k_debug.h"
+#include "fm10k_regs.h"
+#include "fm10k_switch.h"
+#include "fm10k_ext_port.h"
+#include "fm10k_serdes.h"
+#include "fm10k_i2c.h"
+#include "fm10k_ffu.h"
+#include "fm10k_stats.h"
+#include "fm10k_config.h"
+
+
+static struct fm10k_switch fm10k_sw;
+
+#define FM10K_AM_TIMEOUT 16384
+#define FM10K_COMP_PPM_SCALE 1000000
+#define FM10K_LED_POLL_INTERVAL_MS 500
+#define FM10K_LED_BLINKS_PER_SECOND 2
+
+/* Max try times to acquire switch status */
+#define FM10K_QUERY_SWITCH_STATE_TIMES 10
+/* Wait interval to get switch status */
+#define FM10K_WAIT_SWITCH_MSG_US 100000
+
+/*
+ * use epl as external port map, only support QUAD_ON
+ */
+struct fm10k_sched_prog {
+ uint8_t idle;
+ uint8_t phys; /* physical port */
+ uint8_t log; /* logical port */
+ uint8_t quad; /* now, only support QUAD_ON */
+/* convert entry to idle if EPL in quad port mode */
+#define FM10K_SW_QUAD_OFF 0
+/* always use quad port mode */
+#define FM10K_SW_QUAD_ON 1
+/* use quad port mode if link speed is 40/100G */
+#define FM10K_SW_QUAD_40_100 2
+};
+
+static struct fm10k_sched_prog
+ fm10k_sched_prog[FM10K_SW_PEPS_SUPPORTED + FM10K_SW_EPLS_SUPPORTED + 1];
+
+
+/*
+ * Note that for physical port numbers, the initial values used for
+ * the EPL entries assume EPL[A] is EPL[0] and EPL[B] is EPL[1].
+ * fm10k_switch_determine_epls() will update these physical port
+ * numbers based on the actual A and B indices.
+ */
+static void
+fm10k_switch_set_sched_prog(void)
+{
+ int i;
+ int start = 0;
+
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_sched_prog[i].idle = 0;
+ fm10k_sched_prog[i].phys = fm10k_epl_port_map[i].physical_port;
+ fm10k_sched_prog[i].log = fm10k_epl_port_map[i].logical_port;
+ fm10k_sched_prog[i].quad = FM10K_SW_QUAD_ON;
+ }
+ start += FM10K_SW_EPLS_SUPPORTED;
+ for (i = 0; i < FM10K_SW_PEPS_SUPPORTED; i++) {
+ fm10k_sched_prog[start + i].idle = 0;
+ fm10k_sched_prog[start + i].phys =
+ fm10k_pep_port_map[i].physical_port;
+ fm10k_sched_prog[start + i].log =
+ fm10k_pep_port_map[i].logical_port;
+ fm10k_sched_prog[start + i].quad = FM10K_SW_QUAD_40_100;
+ }
+ start += FM10K_SW_PEPS_SUPPORTED;
+ fm10k_sched_prog[start].idle = 1;
+}
+
+
+static void
+fm10k_switch_determine_epls(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t phys;
+
+ sw->epla_no = 0;
+ sw->eplb_no = 6;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ sw->epla_no = 1;
+ sw->eplb_no = 6;
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE325G2DSIR):
+ sw->epla_no = 1;
+ sw->eplb_no = 7;
+ break;
+ }
+
+ for (i = 0; i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]); i++) {
+ if (fm10k_sched_prog[i].idle)
+ continue;
+
+ phys = fm10k_sched_prog[i].phys;
+ if (phys <= 3) {
+ /* Substitute actual epla phys port number */
+ fm10k_sched_prog[i].phys = sw->epla_no * 4 + phys;
+ } else if (phys >= 4 && phys <= 7) {
+ /* Substitute actual eplb phys port number */
+ fm10k_sched_prog[i].phys = sw->eplb_no * 4 + (phys - 4);
+ }
+ }
+}
+
+static void
+fm10k_hw_eicr_disable_source(struct fm10k_switch *sw, unsigned int source)
+{
+ unsigned int shift;
+
+ switch (source) {
+ case FM10K_SW_EICR_PCA_FAULT:
+ shift = FM10K_SW_EICR_PCA_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_THI_FAULT:
+ shift = FM10K_SW_EICR_THI_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_FUM_FAULT:
+ shift = FM10K_SW_EICR_FUM_FAULT_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAILBOX:
+ shift = FM10K_SW_EICR_MAILBOX_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_READY:
+ shift = FM10K_SW_EICR_SWITCH_READY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_NREADY:
+ shift = FM10K_SW_EICR_SWITCH_NREADY_SHIFT;
+ break;
+ case FM10K_SW_EICR_SWITCH_INT:
+ shift = FM10K_SW_EICR_SWITCH_INT_SHIFT;
+ break;
+ case FM10K_SW_EICR_SRAM_ERROR:
+ shift = FM10K_SW_EICR_SRAM_ERROR_SHIFT;
+ break;
+ case FM10K_SW_EICR_VFLR:
+ shift = FM10K_SW_EICR_VFLR_SHIFT;
+ break;
+ case FM10K_SW_EICR_MAX_HOLD_TIME:
+ shift = FM10K_SW_EICR_MAX_HOLD_TIME_SHIFT;
+ break;
+ default:
+ return;
+ }
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_DISABLE << (shift * 2));
+ fm10k_write_flush(sw);
+}
+
+
+unsigned int
+fm10k_switch_eplidx_to_eplno(struct fm10k_switch *sw, unsigned int eplidx)
+{
+ return eplidx ? sw->eplb_no : sw->epla_no;
+}
+
+
+static uint16_t
+fm10k_switch_ppm_to_tx_clk_compensation_timeout(uint32_t pcs, uint32_t num_ppm)
+{
+ unsigned int scale;
+ unsigned int ppm;
+ unsigned int timeout;
+ unsigned int am_ppm;
+
+ if (num_ppm == 0 || pcs == FM10K_SW_EPL_PCS_SEL_DISABLE) {
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER)
+ return (FM10K_AM_TIMEOUT / 2);
+ else
+ return 0;
+ }
+
+ if (pcs == FM10K_SW_EPL_PCS_SEL_40GBASER ||
+ pcs == FM10K_SW_EPL_PCS_SEL_100GBASER) {
+ am_ppm = 1000000 / FM10K_AM_TIMEOUT;
+ scale = FM10K_COMP_PPM_SCALE / 2;
+ } else {
+ am_ppm = 0;
+ scale = FM10K_COMP_PPM_SCALE;
+ }
+
+ ppm = num_ppm + am_ppm;
+ timeout = scale / ppm;
+
+ if (timeout >= 0xffff)
+ return 0xffff;
+ else
+ return timeout;
+}
+
+
+static int
+fm10k_switch_configure_epls(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ struct fm10k_ext_ports *ext_ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ struct fm10k_device_info *cfg = sw->info;
+ u32 mac_cfg[FM10K_SW_MAC_CFG_ARRAY_SIZE];
+ u32 data, pcs, qpl;
+ u32 pcstmp;
+ unsigned int i, j;
+ unsigned int dic_enable;
+ unsigned int anti_bubble_wm;
+ unsigned int rate_fifo_wm;
+ unsigned int rate_fifo_slow_inc, rate_fifo_fast_inc;
+ int error;
+ u16 timeout;
+
+ cfg = fm10k_get_device_info(hw);
+ if (cfg == NULL)
+ return -1;
+
+ /*
+ * Assumptions:
+ * - All external interfaces are the same speed
+ * - 1G/10G ports are packed into the minimum number of EPLs
+ * - quad-mode EPLs use lane 0 as master
+ * - The lowest numbered PEPs are used
+ * - PEPs are always used in x8 mode.
+ */
+ switch (cfg->ext_port_speed) {
+ case 10:
+ pcs = FM10K_SW_EPL_PCS_SEL_10GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 1;
+ anti_bubble_wm = 5;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 12;
+ rate_fifo_fast_inc = 13;
+ break;
+ case 25:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L1_L1_L1_L1;
+ dic_enable = 0;
+ anti_bubble_wm = 6;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 32;
+ rate_fifo_fast_inc = 33;
+ break;
+ case 40:
+ pcs = FM10K_SW_EPL_PCS_SEL_40GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 5;
+ rate_fifo_slow_inc = 51;
+ rate_fifo_fast_inc = 52;
+ break;
+ case 100:
+ pcs = FM10K_SW_EPL_PCS_SEL_100GBASER;
+ qpl = FM10K_SW_EPL_QPL_MODE_L4_XX_XX_XX;
+ dic_enable = 1;
+ anti_bubble_wm = 4;
+ rate_fifo_wm = 3;
+ rate_fifo_slow_inc = 129;
+ rate_fifo_fast_inc = 130;
+ break;
+ default:
+ error = -1;
+ goto done;
+ }
+
+ /*
+ * EPL_CFG_A
+ *
+ * Mark all used lanes as active and all unused lanes as
+ * inactive. Adjust timeout and skew tolerance values for EPLs that
+ * are used.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ data = fm10k_read_switch_reg(sw, FM10K_SW_EPL_CFG_A(i));
+ data &= ~FM10K_SW_EPL_CFG_A_ACTIVE_QUAD;
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < ext_ports->num_ports; j++) {
+ port = &ext_ports->ports[j];
+ if (port->eplno == i)
+ data |= port->is_quad ?
+ FM10K_SW_EPL_CFG_A_ACTIVE_QUAD :
+ FM10K_SW_EPL_CFG_A_ACTIVE
+ (port->first_lane);
+ }
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_TIMEOUT, 19, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ EPL_CFG_A_SKEW_TOLERANCE, 38, data);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_A(i), data);
+ }
+
+ /*
+ * EPL_CFG_B
+ *
+ * Disable all unused lanes and configure all used ones
+ * appropriately. For EPLs used in quad port mode, the master lane
+ * is the only one that gets configured.
+ */
+ data = 0;
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i)) {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++) {
+ if (j < ext_ports->ports_per_epl)
+ pcstmp = pcs;
+ else
+ pcstmp = FM10K_SW_EPL_PCS_SEL_DISABLE;
+ data |= FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j, pcstmp, j, j);
+ }
+ data |= FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE, qpl);
+ } else {
+ for (j = 0; j < FM10K_SW_EPL_LANES; j++)
+ data |= FM10K_SW_MAKE_REG_FIELD_IDX
+ (EPL_CFG_B_PCS_SEL, j,
+ FM10K_SW_EPL_PCS_SEL_DISABLE, j, j);
+ data |= FM10K_SW_MAKE_REG_FIELD
+ (EPL_CFG_B_QPL_MODE,
+ FM10K_SW_EPL_QPL_MODE_XX_XX_XX_XX);
+ }
+ fm10k_write_switch_reg(sw, FM10K_SW_EPL_CFG_B(i), data);
+ }
+
+ /*
+ * MAC_CFG, LINK_RULES and PCS_ML_BASER_CFG
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_read_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ /* dic enable */
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_IDLE_ENABLE_DIC,
+ dic_enable, mac_cfg);
+
+ /* ifg */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_IDLE_MIN_IFG_BYTES,
+ 12, mac_cfg);
+
+ /* tx pad size: (64 bytes + 8 preamble) / 4 */
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_MIN_COLUMNS,
+ 18, mac_cfg);
+
+ /* tx clock compensation */
+ timeout =
+ fm10k_switch_ppm_to_tx_clk_compensation_timeout(pcs,
+ 100);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_ENABLE,
+ (timeout > 0), mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_CLOCK_COMPENSATION_TIMEOUT,
+ timeout, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_PREAMBLE_MODE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MIN_FRAME_LENGTH,
+ 64, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_RX_MAX_FRAME_LENGTH,
+ FM10K_SW_PACKET_SIZE_MAX, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_RX_IGNORE_IFG_ERRORS,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FCS_MODE,
+ FM10K_SW_TX_MAX_FCS_MODE_REPLACE_NORMAL, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_IEEE_1588_ENABLE,
+ 0, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_DRAIN_MODE,
+ FM10K_SW_TX_MAC_DRAIN_MODE_DRAIN_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIMEOUT,
+ 100, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_PC_ACT_TIME_SCALE,
+ 3, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIMEOUT,
+ 180, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_TIME_SCALE,
+ 1, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIMEOUT,
+ 20, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_LPI_HOLD_TIME_SCALE,
+ 1, mac_cfg);
+
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LPI_AUTOMATIC,
+ 1, mac_cfg);
+ FM10K_SW_SET_ARRAY_BIT(mac_cfg,
+ MAC_CFG_TX_LP_IDLE_REQUEST,
+ 0, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_FAULT_MODE,
+ FM10K_SW_TX_MAC_FAULT_MODE_NORMAL, mac_cfg);
+
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_ANTI_BUBBLE_WATERMARK,
+ anti_bubble_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_WATERMARK,
+ rate_fifo_wm, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_FAST_INC,
+ rate_fifo_fast_inc, mac_cfg);
+ FM10K_SW_REPLACE_ARRAY_FIELD(mac_cfg,
+ MAC_CFG_TX_RATE_FIFO_SLOW_INC,
+ rate_fifo_slow_inc, mac_cfg);
+
+ fm10k_write_switch_array(sw, FM10K_SW_MAC_CFG(i, j),
+ mac_cfg, FM10K_SW_MAC_CFG_ARRAY_SIZE);
+
+ data =
+ fm10k_read_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j));
+
+ /* link-up debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_UP,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_UP,
+ 30, data);
+
+ /* link-down debounce params */
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TIME_SCALE_DOWN,
+ 4, data);
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_FAULT_TICKS_DOWN,
+ 5, data);
+
+ FM10K_SW_REPLACE_REG_FIELD(data,
+ LINK_RULES_HEARTBEAT_TIME_SCALE,
+ cfg->ext_port_speed == 10 ? 4 : 0, data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_RULES(i, j), data);
+ }
+
+ if (cfg->ext_port_speed != 10)
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PCS_ML_BASER_CFG(i), 0x00003fff);
+ }
+
+
+ /*
+ * LANE_CFG, LANE_SERDES_CFG, LANE_ENERGY_DETECT_CFG,
+ * LANE_SIGNAL_DETECT_CFG, and EPL_FIFO_ERROR_STATUS
+ *
+ * Only EPLs/lanes that are being used are initialized. All others
+ * are left at their defaults.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_CFG(i, j), 0);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SERDES_CFG(i, j), 0x1106);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j));
+
+ data |=
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_RDY |
+ FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_RX_ACTIVITY;
+
+ data &=
+ ~FM10K_SW_LANE_ENERGY_DETECT_CFG_ED_MASK_ENERGY_DETECT;
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_ENERGY_DETECT_CFG(i, j), data);
+
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j));
+
+ data &=
+ ~(FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_SIGNAL_OK |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_RDY);
+
+ data |=
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_RX_ACTIVITY |
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG_SD_MASK_ENERGY_DETECT;
+
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LANE_SIGNAL_DETECT_CFG(i, j), data);
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+ }
+
+ error = fm10k_epl_serdes_reset_and_load_all(sw);
+ if (error)
+ goto done;
+
+ /*
+ * EPL_FIFO_ERROR_STATUS LINK_IP
+ */
+ for (i = 0; i < FM10K_SW_EPLS_MAX; i++) {
+ if (!FM10K_SW_EXT_PORTS_EPL_USED(ext_ports, i))
+ continue;
+ for (j = 0; j < ext_ports->ports_per_epl; j++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_LINK_IP(i, j), FM10K_SW_MASK32(31, 0));
+ }
+
+ data = 0;
+ data |= ((1 << j) << 4) | (1 << j);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_EPL_FIFO_ERROR_STATUS(i), data);
+ }
+done:
+ return (error);
+}
+
+int fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw);
+
+int
+fm10k_switch_mirror_set(struct fm10k_hw *hw, u16 dest_port, u16 vlan)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ /* source port is external port number */
+ if (src_port < 0 || src_port == dest_port)
+ return -1;
+
+ return fm10k_ffu_mirror_set(sw, src_port, dest_port, vlan);
+}
+
+int fm10k_switch_mirror_reset(struct fm10k_hw *hw)
+{
+ struct fm10k_switch *sw = &fm10k_sw;
+ int src_port = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (src_port < 0)
+ return -1;
+
+ return fm10k_ffu_mirror_reset(sw, src_port);
+}
+
+
+typedef struct {
+ u8 lport;
+ u8 has_ftag;
+} fm10k_sw_lport;
+
+static int
+fm10k_switch_init(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ u32 data;
+ unsigned int num_lports = sw->info->num_peps + FM10K_SW_EPLS_SUPPORTED;
+ fm10k_sw_lport all_lports[num_lports];
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ unsigned int is_quad;
+ unsigned int table_idx;
+ int error;
+ u32 watermark;
+ u64 data64, data64_2;
+
+ /*
+ * Build list of all logical ports that might appear in the
+ * scheduler program. Note that for any particular card
+ * configuration, not all of these logical ports may be used.
+ */
+ table_idx = 0;
+ for (i = 0; i < sw->info->num_peps; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->pep_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 1;
+ }
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++, table_idx++) {
+ all_lports[table_idx].lport = sw->epl_map[i].logical_port;
+ all_lports[table_idx].has_ftag = 0;
+ }
+
+ if (table_idx != num_lports) {
+ FM10K_SW_ERR("fm10k switch lport table construction error");
+ return -1;
+ }
+
+ /*
+ * Reset the switch to get to the default state
+ */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+
+ usec_delay(100);
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ usec_delay(1000);
+
+ /* Clear memories */
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL);
+ fm10k_write_flush(sw);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL,
+ FM10K_SW_BIST_CTRL_BIST_MODE_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_MODE_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_MODE_EPL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC |
+ FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL |
+ FM10K_SW_BIST_CTRL_BIST_RUN_EPL);
+ fm10k_write_flush(sw);
+ usec_delay(800);
+
+ fm10k_write_switch_reg64(sw, FM10K_SW_BIST_CTRL, 0);
+ fm10k_write_flush(sw);
+
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data &= ~(FM10K_SW_SOFT_RESET_SWITCH_RESET |
+ FM10K_SW_SOFT_RESET_EPL_RESET);
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+ fm10k_write_flush(sw);
+ /* ensure switch reset is deasserted for at least 100ns */
+ usec_delay(1);
+
+ sw->epl_sbus = fm10k_sbus_attach(sw, "EPL", FM10K_SW_SBUS_EPL_CFG);
+ if (sw->epl_sbus == NULL) {
+ error = -1;
+ goto done;
+ }
+
+ /* Clear non-BIST accessible pause state memories */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 0), 0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_CM_EGRESS_PAUSE_COUNT(i, 1), 0);
+ }
+
+ /* Initialize RXQ_MCAST list */
+ for (i = 0; i < FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS(i),
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_HEAD_PAGE, i) |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_STORAGE_POINTERS_TAIL_PAGE, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_RXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_RXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_RXQ_STORAGE_POINTERS_ENTRIES));
+ }
+ /* Initialize TXQ list */
+ for (i = 0; i < FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_HEAD_PERQ_HEAD, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL0_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL0_PERQ_TAIL, i));
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_TAIL1_PERQ(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_TXQ_TAIL1_PERQ_TAIL, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_TXQ_FREELIST_INIT_ENTRIES -
+ FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TXQ_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_TXQ_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_SCHED_TXQ_HEAD_PERQ_ENTRIES));
+ }
+ /* Initialize free segment list */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_SSCHED_RX_PERPORT(i),
+ FM10K_SW_MAKE_REG_FIELD(SCHED_SSCHED_RX_PERPORT_NEXT, i));
+ }
+ for (i = 0;
+ i < FM10K_SW_SCHED_FREELIST_INIT_ENTRIES -
+ FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_FREELIST_INIT,
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_FREELIST_INIT_ADDRESS,
+ i + FM10K_SW_LOGICAL_PORTS_MAX));
+ }
+ /* Disable switch scan chain */
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCAN_DATA_IN,
+ FM10K_SW_SCAN_DATA_IN_UPDATE_NODES |
+ FM10K_SW_SCAN_DATA_IN_PASSTHRU);
+
+ error = fm10k_switch_configure_epls(hw, sw);
+ if (error)
+ goto done;
+
+ /*
+ * for now configure store-and-forward between PEPs and external
+ * ports regardless of relative speeds
+ */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_SAF_MATRIX(all_lports[i].lport),
+ FM10K_SW_SAF_MATRIX_ENABLE_SNF_ALL_PORTS);
+ }
+
+ /* Disable MTU violation trap */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SYS_CFG_1);
+ data &= ~FM10K_SW_SYS_CFG_1_TRAP_MTU_VIOLATIONS;
+ fm10k_write_switch_reg(sw, FM10K_SW_SYS_CFG_1, data);
+
+ /* Disable ingress VLAN filtering and learning */
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_3(all_lports[i].lport), 0);
+ }
+ /*
+ * Make all ports members of every VLAN, and configure every VLAN to
+ * use MST instance 0.
+ */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_INGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ fm10k_write_switch_reg128(sw, FM10K_SW_INGRESS_VID_TABLE(i),
+ FM10K_SW_INGRESS_VID_TABLE_REFLECT, data64);
+ }
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_VID_TABLE_MEMBERSHIP
+ (all_lports[i].lport);
+ for (i = 0; i < FM10K_SW_EGRESS_VID_TABLE_ENTRIES; i++)
+ fm10k_write_switch_reg128(sw,
+ FM10K_SW_EGRESS_VID_TABLE(i), 0, data64);
+
+ /* Init MOD_VLAN_TAG_VID1_MAP */
+ for (i = 0; i < FM10K_SW_INGRESS_VID_TABLE_ENTRIES; i++) {
+ data64 = i;
+ data64 = data64 << 48;
+ fm10k_write_switch_reg64(sw,
+ 0xE80000 + 0x2 * i + 0x20000, data64);
+ }
+
+ /* Configure MST instance 0 to forward for all ports */
+ data64 = 0;
+ for (i = 0; i < num_lports; i++)
+ data64 |=
+ FM10K_SW_EGRESS_MST_TABLE_FORWARDING
+ (all_lports[i].lport);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_EGRESS_MST_TABLE(0), data64);
+
+ data64 = 0;
+ data64_2 = 0;
+ for (i = 0; i < num_lports; i++) {
+ if (all_lports[i].lport <
+ FM10K_SW_INGRESS_MST_TABLE_PORTS_PER_TABLE) {
+ data64 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ } else {
+ data64_2 |=
+ FM10K_SW_MAKE_REG_FIELD_IDX64
+ (INGRESS_MST_TABLE_STP_STATE,
+ all_lports[i].lport,
+ FM10K_SW_INGRESS_MST_TABLE_STP_STATE_FORWARD,
+ all_lports[i].lport, all_lports[i].lport);
+ }
+ }
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(0, 0), data64);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INGRESS_MST_TABLE(1, 0), data64_2);
+
+ for (i = 0; i < num_lports; i++) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_VLAN1_TAG(0) |
+ FM10K_SW_PARSER_PORT_CFG_1_VLAN2_TAG(0);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport), data64);
+
+ /*
+ * Configure tags for f-tagged lports
+ */
+ if (all_lports[i].has_ftag) {
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_1_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_1(all_lports[i].lport),
+ data64);
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_MOD_PER_PORT_CFG_2_FTAG;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_PER_PORT_CFG_2(all_lports[i].lport),
+ data64);
+ }
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport));
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L3;
+ data64 |= FM10K_SW_PARSER_PORT_CFG_2_PARSE_L4;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_PARSER_PORT_CFG_2(all_lports[i].lport), data64);
+ }
+
+ /*
+ * Assign default SGLORTs for the EPL ports in the PARSER config.
+ * This isn't necessary for the PEPs as for those, as all frames
+ * ingressing from PEPs are tagged with an SGLORT that is derived
+ * from a per-tx-queue setting.
+ *
+ * The PORT_CFG_ISL register offset is determined by logical port
+ * number and the register contents determined by the corresponding
+ * SGLORT.
+ */
+ for (i = 0; i < FM10K_SW_EPLS_SUPPORTED; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_PORT_CFG_ISL
+ (fm10k_sw.epl_map[i].logical_port),
+ FM10K_SW_MAKE_REG_FIELD
+ (PORT_CFG_ISL_SGLORT,
+ fm10k_sw.epl_map[i].glort));
+ }
+
+ /*
+ * FFU counter, 11.10.3.5 POLICER_CFG[0..3]
+ */
+ fm10k_write_switch_reg64(sw,
+ 0xE40000 + 0x2 * FM10K_SW_FFU_CNT_BANK + 0x11000,
+ FM10K_SW_POLICER_LAST);
+
+ /*
+ * 11.10.3.1 POLICER_CFG_4K[0..1][0..4095]
+ * 11.10.3.2 POLICER_CFG_512[0..1][0..511]
+ */
+ for (i = 0; i < FM10K_SW_FFU_CNT_MAX; i++) {
+ if (FM10K_SW_FFU_CNT_BANK < 2) {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x2000 * FM10K_SW_FFU_CNT_BANK +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x0);
+ } else {
+ fm10k_read_switch_reg64(sw,
+ 0xE40000 + 0x400 * (FM10K_SW_FFU_CNT_BANK - 2) +
+ 0x2 * (i + FM10K_SW_FFU_CNT_START) + 0x4000);
+ }
+ }
+
+ /*
+ * FFU
+ *
+ * The FFU is programmed to match on SGLORT and to set the DGLORT to
+ * a unique value corresponding to the given SGLORT. One TCAM entry
+ * is required per host port per PEP and one per external interface.
+ * Only slice 0 is used.
+ */
+ if (fm10k_ffu_init(sw, sw->dpdk_cfg) < 0)
+ return -1;
+
+ /*
+ * Program the segment scheduler (see tables at top)
+ *
+ * The TX and RX schedules are the same. Page 0 is used.
+ */
+ if (cfg->ext_port_speed == 40 || cfg->ext_port_speed == 100)
+ is_quad = 1;
+ else
+ is_quad = 0;
+ for (i = 0; i < sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0]); i++) {
+ /*
+ * In addition to explicit idle cycles, non-quad port
+ * entries are converted to idle cycles if the interfaces
+ * are configured in quad-port mode.
+ */
+ if (fm10k_sched_prog[i].idle ||
+ (!fm10k_sched_prog[i].quad && is_quad))
+ data = FM10K_SW_SCHED_SCHEDULE_IDLE;
+ else
+ data = FM10K_SW_SCHED_SCHEDULE_ENTRY
+ (fm10k_sched_prog[i].phys,
+ fm10k_sched_prog[i].log,
+ (fm10k_sched_prog[i].quad == FM10K_SW_QUAD_40_100) ?
+ is_quad : fm10k_sched_prog[i].quad);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_RX_SCHEDULE(0, i), data);
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_SCHED_TX_SCHEDULE(0, i), data);
+ }
+
+ fm10k_write_switch_reg(sw, FM10K_SW_SCHED_SCHEDULE_CTRL,
+ FM10K_SW_SCHED_SCHEDULE_CTRL_RX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_RX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1) |
+ FM10K_SW_SCHED_SCHEDULE_CTRL_TX_ENABLE |
+ FM10K_SW_MAKE_REG_FIELD
+ (SCHED_SCHEDULE_CTRL_TX_MAX_INDEX,
+ (sizeof(fm10k_sched_prog) /
+ sizeof(fm10k_sched_prog[0])) - 1));
+
+ /* Per 5.7.10.4 */
+ watermark = FM10K_SW_MEM_POOL_SEGS_MAX -
+ ((sw->info->num_peps +
+ FM10K_SW_EPLS_SUPPORTED * FM10K_SW_EPL_LANES) *
+ FM10K_SW_HOWMANY(FM10K_SW_PACKET_SIZE_MAX,
+ FM10K_SW_MEM_POOL_SEG_SIZE, FM10K_SW_MEM_POOL_SEG_SIZE)) -
+ FM10K_SW_MEM_POOL_SEGS_RSVD;
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_WM,
+ FM10K_SW_MAKE_REG_FIELD(CM_GLOBAL_WM_WATERMARK, watermark));
+ fm10k_write_switch_reg(sw, FM10K_SW_CM_GLOBAL_CFG,
+ FM10K_SW_CM_GLOBAL_CFG_WM_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_GEN_SWEEP_EN |
+ FM10K_SW_CM_GLOBAL_CFG_PAUSE_REC_SWEEP_EN |
+ FM10K_SW_MAKE_REG_FIELD
+ (CM_GLOBAL_CFG_NUM_SWEEPER_PORTS,
+ FM10K_SW_LOGICAL_PORTS_MAX));
+
+ /* Configure stats counters */
+ data = FM10K_SW_RX_STATS_CFG_ENABLE_ALL_BANKS;
+ data64 =
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_7 |
+ FM10K_SW_MOD_STATS_CFG_ENABLE_GROUP_8;
+ for (i = 0; i < num_lports; i++) {
+ fm10k_write_switch_reg(sw,
+ FM10K_SW_RX_STATS_CFG(all_lports[i].lport),
+ data | ((all_lports[i].has_ftag) ?
+ FM10K_SW_MAKE_REG_FIELD
+ (RX_STATS_CFG_PER_FRAME_ADJUSTMENT,
+ FM10K_SW_FTAG_SIZE) :
+ 0));
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_MOD_STATS_CFG(all_lports[i].lport),
+ data64);
+ }
+
+ /* Transition switch to ready */
+ data = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ data |= FM10K_SW_SOFT_RESET_SWITCH_READY;
+ fm10k_write_switch_reg(sw, FM10K_SW_SOFT_RESET, data);
+
+ done:
+ return (error);
+}
+
+
+static void
+fm10k_switch_leds_init(struct fm10k_switch *sw)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ unsigned int i;
+ uint8_t addr;
+ uint32_t data;
+ int max;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ /* MODE1 - put into sleep mode to ensure it is brought out
+ * of sleep without violating the oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x00, 0x00);
+
+ /* Wait for oscillator to stabilize after coming out of sleep */
+ fm10k_udelay(500);
+
+ /* MODE2 - group control is blinking, open drain outputs,
+ * OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x01, 0x20);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x02, 0xff);
+
+ /* PWM1 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x03, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x13, max - 1);
+
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /*
+ * PCA9635 initialization
+ * only differences from defaults are noted
+ */
+
+ addr = 0x6a;
+ for (i = 0; i < 2; i++) {
+ /* MODE1 - put into sleep mode to ensure it is
+ * brought out of sleep without violating the
+ * oscillator startup wait
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x10);
+ fm10k_udelay(10);
+
+ /* MODE1 - normal mode, disable all call */
+ fm10k_i2c_write16(sw->i2c, addr, 0x00, 0x00);
+
+ /* MODE2 - group control is blinking, open drain
+ * outputs, OE high -> LEDn = 0
+ */
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x20);
+
+ /* Wait for oscillator to stabilize
+ * after coming out of sleep
+ */
+ fm10k_udelay(500);
+
+ /* PWM0 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x02, 0xff);
+
+ /* PWM3 - 100% duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x05, 0xff);
+
+ /* GRPPWM - 50% blink duty cycle */
+ fm10k_i2c_write16(sw->i2c, addr, 0x12, 0x80);
+
+ /* GRPFREQ - FM10K_LED_BLINKS_PER_SECOND */
+ if (24 / FM10K_LED_BLINKS_PER_SECOND > 1)
+ max = 24 / FM10K_LED_BLINKS_PER_SECOND;
+ else
+ max = 1;
+ fm10k_i2c_write16(sw->i2c, addr, 0x13, max - 1);
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x88);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * PCA9538 initialization
+ * only differences from defaults are noted
+ */
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = 0x62;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x65;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0xf0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ addr = 0x66;
+ fm10k_i2c_write16(sw->i2c, addr, 0x03, 0x0);
+ fm10k_i2c_write16(sw->i2c, addr, 0x01, 0x0);
+
+ /* set LEC_CFG */
+ data = fm10k_read_switch_reg(sw, 0xc2b);
+ data |= 1 << 24;
+ fm10k_write_switch_reg(sw, 0xc2b, data);
+
+ /* port from rdifd, LED */
+ fm10k_gpio_output_set(sw, 4, 0);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ default:
+ FM10K_SW_ERR("don't know how to operate LEDs for this card "
+ "(subvendor=0x%04x subdevice=0x%04x)",
+ cfg->subvendor, cfg->subdevice);
+ break;
+ }
+}
+
+
+static unsigned int
+fm10k_switch_pca9635_led_bits(uint8_t led_flags)
+{
+ unsigned int bits;
+
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ if (led_flags & FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE)
+ bits = 0x3; /* group blink */
+ else
+ bits = 0x1; /* full on */
+ } else {
+ bits = 0; /* off */
+ }
+ return (bits);
+}
+
+static void
+fm10k_switch_process_leds(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_ports *ports = sw->ext_ports;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ unsigned int num_ports = ports->num_ports;
+ uint32_t data;
+ uint8_t update_port[num_ports];
+ uint8_t led_flags = 0, read;
+ uint8_t addr;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+
+ if (sw->master_hw->sw_addr == NULL) {
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ return;
+ }
+
+ for (i = 0; i < num_ports; i++) {
+ port = &ports->ports[i];
+ data = fm10k_read_switch_reg(sw,
+ FM10K_SW_EPL_LED_STATUS(port->eplno));
+ led_flags =
+ ((data & FM10K_SW_EPL_LED_STATUS_PORT_LINK_UP
+ (port->first_lane)) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_UP : 0) |
+ ((data &
+ (FM10K_SW_EPL_LED_STATUS_PORT_TRANSMITTING
+ (port->first_lane) |
+ FM10K_SW_EPL_LED_STATUS_PORT_RECEIVING
+ (port->first_lane))) ?
+ FM10K_SW_EXT_PORT_LED_FLAG_ACTIVE : 0);
+ update_port[i] = (led_flags != port->last_led_flags);
+ port->last_led_flags = led_flags;
+ }
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ if (!(update_port[0] || update_port[1])) {
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (ports->ports[0].last_led_flags) |
+ (fm10k_switch_pca9635_led_bits
+ (ports->ports[1].last_led_flags) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs operating
+ * as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (update_port[i] == 0)
+ continue;
+
+ addr = (i == 0) ? 0x6a : 0x69;
+
+ port = &ports->ports[i];
+ led_flags =
+ fm10k_switch_pca9635_led_bits
+ (port->last_led_flags);
+
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ led_flags = 0;
+ addr = 0x62;
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ case 25:
+ led_flags |= 0x6 << (4 * i); /* 100G */
+ break;
+ case 40:
+ case 10:
+ led_flags |= 0x4 << (4 * i); /* 40G */
+ break;
+ default:
+ led_flags = 0;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ }
+
+ if (update_port[0] || update_port[1]) {
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ if (update_port[0])
+ led_flags |= read & 0xf0;
+ else
+ led_flags |= read & 0xf;
+ fm10k_i2c_write16(sw->i2c, addr, 0x1, led_flags);
+ fm10k_i2c_read8_ext(sw->i2c, addr, 0x1, &read);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ for (i = 0; i < 2; i++) {
+ if (!update_port[i])
+ continue;
+ /*
+ * 00000000: No link
+ * 00000001: 10G Port 0
+ * 00000010: 40G Port 0
+ * 00000100: 25G Port 0
+ * 00001000: 100G Port 0
+ */
+ addr = (i == 0) ? 0x62 : 0x66;
+ port = &ports->ports[i];
+ if (port->last_led_flags &
+ FM10K_SW_EXT_PORT_LED_FLAG_UP) {
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ led_flags = 0x08; /* 100G */
+ break;
+ case 40:
+ led_flags = 0x02; /* 40G */
+ break;
+ }
+ } else {
+ led_flags = 0; /* off */
+ }
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x1, led_flags);
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+static void *
+fm10k_switch_leds_update(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+
+ while (sw->detaching == 0) {
+ fm10k_switch_process_leds(ctx);
+ usec_delay(FM10K_LED_POLL_INTERVAL_MS * 1000);
+ }
+ return NULL;
+}
+
+static void
+fm10k_switch_leds_off(struct fm10k_switch *sw, struct fm10k_ext_ports *ports)
+{
+ struct fm10k_device_info *cfg = sw->info;
+ struct fm10k_ext_port *port;
+ unsigned int i;
+ uint8_t led_flags;
+ uint8_t addr;
+
+ switch (FM10K_SW_CARD_ID(cfg->subvendor, cfg->subdevice)) {
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS41):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QS43):
+ case FM10K_SW_CARD(SILICOM, PE340G2DBIR_QL4):
+ case FM10K_SW_CARD(SILICOM_RB, PE340G2DBIR_QL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x04);
+
+ led_flags =
+ fm10k_switch_pca9635_led_bits(0) |
+ (fm10k_switch_pca9635_led_bits(0) << 2);
+
+ fm10k_i2c_write16(sw->i2c, 0x6a, 0x14, led_flags);
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIR_QXSL4_REV_2):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRL_QXSL4):
+ case FM10K_SW_CARD(SILICOM, PE3100G2DQIRM_QXSL4):
+ FM10K_SW_I2C_REQ_LOCK(sw->i2c);
+ /*
+ * Set up the first PCA9545 mux so we can get at the PCA9635s
+ * that the LED control lines are connected to.
+ */
+ fm10k_i2c_write8(sw->i2c, 0x70, 0x01);
+
+ /* will need to update for QSFPs
+ * operating as four independent lanes/ports
+ */
+ addr = 0x6a;
+ led_flags = fm10k_switch_pca9635_led_bits(0);
+ for (i = 0; i < 2; i++) {
+ port = &ports->ports[i];
+ switch (port->lane_speed * port->num_lanes) {
+ case 100:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags);
+ break;
+ case 40:
+ fm10k_i2c_write16(sw->i2c,
+ addr, 0x14, led_flags << 6);
+ break;
+ }
+
+ addr = 0x69;
+ }
+ FM10K_SW_I2C_REQ_UNLOCK(sw->i2c);
+ break;
+ }
+}
+
+
+static void *
+fm10k_switch_process_intr(void *ctx)
+{
+ struct fm10k_switch *sw = ctx;
+ struct fm10k_ext_ports *ports;
+ uint64_t gid;
+ uint64_t reenable_mask;
+ uint16_t epl_mask;
+
+ while (sw->detaching == 0) {
+ sem_wait(&sw->intr_tq);
+ /*
+ * Mask off all global interrupt detect interrupts
+ * toward PCIe to ensure that the PEP sees the
+ * interrupt condition clear so that all subsequent
+ * events will be recognized. The same result could
+ * be achieved by masking off all block-internal
+ * interrupt sources in all blocks prior to handling
+ * any global interrupt detect interrupts, but the
+ * approach taken here makes the interrupt processing
+ * easier to decompose and increases the likelihood that
+ * more work will be coalesced into fewer interrupts.
+ */
+ FM10K_SW_TRACE("masking off PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ /*
+ * sw->ext_ports will be set to NULL during detach
+ * to indicate that we are to no longer process EPL
+ * interrupts.
+ */
+ ports = sw->ext_ports;
+ /*
+ * sw->sm_mailbox_enabled will be set to 0 during
+ * detach to indicate that we are to no longer process
+ * PEP interrupts.
+ */
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ FM10K_SW_INTERRUPT_MASK_PCIE_ALL);
+ gid = fm10k_read_switch_reg64(sw,
+ FM10K_SW_GLOBAL_INTERRUPT_DETECT);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ reenable_mask = 0;
+ if (ports) {
+ epl_mask = fm10k_ext_ports_epl_intrs(ports, gid);
+ reenable_mask |=
+ FM10K_SW_MAKE_REG_FIELD64
+ (INTERRUPT_MASK_PCIE_EPL, epl_mask);
+ }
+
+ if (gid & FM10K_SW_GLOBAL_INTERRUPT_DETECT_I2C)
+ fm10k_i2c_intr(sw->i2c);
+
+ reenable_mask |= FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+
+ FM10K_SW_TRACE("re-enabling PCIe gid interrupts");
+ FM10K_SW_SWITCH_LOCK(sw);
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno),
+ ~reenable_mask);
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ }
+ return NULL;
+}
+
+void
+fm10k_switch_intr(struct fm10k_hw *hw)
+{
+ if (hw)
+ sem_post(&fm10k_sw.intr_tq);
+}
+
+
+static void
+fm10k_switch_enable_interrupts(struct fm10k_switch *sw)
+{
+ uint64_t data64;
+ unsigned int i;
+
+ data64 = fm10k_read_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno));
+
+ /* enable interrupts from all EPLs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_EPL,
+ ~sw->ext_ports->epl_mask, data64);
+
+ /* enable interrupts from all non-master PEPs in use */
+ FM10K_SW_REPLACE_REG_FIELD64(data64, INTERRUPT_MASK_PCIE_PCIE,
+ ~sw->pep_mask, data64);
+
+ /* enable I2C interrupt */
+ data64 &= ~FM10K_SW_INTERRUPT_MASK_PCIE_I2C;
+ fm10k_write_switch_reg64(sw,
+ FM10K_SW_INTERRUPT_MASK_PCIE(sw->pepno), data64);
+
+ /* enable outbound mailbox interrupts from all non-master PEPs */
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (sw->pep_mask & (1 << i))
+ fm10k_write_switch_reg(sw, FM10K_SW_PCIE_GLOBAL(i, IM),
+ ~FM10K_SW_IM_MAILBOX);
+ }
+ FM10K_WRITE_REG(sw->master_hw,
+ FM10K_SW_EIMR, FM10K_SW_EIMR_FIELD(SWITCH_INT, ENABLE));
+}
+
+
+static void
+fm10k_switch_detach(struct fm10k_hw *hw)
+{
+ struct fm10k_ext_ports *ports;
+ struct fm10k_switch *sw = &fm10k_sw;
+
+ if (hw == NULL || sw == NULL)
+ return;
+
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->detaching = 1;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+
+ if (sw->ext_ports) {
+ ports = sw->ext_ports;
+ /*
+ * Ensure that any further switch interrupts will not
+ * process sw->ext_ports before detaching them.
+ */
+ FM10K_SW_SWITCH_LOCK(sw);
+ sw->ext_ports = NULL;
+ FM10K_SW_SWITCH_UNLOCK(sw);
+ fm10k_switch_leds_off(sw, ports);
+ fm10k_ext_ports_detach(ports);
+ }
+
+ if (sw->epl_sbus)
+ fm10k_sbus_detach(sw->epl_sbus);
+
+ /*
+ * Detach i2c after ext_ports so ext_ports can use i2c to disable
+ * phys.
+ */
+ if (sw->i2c)
+ fm10k_i2c_detach(sw->i2c);
+
+ fm10k_hw_eicr_disable_source(sw, FM10K_SW_EICR_SWITCH_INT);
+
+ pthread_join(sw->intr_task, NULL);
+ pthread_join(sw->led_task, NULL);
+ pthread_join(sw->stats_task, NULL);
+}
+
+
+static unsigned int
+fm10k_switch_get_fabric_clock(struct fm10k_switch *sw)
+{
+ uint32_t pll_fabric;
+ unsigned int freq;
+
+ pll_fabric = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_LOCK);
+
+ switch (FM10K_SW_REG_FIELD(pll_fabric, PLL_FABRIC_FREQSEL)) {
+ case FM10K_SW_PLL_FABRIC_FREQSEL_CTRL: {
+ uint32_t ctrl;
+ unsigned int refdiv, fbdiv4, fbdiv255;
+
+ ctrl = fm10k_read_switch_reg(sw, FM10K_SW_PLL_FABRIC_CTRL);
+ refdiv = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_REFDIV);
+ fbdiv4 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV4);
+ fbdiv255 = FM10K_SW_REG_FIELD(ctrl, PLL_FABRIC_FBDIV255);
+
+ freq = (15625 * 4 * fbdiv255 * (1 + fbdiv4)) / (refdiv * 100);
+ break;
+ }
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F600:
+ freq = 600;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F500:
+ freq = 500;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F400:
+ freq = 400;
+ break;
+ case FM10K_SW_PLL_FABRIC_FREQSEL_F300:
+ freq = 300;
+ break;
+ default:
+ freq = 0;
+ break;
+ }
+
+ return freq;
+}
+
+static unsigned int
+fm10k_switch_get_sku(struct fm10k_switch *sw)
+{
+ uint32_t fuse_data;
+
+ fuse_data = fm10k_read_switch_reg(sw, FM10K_SW_FUSE_DATA_0);
+ return FM10K_SW_REG_FIELD(fuse_data, FUSE_SKU);
+}
+
+static const char *
+fm10k_switch_get_sku_name(unsigned int sku)
+{
+ const char *name;
+
+ switch (sku) {
+ case FM10K_SW_FUSE_SKU_FM10840:
+ name = "FM10840";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10420:
+ name = "FM10420";
+ break;
+ case FM10K_SW_FUSE_SKU_FM10064:
+ name = "FM10064";
+ break;
+ default:
+ name = "FM10xxx-unknown";
+ break;
+ }
+
+ return name;
+}
+
+
+static void
+fm10k_switch_describe(struct fm10k_switch *sw)
+{
+ unsigned int i, pair;
+ unsigned int reset_state, active_state;
+ uint32_t device_cfg, resets;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("switch: device is %s, fabric clock %u MHz",
+ fm10k_switch_get_sku_name(fm10k_switch_get_sku(sw)),
+ fm10k_switch_get_fabric_clock(sw));
+
+ device_cfg = fm10k_read_switch_reg(sw, FM10K_SW_DEVICE_CFG);
+ FM10K_SW_INFO("switch: 100G Ethernet is %s",
+ (device_cfg & FM10K_SW_DEVICE_CFG_PCIE_100G_DIS) ?
+ "disabled" : "enabled");
+ switch (FM10K_SW_REG_FIELD(device_cfg, DEVICE_CFG_FEATURE)) {
+ case FM10K_SW_DEVICE_CFG_PCIE_FULL:
+ FM10K_SW_INFO("switch: PEPs 0-7 support 2x4 and 1x8 "
+ "modes, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_HALF:
+ FM10K_SW_INFO("switch: PEPs 0-3 support 2x4 and 1x8 "
+ "modes, PEPs 4 and 6 are 1x4, PEP 8 is x1");
+ break;
+ case FM10K_SW_DEVICE_CFG_PCIE_BASIC:
+ FM10K_SW_INFO("switch: PEP 0 supports, PEP 8 is x1, "
+ "only one can be used");
+ break;
+ }
+
+ resets = fm10k_read_switch_reg(sw, FM10K_SW_SOFT_RESET);
+ for (i = 0; i < FM10K_SW_PEPS_MAX; i++) {
+ if (!(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_EN(i)))
+ continue;
+
+ pair = i / 2;
+ reset_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_RESET(i));
+ active_state = !!(resets & FM10K_SW_SOFT_RESET_PCIE_ACTIVE(i));
+ if (pair < FM10K_SW_DEVICE_CFG_PCIE_MODE_PAIRS &&
+ !(device_cfg & FM10K_SW_DEVICE_CFG_PCIE_MODE_2X4(pair))) {
+ if (i % 2 == 0)
+ FM10K_SW_INFO("switch: PEP[%u,%u] is enabled in 1x8 "
+ "mode (Reset=%u, Active=%u)", i, i + 1,
+ reset_state, active_state);
+ else
+ FM10K_SW_INFO("switch: PEP[%u] unexpectedly enabled when "
+ "PEP[%u,%u] is in 1x8 mode (Reset=%u, "
+ "Active=%u)", i, i - 1, i, reset_state,
+ active_state);
+ } else {
+ FM10K_SW_INFO("switch: PEP[%u] is enabled in 1x4 "
+ "mode (Reset=%u, Active=%u)", i, reset_state,
+ active_state);
+ }
+ }
+}
+
+
+static struct fm10k_switch *
+fm10k_switch_attach(struct fm10k_hw *hw, struct fm10k_switch *sw)
+{
+ int i;
+ int error = 0;
+
+ /*
+ * apparently no way to determine one's own PEP number.
+ */
+ switch (sw->info->num_peps) {
+ case 1:
+ sw->pepno = 0;
+ break;
+
+ case 2:
+ /*
+ * assumption is using even numbered PEPs
+ * starting with 0, and the highest numbered PEP
+ * is the master
+ */
+ sw->pepno = 2;
+ for (i = 0; i < sw->info->num_peps - 1; i++)
+ sw->pep_mask |= (1 << (i * 2));
+ break;
+
+ case 4:
+ sw->pepno = 6;
+ sw->pep_mask = 0x7f;
+ break;
+
+ default:
+ sw->pepno = 0;
+ }
+
+ /* When not zero, initialize serdes in 'near loopback' mode */
+ sw->serdes_loopback = 0;
+
+ pthread_mutex_init(&sw->lock, NULL);
+
+ fm10k_switch_determine_epls(sw);
+
+ sw->ext_ports = fm10k_ext_ports_attach(sw);
+ if (sw->ext_ports == NULL)
+ goto fail;
+
+ /* label all external ports with their logical port numbers */
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ sw->ext_ports->ports[i].lport = sw->epl_map[i].logical_port;
+
+ /* interrupt */
+ sem_init(&sw->intr_tq, 0, 0);
+ pthread_create(&sw->intr_task, NULL, fm10k_switch_process_intr, sw);
+
+ fm10k_switch_enable_interrupts(sw);
+ fm10k_switch_describe(sw);
+
+ sw->i2c = fm10k_i2c_attach(sw);
+ if (sw->i2c == NULL)
+ goto fail;
+
+ error = fm10k_switch_init(hw, sw);
+ if (error)
+ goto fail;
+
+ for (i = 0; i < sw->ext_ports->num_ports; i++)
+ fm10k_ext_port_up(&sw->ext_ports->ports[i]);
+
+ usec_delay(10000);
+ fm10k_switch_leds_init(sw);
+
+ pthread_create(&sw->led_task, NULL, &fm10k_switch_leds_update, sw);
+ pthread_create(&sw->stats_task, NULL, &fm10k_switch_process_stats, sw);
+ sw->inited = 1;
+ return sw;
+fail:
+ if (sw != NULL)
+ fm10k_switch_detach(hw);
+ return NULL;
+}
+
+struct fm10k_switch*
+fm10k_switch_get(void)
+{
+ return &fm10k_sw;
+}
+
+static int
+fm10k_switch_dpdk_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].pf_no == pf_no)
+ return i;
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_mapped_ext_port_get(struct fm10k_switch *sw, int pf_no)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ for (i = 0; i < FM10K_SW_EXT_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ if (map->map_no[0] == pf_no)
+ return i;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ if (map->map_no[0] == pf_no || map->map_no[1] == pf_no)
+ return i;
+ }
+ }
+ return -1;
+}
+
+static int
+fm10k_switch_start(struct fm10k_switch *sw,
+ struct fm10k_hw *master_hw, eth_fm10k_dev_init_half_func *func)
+{
+ int i, j;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_cfg_port_pf_map *dpdk_map;
+ int dpdk_port_no, ext_port_no = 0;
+ int pf_no;
+
+ sw->info = fm10k_get_device_info(master_hw);
+ sw->master_hw = master_hw;
+ sw->pep_map = fm10k_pep_port_map;
+ sw->epl_map = fm10k_epl_port_map;
+
+ sw->info->ext_port_speed = sw->dpdk_cfg->ext_port_speed;
+ fm10k_switch_set_sched_prog();
+ fm10k_switch_attach(master_hw, &fm10k_sw);
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type != FM10K_CONFIG_DPDK_PF ||
+ map->type != FM10K_CONFIG_PORT_MAP_PFS)
+ continue;
+
+ for (j = 0; j < 2; j++) {
+ pf_no = map->map_no[j];
+ dpdk_port_no =
+ fm10k_switch_dpdk_port_get(sw, pf_no);
+ dpdk_map =
+ &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ dpdk_map->type = FM10K_CONFIG_PORT_MAP_PFSS;
+ dpdk_map->map_no[0] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[0];
+ dpdk_map->map_no[1] =
+ sw->dpdk_cfg->dpdk_port_map[i].map_no[1];
+ }
+ }
+ }
+
+ /* Make sure Switch Manager is ready before going forward. */
+ for (i = 0; i < FM10K_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (master_hw->switch_ready)
+ break;
+ /* Delay some time to acquire async LPORT_MAP info. */
+ rte_delay_us(FM10K_WAIT_SWITCH_MSG_US);
+ }
+
+ if (!master_hw->switch_ready) {
+ PMD_INIT_LOG(ERR, "switch is not ready");
+ return -1;
+ }
+
+ /* do initialize all ports, after switch is ready */
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ uint32_t sglort;
+ struct fm10k_dpdk_port *port = &sw->dpdk_cfg->ports[i];
+ struct fm10k_hw *hw = port->hw;
+
+ if (hw == NULL)
+ break;
+
+ map = &sw->dpdk_cfg->ext_port_map[ext_port_no];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ pf_no = fm10k_switch_dpdk_pf_no_get(hw);
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ sglort = fm10k_switch_pf_glort_get(pf_no);
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ ext_port_no =
+ fm10k_switch_mapped_ext_port_get
+ (sw, pf_no);
+ sglort =
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]);
+ } else {
+ FM10K_SW_ERR("Unknown mapped port type %d!",
+ port->type);
+ return -1;
+ }
+ hw->mac.dglort_map = sglort | 0xffff0000;
+ hw->switch_ready = master_hw->switch_ready;
+ func(hw);
+ }
+ }
+ return 0;
+}
+
+
+static int
+fm10k_switch_dpdk_port_reg(struct fm10k_switch *sw,
+ struct fm10k_hw *hw, void *rte_dev, uint8_t is_pf, bool master)
+{
+ int i;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg == NULL && is_pf) {
+ if (fm10k_config_init(sw, hw) < 0)
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ if (master && port->hw &&
+ port->type == FM10K_CONFIG_DPDK_PF) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (hw->mac.addr[5] - port->hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ if (port->hw == NULL) {
+ port->hw = hw;
+ port->rte_dev = rte_dev;
+ fm10k_flow_list_init(&port->flow_list);
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (is_pf) {
+ sw->dpdk_cfg->pf_bind++;
+ port->type = FM10K_CONFIG_DPDK_PF;
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ } else {
+ port->type = FM10K_CONFIG_DPDK_VF;
+ }
+ if (master)
+ sw->dpdk_cfg->master_hw = hw;
+ if (sw->dpdk_cfg->master_hw) {
+ port->pf_no =
+ sw->dpdk_cfg->pf_max - 1 -
+ (sw->dpdk_cfg->master_hw->mac.addr[5] -
+ hw->mac.addr[5]);
+ sw->dpdk_cfg->pf_hw[port->pf_no] = port->hw;
+ }
+
+ return 0;
+ }
+ }
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_port_no_get(struct fm10k_hw *hw)
+{
+ int i;
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == NULL)
+ break;
+
+ if (fm10k_sw.dpdk_cfg->ports[i].hw == hw)
+ return i;
+ }
+ return -1;
+}
+
+void *
+fm10k_switch_dpdk_port_rte_dev_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].rte_dev;
+}
+
+struct fm10k_flow_list *
+fm10k_switch_dpdk_port_flow_list_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+
+ if (port_no < 0)
+ return NULL;
+
+ return fm10k_switch_get()->dpdk_cfg->ports[port_no].flow_list;
+}
+
+static int
+fm10k_switch_dpdk_cfg_check(struct fm10k_switch *sw)
+{
+ int i;
+ bool need_default = true;
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (sw->dpdk_cfg->master_hw == NULL) {
+ FM10K_SW_ERR("Master PF is not bound!!!");
+ return -1;
+ }
+
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ port = &sw->dpdk_cfg->ports[i];
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF &&
+ map->type != FM10K_CONFIG_PORT_MAP_NULL) {
+ need_default = false;
+ break;
+ }
+ }
+
+ if (need_default) {
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (port->type == FM10K_CONFIG_DPDK_PF) {
+ map->type = FM10K_CONFIG_PORT_MAP_PF;
+ map->map_no[0] = i;
+ }
+ }
+ }
+
+ return 0;
+}
+
+
+static void
+fm10k_switch_dpdk_cfg_describe(struct fm10k_switch *sw)
+{
+ int i;
+ struct fm10k_cfg_port_pf_map *map;
+
+ if (!fm10k_config_check_debug(sw->dpdk_cfg, FM10K_CONFIG_DEBUG_CONFIG))
+ return;
+
+ FM10K_SW_INFO("--- FM10K DYNAMIC CONFIG ---\n");
+ FM10K_SW_INFO(" PF Bind : %d\n", sw->dpdk_cfg->pf_bind);
+
+ FM10K_SW_INFO("--- PF ---\n");
+ for (i = 0; i < FM10K_SW_PEP_PORTS_MAX; i++) {
+ uint8_t *mac;
+ struct rte_eth_dev *dev;
+ struct rte_pci_device *pdev;
+
+ if (sw->dpdk_cfg->pf_hw[i] == NULL)
+ continue;
+ dev = (struct rte_eth_dev *)
+ fm10k_switch_dpdk_port_rte_dev_get
+ (sw->dpdk_cfg->pf_hw[i]);
+ pdev = RTE_ETH_DEV_TO_PCI(dev);
+ mac = sw->dpdk_cfg->pf_hw[i]->mac.addr;
+ FM10K_SW_INFO(" PF%d : Logical %d Glort %#x "
+ "PCI Addr %02x:%02x.%x "
+ "MAC Addr %02x:%02x:%02x:%02x:%02x:%02x\n",
+ i, fm10k_pep_port_map[i].logical_port,
+ fm10k_switch_pf_glort_get(i), pdev->addr.bus,
+ pdev->addr.devid,
+ pdev->addr.function,
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+ }
+
+ FM10K_SW_INFO("--- DPDK PORT ---\n");
+ for (i = 0; i < FM10K_SW_LOGICAL_PORTS_MAX; i++) {
+ if (sw->dpdk_cfg->ports[i].type == FM10K_CONFIG_DPDK_NULL)
+ break;
+ map = &sw->dpdk_cfg->dpdk_port_map[i];
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ break;
+ if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d\n",
+ i, map->map_no[0]);
+ } else {
+ FM10K_SW_INFO(" DPDK PORT%d : Map to PF%d&PF%d Glort %#x\n",
+ i,
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+ }
+
+ FM10K_SW_INFO("--- EXT PORT ---\n");
+ for (i = 0; i < sw->dpdk_cfg->ext_port_num; i++) {
+ map = &sw->dpdk_cfg->ext_port_map[i];
+ FM10K_SW_INFO(" EXT PORT%d : Logical %d Glort %#x", i,
+ fm10k_epl_port_map[i].logical_port,
+ fm10k_switch_epl_glort_get(i));
+
+ if (map->type == FM10K_CONFIG_PORT_MAP_NULL)
+ FM10K_SW_INFO("\n");
+ else if (map->type == FM10K_CONFIG_PORT_MAP_PF)
+ FM10K_SW_INFO(" Map to PF%d\n", map->map_no[0]);
+ else
+ FM10K_SW_INFO(" Map to PF%d&PF%d Glort %#x\n",
+ map->map_no[0], map->map_no[1],
+ fm10k_switch_pfs_glort_get
+ (map->map_no[0], map->map_no[1]));
+ }
+}
+
+
+int
+fm10k_switch_dpdk_port_start(struct fm10k_hw *hw, void *rte_dev,
+ uint8_t is_pf, bool master, eth_fm10k_dev_init_half_func *func)
+{
+ int ret;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (fm10k_switch_dpdk_port_reg(sw, hw, rte_dev, is_pf, master) != 0) {
+ FM10K_SW_ERR("Register ports failed!!!");
+ return -1;
+ }
+
+ /*
+ * After all pfs are started
+ * start switch here
+ */
+ if (fm10k_sw.dpdk_cfg->pf_max != 0 &&
+ fm10k_sw.dpdk_cfg->pf_bind == fm10k_sw.dpdk_cfg->pf_num) {
+ if (fm10k_switch_dpdk_cfg_check(&fm10k_sw) != 0)
+ return -1;
+
+ ret = fm10k_switch_start(&fm10k_sw,
+ fm10k_sw.dpdk_cfg->master_hw, func);
+ fm10k_switch_dpdk_cfg_describe(&fm10k_sw);
+ return ret;
+ }
+ return 0;
+}
+
+void fm10k_switch_dpdk_port_stop(struct fm10k_hw *hw)
+{
+ if (hw)
+ return;
+}
+
+
+/*
+ * for multi-host, only dpdk port 0 and 1 are real rx/tx packets
+ * so we need init/setup/start dpdk port queue for them.
+ * for dpdk port 2/3, we need init/setup except start.
+ * we map the pf queue to 0/1 dpdk queue first, then map
+ * the other pf queue to 2/3 dpdk queue.
+ */
+int
+fm10k_switch_dpdk_hw_queue_map(struct fm10k_hw *hw,
+ uint16_t queue, uint16_t max_queue,
+ struct fm10k_hw **map_hw, uint16_t *map_queue)
+{
+ int idx, pf_no;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ FM10K_SW_ERR("Not support yet!!!");
+ return -1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ }
+
+ if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[0];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = queue / 2;
+ return 1;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_PFSS) {
+ idx = queue % 2;
+ pf_no = sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].map_no[idx];
+ *map_hw = sw->dpdk_cfg->pf_hw[pf_no];
+ *map_queue = (max_queue + 1) / 2 + queue / 2;
+ return 0;
+ } else if (sw->dpdk_cfg->dpdk_port_map[dpdk_port_no].type ==
+ FM10K_CONFIG_PORT_MAP_NULL) {
+ FM10K_SW_ERR("Unmapped dpdk port %d!!!", dpdk_port_no);
+ return -1;
+ }
+ FM10K_SW_ERR("Unknown mapped type!!!");
+ return -1;
+}
+
+int
+fm10k_switch_dpdk_mapped_hw_get(struct fm10k_hw *hw, struct fm10k_hw *hw_list[])
+{
+ int pf_no, pf_no_ext;
+ int dpdk_port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_dpdk_port *port;
+ struct fm10k_cfg_port_pf_map *map;
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (dpdk_port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ port = &sw->dpdk_cfg->ports[dpdk_port_no];
+ map = &sw->dpdk_cfg->dpdk_port_map[dpdk_port_no];
+
+ if (port->type == FM10K_CONFIG_DPDK_VF) {
+ hw_list[0] = hw;
+ hw_list[1] = NULL;
+ return 1;
+ }
+
+ if (port->type != FM10K_CONFIG_DPDK_PF) {
+ FM10K_SW_ERR("Can not be here!!!");
+ return -1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PF) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ hw_list[1] = NULL;
+ return 1;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 2;
+ } else if (map->type == FM10K_CONFIG_PORT_MAP_PFSS) {
+ pf_no = map->map_no[0];
+ hw_list[0] = sw->dpdk_cfg->pf_hw[pf_no];
+ pf_no_ext = map->map_no[1];
+ hw_list[1] = sw->dpdk_cfg->pf_hw[pf_no_ext];
+ return 0;
+ }
+
+ hw_list[0] = NULL;
+ hw_list[1] = NULL;
+ return 0;
+}
+
+void
+fm10k_switch_dpdk_tx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].tx_queue_num = num;
+}
+
+void
+fm10k_switch_dpdk_rx_queue_num_set(struct fm10k_hw *hw, uint8_t num)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return;
+ }
+
+ sw->dpdk_cfg->ports[port_no].rx_queue_num = num;
+}
+
+int
+fm10k_switch_dpdk_pf_no_get(struct fm10k_hw *hw)
+{
+ int port_no = fm10k_switch_dpdk_port_no_get(hw);
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ if (port_no < 0) {
+ FM10K_SW_ERR("Can not find the dpdk port!!!");
+ return -1;
+ }
+
+ return sw->dpdk_cfg->ports[port_no].pf_no;
+}
+
+
+void
+fm10k_switch_flowset_switchto(const char *name)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_ffu_flowset_switch(sw, name);
+}
+
+void
+fm10k_switch_show_port(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_epl_port_print(sw);
+ fm10k_stats_dpdk_port_print(sw);
+}
+
+void
+fm10k_switch_show_ffu(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_ffu_count_print(sw);
+}
+
+void
+fm10k_switch_show_bank(void)
+{
+ struct fm10k_switch *sw = fm10k_switch_get();
+
+ fm10k_stats_port_bank_print(sw);
+}
diff --git a/drivers/net/fm10k/switch/fm10k_switch.h b/drivers/net/fm10k/switch/fm10k_switch.h
index b4b363c..a2aa0d0 100644
--- a/drivers/net/fm10k/switch/fm10k_switch.h
+++ b/drivers/net/fm10k/switch/fm10k_switch.h
@@ -34,6 +34,7 @@
#define FM10K_SW_PEPS_MAX 9
#define FM10K_SW_PEPS_SUPPORTED 4
#define FM10K_SW_PEP_PORTS_MAX 4
+#define FM10K_SW_PEP_MAP_NUM_MAX 2
/*
* GLORT
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
` (3 preceding siblings ...)
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
@ 2020-03-20 6:58 ` Xiaojun Liu
2020-04-02 9:12 ` Wang, Xiao W
4 siblings, 1 reply; 50+ messages in thread
From: Xiaojun Liu @ 2020-03-20 6:58 UTC (permalink / raw)
To: xiao.w.wang, qi.z.zhang, ngai-mint.kwan, jacob.e.keller; +Cc: dev, Xiaojun Liu
Split dev init to 2 parts.
First only register the port in switch
management; second init hook will be
called after all the pf are registered
and switch initialization. It will finish
dev init. Also add switch interrupt support.
Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
to support mirror operation. Add fm10k_dev_filter_ctrl
to support flow operation.
Add dpdk port and pf mapping, so
the dpdk port can map to a specific pf
and 1 dpdk port can map to 2 pf to get
total 100G throughput. Update fm10k.rst.
Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
---
doc/guides/nics/fm10k.rst | 5 +
drivers/net/fm10k/fm10k_ethdev.c | 559 ++++++++++++++++++++++++++++++++++++---
2 files changed, 525 insertions(+), 39 deletions(-)
diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
index 4e178c2..75f9929 100644
--- a/doc/guides/nics/fm10k.rst
+++ b/doc/guides/nics/fm10k.rst
@@ -124,6 +124,11 @@ FM10000 PMD driver. The switch driver can be acquired from Intel support.
Only Testpoint is validated with DPDK, the latest version that has been
validated with DPDK is 4.1.6.
+To avoid loading other switch driver from outside of DPDK, there is another
+way to load switch driver in DPDK. If you want to use the inline switch driver,
+you need to set ''CONFIG_RTE_FM10K_SWITCH_MANAGEMENT=y'' in the configure file.
+The inline switch driver supports mirror and some flow forward rules.
+
Support for Switch Restart
~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 581c690..77dc5b0 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -13,6 +13,10 @@
#include "fm10k.h"
#include "base/fm10k_api.h"
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+#include "switch/fm10k_regs.h"
+#include "switch/fm10k_switch.h"
+#endif
/* Default delay to acquire mailbox lock */
#define FM10K_MBXLOCK_DELAY_US 20
@@ -512,6 +516,15 @@ struct fm10k_xstats_name_off {
struct rte_eth_conf *dev_conf = &dev->data->dev_conf;
uint32_t mrqc, *key, i, reta, j;
uint64_t hf;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ uint16_t nb_rx_queues = dev->data->nb_rx_queues;
+ int mapped_num;
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num == 2)
+ nb_rx_queues /= 2;
+#endif
#define RSS_KEY_SIZE 40
static uint8_t rss_intel_key[RSS_KEY_SIZE] = {
@@ -642,26 +655,44 @@ struct fm10k_xstats_name_off {
fm10k_dev_tx_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ uint32_t data;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_tx_queue *txq;
uint64_t base_addr;
uint32_t size;
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
/* Disable TXINT to avoid possible interrupt */
for (i = 0; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_TXINT(i),
3 << FM10K_TXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_tx_queue_num_set(unmap_hw,
+ dev->data->nb_tx_queues);
+#endif
/* Setup TX queue */
for (i = 0; i < dev->data->nb_tx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id);
+#endif
txq = dev->data->tx_queues[i];
base_addr = txq->hw_ring_phys_addr;
size = txq->nb_desc * sizeof(struct fm10k_tx_desc);
/* disable queue to avoid issues while updating state */
- ret = tx_queue_disable(hw, i);
+ ret = tx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Enable use of FTAG bit in TX descriptor, PFVTCTL
@@ -669,7 +700,7 @@ struct fm10k_xstats_name_off {
*/
if (fm10k_check_ftag(dev->device->devargs)) {
if (hw->mac.type == fm10k_mac_pf) {
- FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_PFVTCTL(hw_queue_id),
FM10K_PFVTCTL_FTAG_DESC_ENABLE);
PMD_INIT_LOG(DEBUG, "FTAG mode is enabled");
} else {
@@ -679,15 +710,25 @@ struct fm10k_xstats_name_off {
}
/* set location and size for descriptor ring */
- FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_TDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_TDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_TDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_TDLEN(hw_queue_id), size);
/* assign default SGLORT for each TX queue by PF */
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf)
- FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(i), hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw,
+ FM10K_TX_SGLORT(hw_queue_id),
+ hw->mac.dglort_map);
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ data = FM10K_SW_MAKE_REG_FIELD
+ (TX_SGLORT_SGLORT, hw->mac.dglort_map);
+ FM10K_WRITE_REG(hw, FM10K_TX_SGLORT(hw_queue_id), data);
+ }
+#endif
}
/* set up vector or scalar TX function as appropriate */
@@ -700,18 +741,24 @@ struct fm10k_xstats_name_off {
fm10k_dev_rx_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
struct fm10k_macvlan_filter_info *macvlan;
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+ uint32_t logic_port = hw->mac.dglort_map;
+ uint16_t queue_stride = 0;
+#else
+ struct fm10k_hw *unmap_hw = hw;
+#endif
int i, ret;
+ uint16_t hw_queue_id;
struct fm10k_rx_queue *rxq;
uint64_t base_addr;
uint32_t size;
uint32_t rxdctl = FM10K_RXDCTL_WRITE_BACK_MIN_DELAY;
- uint32_t logic_port = hw->mac.dglort_map;
uint16_t buf_size;
- uint16_t queue_stride = 0;
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
/* enable RXINT for interrupt mode */
i = 0;
if (rte_intr_dp_is_en(intr_handle)) {
@@ -731,26 +778,36 @@ struct fm10k_xstats_name_off {
for (; i < hw->mac.max_queues; i++)
FM10K_WRITE_REG(hw, FM10K_RXINT(i),
3 << FM10K_RXINT_TIMER_SHIFT);
+#else
+ fm10k_switch_dpdk_rx_queue_num_set(unmap_hw, dev->data->nb_rx_queues);
+#endif
/* Setup RX queues */
for (i = 0; i < dev->data->nb_rx_queues; ++i) {
+ hw_queue_id = i;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, dev->data->nb_rx_queues, &hw, &hw_queue_id);
+#endif
rxq = dev->data->rx_queues[i];
base_addr = rxq->hw_ring_phys_addr;
size = rxq->nb_desc * sizeof(union fm10k_rx_desc);
/* disable queue to avoid issues while updating state */
- ret = rx_queue_disable(hw, i);
+ ret = rx_queue_disable(hw, hw_queue_id);
if (ret) {
- PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
+ PMD_INIT_LOG(ERR,
+ "failed to disable queue %d",
+ hw_queue_id);
return -1;
}
/* Setup the Base and Length of the Rx Descriptor Ring */
- FM10K_WRITE_REG(hw, FM10K_RDBAL(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAL(hw_queue_id),
base_addr & UINT64_LOWER_32BITS_MASK);
- FM10K_WRITE_REG(hw, FM10K_RDBAH(i),
+ FM10K_WRITE_REG(hw, FM10K_RDBAH(hw_queue_id),
base_addr >> (CHAR_BIT * sizeof(uint32_t)));
- FM10K_WRITE_REG(hw, FM10K_RDLEN(i), size);
+ FM10K_WRITE_REG(hw, FM10K_RDLEN(hw_queue_id), size);
/* Configure the Rx buffer size for one buff without split */
buf_size = (uint16_t)(rte_pktmbuf_data_room_size(rxq->mp) -
@@ -764,7 +821,7 @@ struct fm10k_xstats_name_off {
*/
buf_size -= FM10K_RX_DATABUF_ALIGN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i),
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id),
(buf_size >> FM10K_SRRCTL_BSIZEPKT_SHIFT) |
FM10K_SRRCTL_LOOPBACK_SUPPRESS);
@@ -774,9 +831,9 @@ struct fm10k_xstats_name_off {
rxq->offloads & DEV_RX_OFFLOAD_SCATTER) {
uint32_t reg;
dev->data->scattered_rx = 1;
- reg = FM10K_READ_REG(hw, FM10K_SRRCTL(i));
+ reg = FM10K_READ_REG(hw, FM10K_SRRCTL(hw_queue_id));
reg |= FM10K_SRRCTL_BUFFER_CHAINING_EN;
- FM10K_WRITE_REG(hw, FM10K_SRRCTL(i), reg);
+ FM10K_WRITE_REG(hw, FM10K_SRRCTL(hw_queue_id), reg);
}
/* Enable drop on empty, it's RO for VF */
@@ -796,6 +853,7 @@ struct fm10k_xstats_name_off {
/* update RX_SGLORT for loopback suppress*/
if (hw->mac.type != fm10k_mac_pf)
return 0;
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
macvlan = FM10K_DEV_PRIVATE_TO_MACVLAN(dev->data->dev_private);
if (macvlan->nb_queue_pools)
queue_stride = dev->data->nb_rx_queues / macvlan->nb_queue_pools;
@@ -804,6 +862,7 @@ struct fm10k_xstats_name_off {
logic_port++;
FM10K_WRITE_REG(hw, FM10K_RX_SGLORT(i), logic_port);
}
+#endif
return 0;
}
@@ -812,12 +871,29 @@ struct fm10k_xstats_name_off {
fm10k_dev_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ int ret;
+#endif
int err;
uint32_t reg;
struct fm10k_rx_queue *rxq;
+ uint16_t hw_queue_id = rx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ rx_queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and rx queue for:"
+ " %s %d", dev->device->name, rx_queue_id);
+ return -EIO;
+ } else if (ret != 1) /* reference port's queue don't need start */
+ return 0;
+#endif
+
rxq = dev->data->rx_queues[rx_queue_id];
err = rx_queue_reset(rxq);
if (err == -ENOMEM) {
@@ -836,23 +912,23 @@ struct fm10k_xstats_name_off {
* this comment and the following two register writes when the
* emulation platform is no longer being used.
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
/* Set PF ownership flag for PF devices */
- reg = FM10K_READ_REG(hw, FM10K_RXQCTL(rx_queue_id));
+ reg = FM10K_READ_REG(hw, FM10K_RXQCTL(hw_queue_id));
if (hw->mac.type == fm10k_mac_pf)
reg |= FM10K_RXQCTL_PF;
reg |= FM10K_RXQCTL_ENABLE;
/* enable RX queue */
- FM10K_WRITE_REG(hw, FM10K_RXQCTL(rx_queue_id), reg);
+ FM10K_WRITE_REG(hw, FM10K_RXQCTL(hw_queue_id), reg);
FM10K_WRITE_FLUSH(hw);
/* Setup the HW Rx Head and Tail Descriptor Pointers
* Note: this must be done AFTER the queue is enabled
*/
- FM10K_WRITE_REG(hw, FM10K_RDH(rx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_RDT(rx_queue_id), rxq->nb_desc - 1);
+ FM10K_WRITE_REG(hw, FM10K_RDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_RDT(hw_queue_id), rxq->nb_desc - 1);
dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
return 0;
@@ -879,21 +955,38 @@ struct fm10k_xstats_name_off {
fm10k_dev_tx_queue_start(struct rte_eth_dev *dev, uint16_t tx_queue_id)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ int ret;
+#endif
/** @todo - this should be defined in the shared code */
#define FM10K_TXDCTL_WRITE_BACK_MIN_DELAY 0x00010000
uint32_t txdctl = FM10K_TXDCTL_WRITE_BACK_MIN_DELAY;
struct fm10k_tx_queue *q = dev->data->tx_queues[tx_queue_id];
+ uint16_t hw_queue_id = tx_queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ ret = fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ tx_queue_id, dev->data->nb_tx_queues, &hw, &hw_queue_id);
+ if (ret < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and tx queue for:"
+ " %s %d", dev->device->name, tx_queue_id);
+ return -EIO;
+ } else if (ret != 1) {
+ return 0;
+ }
+#endif
+
q->ops->reset(q);
/* reset head and tail pointers */
- FM10K_WRITE_REG(hw, FM10K_TDH(tx_queue_id), 0);
- FM10K_WRITE_REG(hw, FM10K_TDT(tx_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDH(hw_queue_id), 0);
+ FM10K_WRITE_REG(hw, FM10K_TDT(hw_queue_id), 0);
/* enable TX queue */
- FM10K_WRITE_REG(hw, FM10K_TXDCTL(tx_queue_id),
+ FM10K_WRITE_REG(hw, FM10K_TXDCTL(hw_queue_id),
FM10K_TXDCTL_ENABLE | txdctl);
FM10K_WRITE_FLUSH(hw);
dev->data->tx_queue_state[tx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -1084,9 +1177,25 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
int i, diag;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+ int j, mapped_num;
+ uint32_t data;
+#endif
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > FM10K_SW_PEP_MAP_NUM_MAX) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW for %s",
+ dev->device->name);
+ return -EIO;
+ }
+#endif
+
+
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
/* stop, init, then start the hw */
diag = fm10k_stop_hw(hw);
if (diag != FM10K_SUCCESS) {
@@ -1105,6 +1214,62 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
return -EIO;
}
+#else
+ for (j = 0; j < mapped_num; j++) {
+ struct rte_pci_device *pdev =
+ RTE_ETH_DEV_TO_PCI((struct rte_eth_dev *)
+ (fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[j])));
+ struct rte_intr_handle *intr_handle = &pdev->intr_handle;
+
+ /* stop, init, then start the hw */
+ diag = fm10k_stop_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware stop failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_init_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware init failed: %d", diag);
+ return -EIO;
+ }
+
+ diag = fm10k_start_hw(mapped_hws[j]);
+ if (diag != FM10K_SUCCESS) {
+ PMD_INIT_LOG(ERR, "Hardware start failed: %d", diag);
+ return -EIO;
+ }
+
+ /* Disable TXINT to avoid possible interrupt */
+ for (i = 0; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_TXINT(i),
+ 3 << FM10K_TXINT_TIMER_SHIFT);
+
+ /* enable RXINT for interrupt mode */
+ i = 0;
+ if (rte_intr_dp_is_en(intr_handle)) {
+ for (; i < dev->data->nb_rx_queues; i++) {
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_RXINT(i), Q2V(pdev, i));
+ if (mapped_hws[j]->mac.type == fm10k_mac_pf)
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_ITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ else
+ FM10K_WRITE_REG(mapped_hws[j],
+ FM10K_VFITR(Q2V(pdev, i)),
+ FM10K_ITR_AUTOMASK |
+ FM10K_ITR_MASK_CLEAR);
+ }
+ }
+
+ /* Disable other RXINT to avoid possible interrupt */
+ for (; i < hw->mac.max_queues; i++)
+ FM10K_WRITE_REG(mapped_hws[j], FM10K_RXINT(i),
+ 3 << FM10K_RXINT_TIMER_SHIFT);
+ }
+#endif
diag = fm10k_dev_tx_init(dev);
if (diag) {
@@ -1156,10 +1321,26 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
}
}
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
/* Update default vlan when not in VMDQ mode */
if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+#endif
+
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ /* Admit all VLANs */
+ for (j = 0; j <= 64; j++) {
+ for (i = 0; i < FM10K_SW_VLAN_TABLE_ENTRIES; i++)
+ FM10K_WRITE_REG(hw,
+ FM10K_SW_VLAN_TABLE_ENTRY(j, i),
+ 0xffffffff);
+ }
+ /* Disable PEP loopback */
+ data = FM10K_READ_REG(hw, FM10K_CTRL_EXT);
+ data &= ~FM10K_SW_CTRL_EXT_SWITCH_LOOPBACK;
+ FM10K_WRITE_REG(hw, FM10K_CTRL_EXT, data);
+#endif
fm10k_link_update(dev, 0);
return 0;
@@ -1327,12 +1508,24 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
struct fm10k_hw_stats *hw_stats =
FM10K_DEV_PRIVATE_TO_STATS(dev->data->dev_private);
int i;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+ struct fm10k_hw *mapped_hws[FM10K_SW_PEP_MAP_NUM_MAX];
+ int mapped_num;
+ uint16_t hw_queue_id;
+#endif
PMD_INIT_FUNC_TRACE();
+ ipackets = 0;
+ opackets = 0;
+ ibytes = 0;
+ obytes = 0;
+ imissed = 0;
+
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
fm10k_update_hw_stats(hw, hw_stats);
- ipackets = opackets = ibytes = obytes = imissed = 0;
for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
(i < hw->mac.max_queues); ++i) {
stats->q_ipackets[i] = hw_stats->q[i].rx_packets.count;
@@ -1346,6 +1539,49 @@ static int fm10k_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
obytes += stats->q_obytes[i];
imissed += stats->q_errors[i];
}
+#else
+ /* Get statistics from mapped device's queue */
+ mapped_num = fm10k_switch_dpdk_mapped_hw_get(unmap_hw, mapped_hws);
+ if (mapped_num < 0 || mapped_num > FM10K_SW_PEP_MAP_NUM_MAX) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW for %s",
+ dev->device->name);
+ return -EIO;
+ }
+
+ for (i = 0; i < mapped_num; i++) {
+ struct rte_eth_dev *mydev =
+ fm10k_switch_dpdk_port_rte_dev_get(mapped_hws[i]);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS(mydev->data->dev_private);
+ fm10k_update_hw_stats(mapped_hws[i], hw_stats);
+ }
+
+ if (mapped_num) {
+ for (i = 0; (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) &&
+ (i < unmap_hw->mac.max_queues); ++i) {
+ fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ i, unmap_hw->mac.max_queues,
+ &hw, &hw_queue_id);
+ if (mapped_hws[1]) {
+ struct rte_eth_dev *mydev;
+ mydev = fm10k_switch_dpdk_port_rte_dev_get(hw);
+ hw_stats = FM10K_DEV_PRIVATE_TO_STATS
+ (mydev->data->dev_private);
+ }
+ stats->q_ipackets[i] =
+ hw_stats->q[hw_queue_id].rx_packets.count;
+ stats->q_opackets[i] =
+ hw_stats->q[hw_queue_id].tx_packets.count;
+ stats->q_ibytes[i] =
+ hw_stats->q[hw_queue_id].rx_bytes.count;
+ stats->q_obytes[i] =
+ hw_stats->q[hw_queue_id].tx_bytes.count;
+ ipackets += stats->q_ipackets[i];
+ opackets += stats->q_opackets[i];
+ ibytes += stats->q_ibytes[i];
+ obytes += stats->q_obytes[i];
+ }
+ }
+#endif
stats->ipackets = ipackets;
stats->opackets = opackets;
stats->ibytes = ibytes;
@@ -1808,8 +2044,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_JUMBO_FRAME |
- DEV_RX_OFFLOAD_HEADER_SPLIT |
- DEV_RX_OFFLOAD_RSS_HASH);
+ DEV_RX_OFFLOAD_HEADER_SPLIT);
}
static int
@@ -1818,14 +2053,28 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
const struct rte_eth_rxconf *conf, struct rte_mempool *mp)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+#endif
struct fm10k_dev_info *dev_info =
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
struct fm10k_rx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_rx_queues,
+ &hw, &hw_queue_id) < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and queue for:"
+ " %s %d", dev->device->name, queue_id);
+ return -EIO;
+ }
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.rxmode.offloads;
/* make sure the mempool element size can account for alignment. */
@@ -1871,7 +2120,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
q->port_id = dev->data->port_id;
q->queue_id = queue_id;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_RDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_RDT(hw_queue_id)];
q->offloads = offloads;
if (handle_rxconf(q, conf))
return -EINVAL;
@@ -2007,12 +2256,26 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
const struct rte_eth_txconf *conf)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *unmap_hw = hw;
+#endif
struct fm10k_tx_queue *q;
const struct rte_memzone *mz;
uint64_t offloads;
+ uint16_t hw_queue_id = queue_id;
PMD_INIT_FUNC_TRACE();
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ if (fm10k_switch_dpdk_hw_queue_map(unmap_hw,
+ queue_id, dev->data->nb_tx_queues,
+ &hw, &hw_queue_id) < 0) {
+ PMD_INIT_LOG(ERR, "Can not get the mapped HW and queue for:"
+ " %s %d", dev->device->name, queue_id);
+ return -EIO;
+ }
+#endif
+
offloads = conf->offloads | dev->data->dev_conf.txmode.offloads;
/* make sure a valid number of descriptors have been requested */
@@ -2054,7 +2317,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
q->offloads = offloads;
q->ops = &def_txq_ops;
q->tail_ptr = (volatile uint32_t *)
- &((uint32_t *)hw->hw_addr)[FM10K_TDT(queue_id)];
+ &((uint32_t *)hw->hw_addr)[FM10K_TDT(hw_queue_id)];
if (handle_txconf(q, conf))
return -EINVAL;
@@ -2284,6 +2547,77 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
return 0;
}
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+static int
+fm10k_mirror_rule_set(struct rte_eth_dev *dev,
+ struct rte_eth_mirror_conf *mirror_conf,
+ uint8_t sw_id, uint8_t on)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG,
+ "Mirror set, switch %d to port %d attach vlan %d on %d",
+ sw_id, mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0], on);
+
+ if (on) {
+ if (fm10k_switch_mirror_set(hw,
+ mirror_conf->dst_pool,
+ mirror_conf->vlan.vlan_id[0]) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ } else {
+ if (fm10k_switch_mirror_reset(hw) < 0) {
+ PMD_INIT_LOG(ERR, "Input wrong port!!!");
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+
+static int
+fm10k_mirror_rule_reset(struct rte_eth_dev *dev, uint8_t sw_id)
+{
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
+ PMD_INIT_LOG(DEBUG, "Mirror reset, switch %d", sw_id);
+
+ fm10k_switch_mirror_reset(hw);
+
+ return 0;
+}
+
+static int
+fm10k_dev_filter_ctrl(struct rte_eth_dev *dev,
+ enum rte_filter_type filter_type,
+ enum rte_filter_op filter_op,
+ void *arg)
+{
+ int ret = 0;
+
+ if (dev == NULL)
+ return -EINVAL;
+
+ switch (filter_type) {
+ case RTE_ETH_FILTER_GENERIC:
+ if (filter_op != RTE_ETH_FILTER_GET)
+ return -EINVAL;
+ *(const void **)arg = fm10k_flow_ops_get();
+ break;
+ default:
+ PMD_DRV_LOG(WARNING, "Filter type (%d) not supported",
+ filter_type);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+#endif
+
static void
fm10k_dev_enable_intr_pf(struct rte_eth_dev *dev)
{
@@ -2592,6 +2926,9 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
int status_mbx;
s32 err;
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ uint32_t writeback = 0;
+#endif
if (hw->mac.type != fm10k_mac_pf)
return;
@@ -2605,11 +2942,20 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle switch up/down */
- if (cause & FM10K_EICR_SWITCHNOTREADY)
- PMD_INIT_LOG(ERR, "INT: Switch is not ready");
+ if (cause & FM10K_EICR_SWITCHNOTREADY) {
+ PMD_INIT_LOG(INFO, "INT: Switch is not ready");
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ hw->switch_ready = false;
+ writeback |= FM10K_EICR_SWITCHNOTREADY;
+#endif
+ }
if (cause & FM10K_EICR_SWITCHREADY) {
PMD_INIT_LOG(INFO, "INT: Switch is ready");
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ hw->switch_ready = true;
+ writeback |= FM10K_EICR_SWITCHREADY;
+#endif
if (dev_info->sm_down == 1) {
fm10k_mbx_lock(hw);
@@ -2660,6 +3006,7 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
}
/* Handle mailbox message */
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
fm10k_mbx_lock(hw);
err = hw->mbx.ops.process(hw, &hw->mbx);
fm10k_mbx_unlock(hw);
@@ -2667,10 +3014,33 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
if (err == FM10K_ERR_RESET_REQUESTED) {
PMD_INIT_LOG(INFO, "INT: Switch is down");
dev_info->sm_down = 1;
- _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
NULL);
}
+#else
+ if (cause & FM10K_EICR_MAILBOX) {
+ fm10k_mbx_lock(hw);
+ err = hw->mbx.ops.process(hw, &hw->mbx);
+ fm10k_mbx_unlock(hw);
+ writeback |= FM10K_EICR_MAILBOX;
+ if (err == FM10K_ERR_RESET_REQUESTED) {
+ PMD_INIT_LOG(INFO, "INT: Switch is down");
+ dev_info->sm_down = 1;
+ _rte_eth_dev_callback_process
+ (dev,
+ RTE_ETH_EVENT_INTR_LSC,
+ NULL);
+ }
+ }
+
+ /* Handle switch interrupt */
+ if (cause & FM10K_SW_EICR_SWITCH_INT)
+ fm10k_switch_intr(hw);
+#endif
+
/* Handle SRAM error */
if (cause & FM10K_EICR_SRAMERROR) {
PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
@@ -2682,15 +3052,27 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
/* Todo: print out error message after shared code updates */
}
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
/* Clear these 3 events if having any */
cause &= FM10K_EICR_SWITCHNOTREADY | FM10K_EICR_MAILBOX |
FM10K_EICR_SWITCHREADY;
if (cause)
FM10K_WRITE_REG(hw, FM10K_EICR, cause);
+#else
+ if (writeback)
+ FM10K_WRITE_REG(hw, FM10K_EICR, writeback);
+#endif
/* Re-enable interrupt from device side */
- FM10K_WRITE_REG(hw, FM10K_ITR(0), FM10K_ITR_AUTOMASK |
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_ITR_AUTOMASK |
FM10K_ITR_MASK_CLEAR);
+#else
+ FM10K_WRITE_REG(hw, FM10K_ITR(0),
+ FM10K_SW_ITR_AUTO_MASK |
+ FM10K_SW_MAKE_REG_FIELD(ITR_MASK, FM10K_SW_ITR_MASK_W_ENABLE));
+#endif
/* Re-enable interrupt from host side */
rte_intr_ack(dev->intr_handle);
}
@@ -2897,6 +3279,11 @@ static uint64_t fm10k_get_tx_port_offloads_capa(struct rte_eth_dev *dev)
.reta_query = fm10k_reta_query,
.rss_hash_update = fm10k_rss_hash_update,
.rss_hash_conf_get = fm10k_rss_hash_conf_get,
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ .mirror_rule_set = fm10k_mirror_rule_set,
+ .mirror_rule_reset = fm10k_mirror_rule_reset,
+ .filter_ctrl = fm10k_dev_filter_ctrl,
+#endif
};
static int ftag_check_handler(__rte_unused const char *key,
@@ -3075,13 +3462,67 @@ static void __attribute__((cold))
info->sm_down = false;
}
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+static int eth_fm10k_dev_init_hook(struct fm10k_hw *hw)
+{
+ int i;
+ struct rte_eth_dev *dev =
+ (struct rte_eth_dev *)fm10k_switch_dpdk_port_rte_dev_get(hw);
+
+ /*
+ * Below function will trigger operations on mailbox, acquire lock to
+ * avoid race condition from interrupt handler. Operations on mailbox
+ * FIFO will trigger interrupt to PF/SM, in which interrupt handler
+ * will handle and generate an interrupt to our side. Then, FIFO in
+ * mailbox will be touched.
+ */
+ if (hw->mac.dglort_map == FM10K_DGLORTMAP_NONE) {
+ PMD_INIT_LOG(ERR, "dglort_map is not ready");
+ return -1;
+ }
+
+ fm10k_mbx_lock(hw);
+ /* Enable port first */
+ hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map,
+ MAX_LPORT_NUM, 1);
+ /* Set unicast mode by default. App can change to other mode in other
+ * API func.
+ */
+ hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+ FM10K_XCAST_MODE_NONE);
+ fm10k_mbx_unlock(hw);
+
+ /* Make sure default VID is ready before going forward. */
+ if (hw->mac.type == fm10k_mac_pf) {
+ for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
+ if (hw->mac.default_vid)
+ break;
+ /* Delay some time to acquire async port VLAN info. */
+ rte_delay_us(WAIT_SWITCH_MSG_US);
+ }
+
+ if (hw->mac.default_vid == 0)
+ hw->mac.default_vid = 1;
+ }
+
+ /* Add default mac address */
+ fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+ MAIN_VSI_POOL_NUMBER);
+
+ return 0;
+}
+#endif
+
static int
eth_fm10k_dev_init(struct rte_eth_dev *dev)
{
struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct rte_pci_device *pdev = RTE_ETH_DEV_TO_PCI(dev);
struct rte_intr_handle *intr_handle = &pdev->intr_handle;
- int diag, i;
+ int diag;
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
+ int i;
+#endif
struct fm10k_macvlan_filter_info *macvlan;
PMD_INIT_FUNC_TRACE();
@@ -3118,7 +3559,9 @@ static void __attribute__((cold))
" Try to blacklist unused devices.");
return -EIO;
}
-
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ hw->sw_addr = (void *)pdev->mem_resource[4].addr;
+#endif
/* Store fm10k_adapter pointer */
hw->back = dev->data->dev_private;
@@ -3129,6 +3572,25 @@ static void __attribute__((cold))
return -EIO;
}
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ if (hw->mac.type == fm10k_mac_pf) {
+ if (hw->hw_addr == NULL || hw->sw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ } else {
+ if (hw->hw_addr == NULL) {
+ PMD_INIT_LOG(ERR, "Bad mem resource."
+ " Try to blacklist unused devices.");
+ return -EIO;
+ }
+ }
+
+ /* Store fm10k_adapter pointer */
+ hw->back = dev->data->dev_private;
+#endif
+
/* Initialize parameters */
fm10k_params_init(dev);
@@ -3209,6 +3671,7 @@ static void __attribute__((cold))
hw->mac.ops.update_int_moderator(hw);
/* Make sure Switch Manager is ready before going forward. */
+#ifndef RTE_FM10K_SWITCH_MANAGEMENT
if (hw->mac.type == fm10k_mac_pf) {
bool switch_ready = false;
@@ -3216,13 +3679,13 @@ static void __attribute__((cold))
fm10k_mbx_lock(hw);
hw->mac.ops.get_host_state(hw, &switch_ready);
fm10k_mbx_unlock(hw);
- if (switch_ready == true)
+ if (switch_ready)
break;
/* Delay some time to acquire async LPORT_MAP info. */
rte_delay_us(WAIT_SWITCH_MSG_US);
}
- if (switch_ready == false) {
+ if (switch_ready == 0) {
PMD_INIT_LOG(ERR, "switch is not ready");
return -1;
}
@@ -3268,11 +3731,25 @@ static void __attribute__((cold))
MAIN_VSI_POOL_NUMBER);
return 0;
+#else
+ if (hw->mac.type == fm10k_mac_pf) {
+ bool master = FM10K_READ_REG(hw,
+ FM10K_CTRL) & FM10K_CTRL_BAR4_ALLOWED;
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 1, master, eth_fm10k_dev_init_hook);
+ } else {
+ return fm10k_switch_dpdk_port_start(hw,
+ dev, 0, false, eth_fm10k_dev_init_hook);
+ }
+#endif
}
static int
eth_fm10k_dev_uninit(struct rte_eth_dev *dev)
{
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+#endif
PMD_INIT_FUNC_TRACE();
/* only uninitialize in the primary process */
@@ -3282,6 +3759,10 @@ static void __attribute__((cold))
/* safe to close dev here */
fm10k_dev_close(dev);
+#ifdef RTE_FM10K_SWITCH_MANAGEMENT
+ fm10k_switch_dpdk_port_stop(hw);
+#endif
+
return 0;
}
--
1.8.3.1
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for switch management
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for " Xiaojun Liu
@ 2020-04-02 8:41 ` Wang, Xiao W
2020-04-08 3:25 ` Xiaojun Liu
0 siblings, 1 reply; 50+ messages in thread
From: Wang, Xiao W @ 2020-04-02 8:41 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiaojun,
Just comments on the alignment issue, things like below looks not nice. If you are using vim, I recommend " set tabstop=8 " setting.
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
Check more at http://patches.dpdk.org/patch/66973/. Please take care for next version.
I'm wondering if we could have a tool for automatically check this kind of alignment issue.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 1/5] net/fm10k: add basic functions for switch management
>
> Add I2C to control the inside LED and PHY.
> All the operations of I2C are using fm10k I2C register.
> Add SBUS to communicate with spico(micro code in serdes)
> by using fm10k SBUS register. This is like I2C operations.
> Add registers defination, which include all the registers
> will be used in the driver. Add switch management log API.
> Add switch management structures. Modify Makefile to add
> new files building. Add CONFIG_RTE_FM10K_MANAGEMENT=n
> in config/common_linux.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> config/common_linux | 5 +
> drivers/net/fm10k/Makefile | 11 +
> drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> drivers/net/fm10k/switch/fm10k_i2c.c | 309 +++++
> drivers/net/fm10k/switch/fm10k_i2c.h | 55 +
> drivers/net/fm10k/switch/fm10k_regs.h | 2302
> +++++++++++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_sbus.c | 291 ++++
> drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
> 9 files changed, 3367 insertions(+)
> create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
>
> diff --git a/config/common_linux b/config/common_linux
> index 8168106..6e62b99 100644
> --- a/config/common_linux
> +++ b/config/common_linux
> @@ -66,3 +66,8 @@ CONFIG_RTE_LIBRTE_HINIC_PMD=y
> # Hisilicon HNS3 PMD driver
> #
> CONFIG_RTE_LIBRTE_HNS3_PMD=y
> +
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
@ 2020-04-02 8:55 ` Wang, Xiao W
2020-04-09 6:24 ` Xiaojun Liu
0 siblings, 1 reply; 50+ messages in thread
From: Wang, Xiao W @ 2020-04-02 8:55 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi,
Applying: net/fm10k: add flow interface and switch management
.git/rebase-apply/patch:915: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Besides, I see that you use two empty lines to delimit two functions in some places, please align to 1 empty line.
Please understand upstreaming may mean burden of maintenance, so let's make the big patch in better shape. 😊
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 4/5] net/fm10k: add flow interface and switch management
>
> Add flow interface to support offload flow into HW.
> It supports parse vlan and parse mpls, all these
> data will be transferred to ffu data.
> Add switch management, includes initialization,
> port mapping, epl port link, LED controller, interrupt handler.
> It create 3 threads. One for interrupt handler, one for
> LED controller, one for statistics.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> drivers/net/fm10k/Makefile | 2 +
> drivers/net/fm10k/base/fm10k_type.h | 1 +
> drivers/net/fm10k/switch/fm10k_flow.c | 872 +++++++++++
> drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> drivers/net/fm10k/switch/fm10k_stats.c | 128 ++
> drivers/net/fm10k/switch/fm10k_switch.c | 2382
> +++++++++++++++++++++++++++++++
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support Xiaojun Liu
@ 2020-04-02 9:12 ` Wang, Xiao W
2020-04-09 6:32 ` Xiaojun Liu
0 siblings, 1 reply; 50+ messages in thread
From: Wang, Xiao W @ 2020-04-02 9:12 UTC (permalink / raw)
To: Xiaojun Liu, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi,
This is a feature-add patch, so you need to add release note in doc/guides/rel_notes/release_20_05.rst, that can be included as part of this 5/5 patch.
Another issue, why delete DEV_RX_OFFLOAD_RSS_HASH unconditionally? I think it would impact the existing usage.
@@ -1808,8 +2044,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_JUMBO_FRAME |
- DEV_RX_OFFLOAD_HEADER_SPLIT |
- DEV_RX_OFFLOAD_RSS_HASH);
+ DEV_RX_OFFLOAD_HEADER_SPLIT);
Also some comments on the feature description are inlined below.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput. Update fm10k.rst.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> doc/guides/nics/fm10k.rst | 5 +
> drivers/net/fm10k/fm10k_ethdev.c | 559
> ++++++++++++++++++++++++++++++++++++---
> 2 files changed, 525 insertions(+), 39 deletions(-)
>
> diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
> index 4e178c2..75f9929 100644
> --- a/doc/guides/nics/fm10k.rst
> +++ b/doc/guides/nics/fm10k.rst
> @@ -124,6 +124,11 @@ FM10000 PMD driver. The switch driver can be
> acquired from Intel support.
> Only Testpoint is validated with DPDK, the latest version that has been
> validated with DPDK is 4.1.6.
>
> +To avoid loading other switch driver from outside of DPDK, there is another
> +way to load switch driver in DPDK. If you want to use the inline switch driver,
"There is a way to load..." Since what you do is the only way to load switch driver inside DPDK.
> +you need to set ''CONFIG_RTE_FM10K_SWITCH_MANAGEMENT=y'' in the
> configure file.
Configuration file.
> +The inline switch driver supports mirror and some flow forward rules.
Please be clear on what forwarding rules are supported. Any doc reader will be confused by "some".
> +
> Support for Switch Restart
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for switch management
2020-04-02 8:41 ` Wang, Xiao W
@ 2020-04-08 3:25 ` Xiaojun Liu
0 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-04-08 3:25 UTC (permalink / raw)
To: Wang, Xiao W, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiao,
Please check the source file. It not looks align in patch file because the patch add "+" for each new line.
Best regards,
Xiaojun
-----Original Message-----
From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
Sent: Thursday, April 02, 2020 4:42 PM
To: Xiaojun Liu; Zhang, Qi Z; Kwan, Ngai-mint; Keller, Jacob E
Cc: dev@dpdk.org
Subject: RE: [PATCH v3 1/5] net/fm10k: add basic functions for switch management
Hi Xiaojun,
Just comments on the alignment issue, things like below looks not nice. If you are using vim, I recommend " set tabstop=8 " setting.
+#define FM10K_SW_BIST_CTRL_BIST_RUN_PCIE(p_) (1ULL << (p_))
+#define FM10K_SW_BIST_CTRL_BIST_RUN_EPL (1ULL << 9)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FABRIC (1ULL << 10)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_TUNNEL (1ULL << 11)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_BSM (1ULL << 12)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_CRM (1ULL << 13)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_FIBM (1ULL << 14)
+#define FM10K_SW_BIST_CTRL_BIST_RUN_SBM (1ULL << 15)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_IN (1 << 25)
+#define FM10K_SW_SCAN_DATA_IN_SHIFT_OUT (1 << 26)
+#define FM10K_SW_SCAN_DATA_IN_UPDATE_NODES (1 << 27)
Check more at http://patches.dpdk.org/patch/66973/. Please take care for next version.
I'm wondering if we could have a tool for automatically check this kind of alignment issue.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 1/5] net/fm10k: add basic functions for switch management
>
> Add I2C to control the inside LED and PHY.
> All the operations of I2C are using fm10k I2C register.
> Add SBUS to communicate with spico(micro code in serdes)
> by using fm10k SBUS register. This is like I2C operations.
> Add registers defination, which include all the registers
> will be used in the driver. Add switch management log API.
> Add switch management structures. Modify Makefile to add
> new files building. Add CONFIG_RTE_FM10K_MANAGEMENT=n
> in config/common_linux.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> config/common_linux | 5 +
> drivers/net/fm10k/Makefile | 11 +
> drivers/net/fm10k/switch/fm10k_debug.h | 19 +
> drivers/net/fm10k/switch/fm10k_i2c.c | 309 +++++
> drivers/net/fm10k/switch/fm10k_i2c.h | 55 +
> drivers/net/fm10k/switch/fm10k_regs.h | 2302
> +++++++++++++++++++++++++++++++
> drivers/net/fm10k/switch/fm10k_sbus.c | 291 ++++
> drivers/net/fm10k/switch/fm10k_sbus.h | 40 +
> drivers/net/fm10k/switch/fm10k_switch.h | 335 +++++
> 9 files changed, 3367 insertions(+)
> create mode 100644 drivers/net/fm10k/switch/fm10k_debug.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_i2c.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_regs.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.c
> create mode 100644 drivers/net/fm10k/switch/fm10k_sbus.h
> create mode 100644 drivers/net/fm10k/switch/fm10k_switch.h
>
> diff --git a/config/common_linux b/config/common_linux
> index 8168106..6e62b99 100644
> --- a/config/common_linux
> +++ b/config/common_linux
> @@ -66,3 +66,8 @@ CONFIG_RTE_LIBRTE_HINIC_PMD=y
> # Hisilicon HNS3 PMD driver
> #
> CONFIG_RTE_LIBRTE_HNS3_PMD=y
> +
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management
2020-04-02 8:55 ` Wang, Xiao W
@ 2020-04-09 6:24 ` Xiaojun Liu
0 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-04-09 6:24 UTC (permalink / raw)
To: Wang, Xiao W, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiao,
OK. I will check the two empty lines carefully and check the patch by "git am".
Best regards,
Xiaojun
-----Original Message-----
From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
Sent: Thursday, April 02, 2020 4:56 PM
To: Xiaojun Liu; Zhang, Qi Z; Kwan, Ngai-mint; Keller, Jacob E
Cc: dev@dpdk.org
Subject: RE: [PATCH v3 4/5] net/fm10k: add flow interface and switch management
Hi,
Applying: net/fm10k: add flow interface and switch management
.git/rebase-apply/patch:915: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Besides, I see that you use two empty lines to delimit two functions in some places, please align to 1 empty line.
Please understand upstreaming may mean burden of maintenance, so let's make the big patch in better shape. 😊
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 4/5] net/fm10k: add flow interface and switch management
>
> Add flow interface to support offload flow into HW.
> It supports parse vlan and parse mpls, all these
> data will be transferred to ffu data.
> Add switch management, includes initialization,
> port mapping, epl port link, LED controller, interrupt handler.
> It create 3 threads. One for interrupt handler, one for
> LED controller, one for statistics.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> drivers/net/fm10k/Makefile | 2 +
> drivers/net/fm10k/base/fm10k_type.h | 1 +
> drivers/net/fm10k/switch/fm10k_flow.c | 872 +++++++++++
> drivers/net/fm10k/switch/fm10k_flow.h | 26 +
> drivers/net/fm10k/switch/fm10k_stats.c | 128 ++
> drivers/net/fm10k/switch/fm10k_switch.c | 2382
> +++++++++++++++++++++++++++++++
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support
2020-04-02 9:12 ` Wang, Xiao W
@ 2020-04-09 6:32 ` Xiaojun Liu
0 siblings, 0 replies; 50+ messages in thread
From: Xiaojun Liu @ 2020-04-09 6:32 UTC (permalink / raw)
To: Wang, Xiao W, Zhang, Qi Z, Kwan, Ngai-mint, Keller, Jacob E; +Cc: dev
Hi Xiao,
Thank you! I add release note and remove the DEV_RX_OFFLOAD_RSS_HASH modification(It may be a mistake).
I submit the new patch again:)
Best regards,
Xiaojun
-----Original Message-----
From: Wang, Xiao W [mailto:xiao.w.wang@intel.com]
Sent: Thursday, April 02, 2020 5:12 PM
To: Xiaojun Liu; Zhang, Qi Z; Kwan, Ngai-mint; Keller, Jacob E
Cc: dev@dpdk.org
Subject: RE: [PATCH v3 5/5] net/fm10k: add switch management support
Hi,
This is a feature-add patch, so you need to add release note in doc/guides/rel_notes/release_20_05.rst, that can be included as part of this 5/5 patch.
Another issue, why delete DEV_RX_OFFLOAD_RSS_HASH unconditionally? I think it would impact the existing usage.
@@ -1808,8 +2044,7 @@ static uint64_t fm10k_get_rx_port_offloads_capa(struct rte_eth_dev *dev)
DEV_RX_OFFLOAD_UDP_CKSUM |
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_JUMBO_FRAME |
- DEV_RX_OFFLOAD_HEADER_SPLIT |
- DEV_RX_OFFLOAD_RSS_HASH);
+ DEV_RX_OFFLOAD_HEADER_SPLIT);
Also some comments on the feature description are inlined below.
Best Regards,
Xiao
> -----Original Message-----
> From: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Sent: Friday, March 20, 2020 2:59 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Kwan, Ngai-mint <ngai-mint.kwan@intel.com>; Keller,
> Jacob E <jacob.e.keller@intel.com>
> Cc: dev@dpdk.org; Xiaojun Liu <xiaojun.liu@silicom.co.il>
> Subject: [PATCH v3 5/5] net/fm10k: add switch management support
>
> Split dev init to 2 parts.
> First only register the port in switch
> management; second init hook will be
> called after all the pf are registered
> and switch initialization. It will finish
> dev init. Also add switch interrupt support.
> Add fm10k_mirror_rule_set/fm10k_mirror_rule_reset
> to support mirror operation. Add fm10k_dev_filter_ctrl
> to support flow operation.
> Add dpdk port and pf mapping, so
> the dpdk port can map to a specific pf
> and 1 dpdk port can map to 2 pf to get
> total 100G throughput. Update fm10k.rst.
>
> Signed-off-by: Xiaojun Liu <xiaojun.liu@silicom.co.il>
> ---
> doc/guides/nics/fm10k.rst | 5 +
> drivers/net/fm10k/fm10k_ethdev.c | 559
> ++++++++++++++++++++++++++++++++++++---
> 2 files changed, 525 insertions(+), 39 deletions(-)
>
> diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
> index 4e178c2..75f9929 100644
> --- a/doc/guides/nics/fm10k.rst
> +++ b/doc/guides/nics/fm10k.rst
> @@ -124,6 +124,11 @@ FM10000 PMD driver. The switch driver can be
> acquired from Intel support.
> Only Testpoint is validated with DPDK, the latest version that has been
> validated with DPDK is 4.1.6.
>
> +To avoid loading other switch driver from outside of DPDK, there is another
> +way to load switch driver in DPDK. If you want to use the inline switch driver,
"There is a way to load..." Since what you do is the only way to load switch driver inside DPDK.
> +you need to set ''CONFIG_RTE_FM10K_SWITCH_MANAGEMENT=y'' in the
> configure file.
Configuration file.
> +The inline switch driver supports mirror and some flow forward rules.
Please be clear on what forwarding rules are supported. Any doc reader will be confused by "some".
> +
> Support for Switch Restart
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
[...]
^ permalink raw reply [flat|nested] 50+ messages in thread
end of thread, other threads:[~2020-04-09 6:33 UTC | newest]
Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-11 9:51 [dpdk-dev] [PATCH v2 0/7] support switch management Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 1/7] net/fm10k: add i2c sbus registers definition Xiaojun Liu
2019-12-11 15:48 ` Jerin Jacob
2019-12-12 9:35 ` Xiaojun Liu
2019-12-12 17:12 ` Jerin Jacob
2019-12-13 2:44 ` Xiaojun Liu
2019-12-16 4:54 ` Jerin Jacob
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 0/5] support switch management Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 0/5] support fm10k " Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-03-16 6:37 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
2020-03-16 6:48 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
2020-03-16 7:04 ` Wang, Xiao W
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-02-28 8:38 ` [dpdk-dev] [PATCH v1 5/5] net/fm10k: add switch management support Xiaojun Liu
2020-03-16 7:34 ` Wang, Xiao W
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 0/5] support fm10k switch management Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 1/5] net/fm10k: add basic functions for " Xiaojun Liu
2020-04-02 8:41 ` Wang, Xiao W
2020-04-08 3:25 ` Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-04-02 8:55 ` Wang, Xiao W
2020-04-09 6:24 ` Xiaojun Liu
2020-03-20 6:58 ` [dpdk-dev] [PATCH v3 5/5] net/fm10k: add switch management support Xiaojun Liu
2020-04-02 9:12 ` Wang, Xiao W
2020-04-09 6:32 ` Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 2/5] net/fm10k: add epl serdes and port control functions Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 3/5] net/fm10k: add ffu and statistics and config file functions Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 4/5] net/fm10k: add flow interface and switch management Xiaojun Liu
2020-02-20 13:59 ` [dpdk-dev] [PATCH v2 5/5] net/fm10k: add switch management support Xiaojun Liu
2020-02-25 11:28 ` Wang, Xiao W
2020-02-25 12:56 ` Xiaojun Liu
2020-03-05 1:23 ` Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 2/7] net/fm10k: add some modules of port Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 3/7] net/fm10k: add config ffu statistics support Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 4/7] net/fm10k: add flow and switch management Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 5/7] net/fm10k: add switch initialization Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 6/7] net/fm10k: add mirror and filter ctrl Xiaojun Liu
2019-12-11 9:52 ` [dpdk-dev] [PATCH v2 7/7] net/fm10k: add dpdk port mapping Xiaojun Liu
2020-01-21 2:53 ` [dpdk-dev] [PATCH v2 0/7] support switch management Wang, Xiao W
2020-01-21 6:15 ` Xiaojun Liu
2020-02-11 10:31 ` Wang, Xiao W
2020-02-14 2:46 ` Xiaojun Liu
[not found] ` <ORIGINAL-RELEASE-1581835643311863404-DB7PR04MB5196A5418792DFB1F96DB8B7BD150@DB7PR04MB5196.eurprd04.prod.outlook.com>
2020-02-19 5:58 ` Xiaojun Liu
2020-02-19 10:56 ` Wang, Xiao W
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).